Skip to main content

Senior Software Engineer - Agentic AI & Python

Posted yesterday

briefedEast Boston (MA)

SENIORITY

Lead

Apply (opens in a new tab)

About the role

At Klaviyo, we believe the future of software lies not only in tools that help people work efficiently, but in intelligent systems that can take action, learn from outcomes, and improve customer experiences over time. Serving over 167,000 customers and processing billions of consumer profiles and conversion signals, Klaviyo provides a unique environment for building state-of-the-art AI systems at scale. We are looking for a Senior Software Engineer, Agent Runtime & Customer Workflows to join the Customer Agent team—Klaviyo's AI-native conversational platform. In this role, you will design and build scalable backend systems and AI-driven product experiences that enable autonomous agents to reason, retrieve context, execute tools, and carry out reliable multi-step actions on behalf of customers.
Key Responsibilities: Agentic Infrastructure & APIs: Design and build reliable, high-throughput backend services and APIs powering AI-driven customer workflows, tool use, orchestration, and context retrieval. Applied AI & Hallucination Mitigation: Solve complex applied AI challenges across RAG, context grounding, model selection, hallucination reduction, and hybrid/deterministic logic vs. LLM execution. Evaluation Frameworks: Develop automated and human-in-the-loop evaluation strategies, evaluator calibration frameworks, regression suites, and production signal metrics. Production Guardrails & Reliability: Implement production guardrails, fallbacks, retry logic, observability, and failure-handling mechanisms for agentic systems. Distributed Workflows: Build asynchronous and distributed processing workflows (e.g., event queues, task systems) capable of processing high-volume AI workloads. Customer & Cross-Functional Collaboration: Partner directly with external customers, product managers, and cross-functional teams to translate feedback and workflow friction into production technical solutions. Operational Excellence: Participate in the team's on-call rotation to diagnose, mitigate, and conduct postmortems on production incidents. Experience & Education5–7 years of professional software engineering experience with a strong foundation in backend systems and distributed cloud applications. Hands-on, practical experience building and deploying generative AI or agentic AI applications into production for real users.
Technical Capabilities: Strong proficiency in Python and modern backend application frameworks (e.g., FastAPI, Django).Hands-on experience with asynchronous processing, event-driven architectures, and distributed task queues (Celery, Kafka, AWS SQS, RabbitMQ, or Redis).Deep understanding of database design, data modeling, REST/gRPC APIs, and persistence patterns. Comfort operating within cloud-native environments (AWS, Docker/containers, Kubernetes, infrastructure-as-code, CI/CD pipelines).Proven ability to evaluate tradeoffs across quality, latency, cost, reliability, and architectural complexity.
Preferred Qualifications: Experience training, fine-tuning, distilling, or adapting machine learning and language models for production workflows. Hands-on experience with reinforcement learning or feedback-driven optimization techniques. Demonstrated background engineering AI evaluation platforms (e.g., LLM-as-judge systems, benchmark suites, quality metrics platforms).Experience optimizing inference cost, latency, or throughput for AI systems operating at significant scale.

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this