Staff Software Engineer, Applied AI
$194,000—$267,000
Apply (opens in a new tab)Staff Software Engineer, Applied AI
Posted 3 days ago
SENIORITY
Lead
SALARY
$194,000—$267,000
About the role
- Build the agent harness for planning, tool use, memory, context compaction, interruption and recovery during long-running work.
- Integrate OpenAI and Anthropic model APIs. Select models using measured quality, latency and cost, with clear failure handling.
- Evaluate existing harnesses and build only the missing pieces. Keep the harness connected to AWS workflow execution and approval state.
- Keep authorization and sensitive writes behind application policy checks. Require evidence of the resulting state before marking work complete.
- Build evaluation datasets and replay tests for valid findings, access decisions, app setup and AI agent onboarding.
- Measure false positives, missed risks, unsafe actions, completion rate and cost. Turn production failures into regression tests.
- Test prompt injection, stale evidence, conflicting instructions and unsafe tool use. Protect tenant boundaries in retrieval and memory.
- Improve prompts, retrieval, tools and model routing. Evaluate fine-tuning or other post-training methods when results justify them and data use is approved.
- What you will bring to the role
- Strong Python engineering skills and experience shipping an LLM application that uses tools.
- Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications. For harness work, relevant experience includes Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph or a custom implementation.
- Experience integrating model APIs, structured outputs and asynchronous services. Experience with OpenAI or Anthropic is especially relevant.
- Depth in agent execution or evaluation systems, with working code and failure analysis you can explain.
- Ability to distinguish a plausible model response from a supported finding or verified action.
- Scope by level
- Staff: Lead a domain such as harnesses, evaluations or model quality across workers and implement its critical components.
- Our stack
- Our core stack is Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS and pgvector. OpenAI and Anthropic model APIs are part of the integration plan.
- We plan to add an agent harness and a dedicated analytics layer using ClickHouse or BigQuery. The harness choice is open. Experience with equivalent tools also counts.
- And extra credit for
- Open-source contributions, agent observability, security evaluations or synthetic evaluation data.
- Fine-tuning, preference data, post-training evaluation or model distillation using approved datasets.
- Identity and security workflows. Foundation-model research experience is optional. P26064#LI-Inperson #LI- BB1Below is the annual base salary range for candidates located in San Francisco Bay Area.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
