Skip to main content

Machine learning engineer II, AWS Inference Optimization

Posted today

amazonSanta Clara (CA)

SENIORITY

Lead

Apply (opens in a new tab)

About the role

Overview: In this role you will advance inference optimization for large language models, partnering with a fast-paced, cross-disciplinary team of researchers and engineers. You will own components of inference software and performance improvements across stacks like vLLM, SGLang, and GenAI services, shaping production-ready solutions for SageMaker, Bedrock, and Hyperpod. You will distill requirements into practical implementations that balance price and performance, delivering impactful results to customers. Compensation / Benefits sign-on payments restricted stock units (RSUs) health insurance (medical, dental, vision) 401(k) matching paid time off parential leave
Responsibilities: Develop model servers and inference software across large-scale systems Implement CUDA kernels in vLLM and SGLang for performance gains Build and benchmark GenAI optimization services and tooling Modify mid-size applications or services to support inference improvements Contribute to production-ready solutions for SageMaker, Bedrock, and Hyperpod
Key requirements: 3+ years of non-internship software development experience 2+ years of non-internship design or architecture experience for scalable systems 1+ years of software development engineer or related occupation experience 1+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software using C#, C++, Java, or Perl Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field Knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques cross-functional collaboration problem solving effective communication CUDA kernel development vLLM SGLang

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this