Overview:
In this role you will advance inference optimization for large language models, partnering with a fast-paced, cross-disciplinary team of researchers and engineers. You will own components of inference software and performance improvements across stacks like vLLM, SGLang, and GenAI services, shaping production-ready solutions for SageMaker, Bedrock, and Hyperpod. You will distill requirements into practical implementations that balance price and performance, delivering impactful results to customers. Compensation / Benefits sign-on payments restricted stock units (RSUs) health insurance (medical, dental, vision) 401(k) matching paid time off parential leave
Responsibilities:
Develop model servers and inference software across large-scale systems Implement CUDA kernels in vLLM and SGLang for performance gains Build and benchmark GenAI optimization services and tooling Modify mid-size applications or services to support inference improvements Contribute to production-ready solutions for SageMaker, Bedrock, and Hyperpod
Key requirements:
3+ years of non-internship software development experience 2+ years of non-internship design or architecture experience for scalable systems 1+ years of software development engineer or related occupation experience 1+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software using C#, C++, Java, or Perl Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field Knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques cross-functional collaboration problem solving effective communication CUDA kernel development vLLM SGLang