Overview:
As a Member of Technical Staff on the Model Serving team, you help build, deploy, and operate Cohere’s AI platform that delivers large language models through easy API access. You’ll ship scalable, low-latency NLP services and ensure high availability in production. You’ll collaborate with multiple teams and tailor deployments to customer needs. This role combines cutting-edge ML infrastructure with customer-facing impact, enabling widespread AI adoption. You will work in a fast-paced environment to push the boundaries of scalable model serving. Compensation / Benefits open and inclusive culture health and dental benefits with mental health budget 6 weeks of vacation parental leave top-up (up to 6 months) remote-flexible with global offices and stipend co-working stipend
Responsibilities:
Develop, deploy and operate the AI platform delivering Cohere's LLMs via API endpoints Design and deploy optimized NLP models to production with low latency, high throughput, and high availability Collaborate with cross-functional teams to support, troubleshoot, and scale mission-critical systems Interface with customers to create customized deployments meeting specific requirements Investigate and resolve complex infra issues in distributed, Linux-based environments Manage compute/storage/network resources and costs across multi-cloud/hybrid settings
Key requirements:
5+ years of engineering experience running production infrastructure at large scale Experience designing large, highly available distributed systems with Kubernetes and GPU workloads Experience with Kubernetes dev and production coding and support Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving Experience in complex Linux-based computing environments Experience in compute/storage/network resource and cost management Strong collaboration and troubleshooting skills for mission-critical systems Grit and adaptability to solve evolving technical challenges Familiarity with accelerators (GPUs/TPUs) and their impact on latency and throughput Strong understanding or working experience with distributed systems Experience in Golang, C++ or other high-performance server languages collaboration troubleshooting adaptability Kubernetes distributed systems cloud platforms (GCP, Azure, AWS, OCI)