Skip to main content

Senior/Staff Software Engineer (Back-End), Inference

Posted today

cohereBrooklyn (NY)

SENIORITY

Lead

Apply (opens in a new tab)

About the role

Overview: In this role you will evolve the Model Vault backend to support enterprise-scale deployments with strong security and reliability. You will shape architecture across identity, deployments, billing, and observability to deliver scalable, secure model inference. You’ll build and operate control-plane services that provision, configure, and manage model deployments across clouds. You will tackle complex backend and security challenges where standard SaaS patterns fall short, driving end-to-end delivery from architecture to production rollout. This is an opportunity to influence how Cohere serves large enterprises with trusted AI at scale. Compensation / Benefits weekly lunch stipend full health and dental benefits RRSP matching, 401K, Pension 6 weeks of paid vacation home office stipend remote-friendly with coworking support
Responsibilities: Evolve and scale backend systems powering Model Vault, including authentication/authorization, deployment management, billing, metering, and customer lifecycle Lead major architectural initiatives across identity, payments, usage systems, platform scalability, and other core product foundations Build highly reliable control-plane services for provisioning, configuring, scaling, and operating model deployments across clouds and hardware platforms Design APIs and distributed systems with guarantees around correctness, consistency, isolation, failure recovery, idempotency, and observability Incorporate security and privacy into platform architecture (tenant isolation, least-privilege, auditability, secrets management, encryption, confidential computing) Solve complex problems spanning backend services, cloud infrastructure, security, and AI inference beyond conventional SaaS patterns Drive projects end-to-end from architecture and threat modeling through implementation, rollout, production operations, and iteration based on customer workloads
Key requirements: Strong backend engineer with fundamentals in distributed systems, databases, networking, operating systems, and security Proficiency in production systems using Go, Python, Java, C++, Rust, or similar languages Experience designing and operating production systems with emphasis on correctness, availability, security, and performance at scale Ability to define architecture for ambiguous, high-impact problems and deliver reliable production systems Solid systems-design instincts: concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, graceful degradation Awareness of security boundaries: authentication, authorization, identity, cryptography, secrets, tenant isolation, security risks Comfort across backend services, APIs, databases, distributed systems, and cloud infrastructure Willingness to dive into unfamiliar codebases and debug cross-boundary issues Attention to production engineering details: instrumentation, capacity planning, migrations, backward compatibility, incident response, safe rollouts Track record delivering technically difficult projects that become reliable for engineers and customers ownership mindset ability to drive cross-functional initiatives strong analytical and problem-solving abilities Go Python Java

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this