Remote Senior Software Engineer, Go - LLM Team

Why AssemblyAIAssemblyAI builds the best-in-class Voice AI models powering the next generation of voice applications. Our models serve 600M+ inference calls monthly, process 1M+ hours of audio daily, and power 2 billion+ end-user experiences. The Voice AI space is at an inflection point; we're looking for folks truly excited to join a small team and help define the future of the industry.We are one of the most capital-efficient AI companies on the planet - with under 100 people generating roughly $500K ARR per employee, we sit among the top 5 most revenue-dense teams within the fastest-growing AI companies today. That's not an accident; it's a deliberate choice to stay lean, move fast, and give every person on the team outsized ownership and impact. With thousands of customers including Granola, Fireflies, Figure AI, and CallRail, the company has real scale - processing over 2 million hours of audio daily and handling more than 1 million API calls every day. This is a rare growth-stage opportunity where the business is proven and the trajectory is steep, but the team is still small enough that your fingerprints are on everything.If you've ever felt buried under layers of bureaucracy, starved of real ownership, or frustrated watching your work disappear into a slow-moving org, AssemblyAI is built differently. The company operates as a true meritocracy, with no heavy planning or approval processes and no gatekeeping on the tools or information you need. For anyone who genuinely cares about voice AI, not as a trend to chase, but as a technology to build, this is the place where the most interesting problems at the most interesting scale are being solved by a team small enough that you'll actually know everyone's name.We're committed to creating a space where our employees can bring their full selves to work and have equal opportunity to succeed. No matter your race, gender identity or expression, sexual orientation, religion, origin, ability, age, veteran status, if joining this mission speaks to you, we encourage you to apply!About the RoleWe're seeking an exceptional Senior Software Engineer to join our LLM team. This role is focused on building and maintaining our LLM gateway service—a unified API platform that connects customers to multiple LLM providers. You'll work on high-impact projects that directly solve customer problems, improve their AI and agentic workflows, and ensure reliable access to the best models for their use cases.This is a deeply customer-focused role. You'll work closely with our customer success team to understand customer challenges, help optimize their prompt strategies, build features that address their pain points, and ensure our service reliably delivers value. We're looking for someone who combines software engineering excellence with genuine curiosity about how customers use AI and a drive to make their lives better.As a Senior Engineer, you'll drive technical execution within the team, taking ownership of significant features and integrations while mentoring more junior engineers. You should be passionate about writing clean, maintainable code, implementing comprehensive testing strategies, and building highly reliable systems in service of solving real customer problems.This role requires close collaboration with customer success, product managers, external API providers, and other engineering stakeholders. You'll need to balance technical excellence with pragmatic delivery in a fast-paced startup environment where uptime, reliability, and customer success are critical.What You'll DoSolve Customer ProblemsPartner closely with the customer success team to understand customer use cases, challenges, and integration needsTranslate customer pain points surfaced by the CS team into technical solutions and product improvementsBuild features and tooling that directly address customer needs and improve their workflowsProvide technical guidance and expertise to the customer success team to help them support customers effectivelyDrive Technical ExecutionOwn and deliver complete features and integrations within our LLM gateway serviceBuild and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock etc.)Write clean, maintainable, well-tested code following best practicesDesign and implement scalable, fault-tolerant solutions with appropriate abstractionsProactively identify and address technical debt, reliability issues, and code quality concernsParticipate in on-call rotation to ensure service reliability and rapid incident responseElevate Engineering StandardsConduct thorough code reviews focused on maintainability, testing, reliability, and architectural concernsEnsure proper test coverage across unit, integration, and end-to-end testing levelsImprove code maintainability and extensibility through targeted refactoringContribute to runbooks, incident postmortems, customer-facing documentation, and knowledge sharingIdentify bottlenecks and propose solutions to improve team velocity and system reliabilityBuild and CollaborateWork closely with product, customer success, and engineering teams to deliver solutionsCollaborate with external API providers to troubleshoot integration issues and improve reliabilityTranslate ambiguous customer requirements into well-scoped technical workProactively communicate status, risks, and blockersMentor junior engineers on coding standards, reliability practices, and customer-centric thinkingParticipate in architectural discussions and contribute to technical decisionsWhat You'll NeedCore Requirements5+ years of professional software engineering experienceStrong proficiency in Go with demonstrated ability to write production-quality codeHands-on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering conceptsExperience building or working with API gateways, proxies, and rate limiting systemsDeep experience with prompt engineering, chain-of-thought, few-shot learning, and other LLM optimization techniquesExperience building applications with llms or ai agents with cascading architecturesSolid understanding of software engineering fundamentals:Data structures and algorithmsSystem design and architectural patternsTesting strategies (unit, integration, end-to-end)Code review practices and technical collaborationExperience with:RESTful APIs and distributed systems conceptsBuilding or integrating with third-party APIsSQL and database designVersion control (Git) and CI/CD pipelinesContainerization (Docker) and cloud infrastructureStrong communication skills and ability to collaborate effectively with cross-functional teamsTrack record of delivering high-quality, reliable software in a fast-paced, team environmentAbility to thrive in a startup environment with changing priorities and rapid iterationNice to HavesExperience with AWS services (ECS, Lambda, ElastiCache/Redis, RDS, S3, CloudWatch, API Gateway)Experience with monitoring and observability tools (Datadog, CloudWatch)Experience with request routing, load balancing, and traffic managementExperience with circuit breakers, retry logic, fallback strategies, and fault tolerance patternsUnderstanding of API design, versioning, and backward compatibilityExperience working closely with customer success or support teamsWhat We're Looking ForThis role requires someone who is:Customer-obsessed - You genuinely care about solving customer problems and making their lives easier, working through the customer success team to understand and address their needsExcellent at software fundamentals - You write code that others want to emulateQuality-focused - You care deeply about testing, documentation, and maintainabilityReliability-obsessed - You understand that uptime and performance directly impact customer trustLLM-savvy - You understand prompt engineering, model capabilities, and how to help customers get the best resultsIntegration-savvy - You've wrestled with third-party APIs and know how to build resilient integrationsCollaborative - You work well with diverse stakeholders and partner effectively with customer success teamsCommunicative - You can explain complex technical concepts clearly to teammates and cross-functional partnersGrowth-minded - You're curious, eager to learn, and want to expand into platform and infrastructure engineeringPragmatic - You balance perfection with delivery and understand trade-offsTeam-oriented - You improve not just the code, but the team's overall effectivenessProactive - You anticipate problems before they become incidents and build systems defensivelyEmpathetic - You put yourself in the customer's shoes and design solutions accordinglyWe're looking for the best person for this role - someone who can hit the ground running while growing with the team. The ideal candidate brings strong software engineering discipline, a passion for AI/LLMs, and genuine excitement about solving customer problems through reliable, thoughtfully designed infrastructure.Pay Transparency:AssemblyAI strives to recruit and retain exceptional talent from diverse backgrounds while ensuring pay equity across our team. Our salary ranges are set to be competitive for our size, stage, and industry, and reflect just one component of the full compensation, benefits, and rewards we offer.Salary determinations consider a variety of factors, including relevant experience, technical depth, skills demonstrated during the interview process, and maintaining internal equity with peers on the team. The range shared below represents a general expectation for the posted position. However, we are open to considering candidates who may fall above or below the outlined experience level—in those cases, we will communicate any adjustments to the expected salary range.The range provided applies to candidates located in the United States. For candidates outside of the U.S., compensation ranges may differ; any adjustments will be communicated throughout the interview process.Salary range: $180,000 - $240,000The expected base compensation for this role is listed above. Our total compensation package includes competitive equity grants, 100% employer-paid benefits, and the flexibility of being fully remote. 401k match up to 4% for US-based full time team members.AI to Interview:If you're selected for an interview, please review this resource to better understand how AssemblyAI approaches the use of AI in our interview process.GDPR privacy notice:Candidates from the EU should review this job applicant privacy notice before applying.Keep Exploring AssemblyAI:Speech-to-text | Streaming speech-to-text | Speech Understanding | LLM GatewayTry the PlaygroundOur $50M Series C fundraiseCheck us out on YouTube!