Skip to main content

Evaluation Researcher

Posted 3 days ago

acceler8 talentMillbrae (CA)

SENIORITY

Senior

Apply (opens in a new tab)

About the role

I'm partnering with a well-funded frontier AI research startup developing low-latency speech and multimodal AI systems capable of interacting with humans in real time. They're looking for an experienced Evaluation Researcher to design and build evaluation systems that directly improve the quality of production speech and language models.
The role:
  • Design and develop evaluation frameworks for speech and text models
  • Build automated evaluation pipelines, benchmarks and quality metrics
  • Evaluate conversational AI performance, including response quality, latency and turn-taking
  • Identify model weaknesses and translate evaluation findings into measurable product improvements
  • Work closely with research and engineering teams to improve production AI systems
  • What they're looking for:
  • Strong experience building evaluations for production speech or text models
  • Proven impact on model quality and end-user experience
  • Experience with LLM evaluation, speech evaluation or conversational AIStrong Python and ML research engineering skills
  • Knowledge of modern evaluation methods, benchmarking and experimental design
  • Why consider it?
  • Join a well-backed frontier AI research lab developing next-generation conversational and multimodal AI systems.
  • You'll have direct ownership of model evaluation and play a key role in improving how intelligent agents interact with people in real time.
Location: Downtown San Francisco | Fully On-site

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this