Ai Distilation Expert
rb labsMillbrae (CA)
About the role
About the Role:
We’re building a real-time generative AI system where latency is the core challenge. We’re looking for amodel distillation specialistto help us shrink large models into fast, deployment-ready versions without sacrificing quality.
This is a hands-on role: you’ll design and run distillation experiments, evaluate quality/latency tradeoffs, and ship the results to production GPU infrastructure.
What You’ll Do:
Design and run knowledge distillation pipelines– teacher/student setups for large neural models (LLMs and/or generative vision models)
Optimize for latency– reduce inference cost via distillation, quantization, pruning, and mixed precision
Evaluate rigorously– build empirical quality-vs-latency comparisons and A/B test variants
Own training loops– write and debug full PyTorch training code, not just run existing scripts
Ship to production– deploy distilled models on live GPU servers and measure real-world performance
Must Have 2 years hands-on PyTorch, including training/fine-tuning models from scratch – not just serving pretrained ones
Proven experience with model distillation– you’ve actually distilled a model and measured the results
Strong grasp ofmodel compression techniques : quantization, pruning, knowledge distillation, LoRA/parameter-efficient methods
Solid understanding ofloss designfor distillation (KL divergence, feature/logit matching, perceptual losses)
Comfortable reading and extendingdense, lightly-documented research code
Strong PlusExperience distillingautoregressive LLMs or speech models
Experience withGANs, diffusion, or flow-matchingmodels
GPU inference optimization: torch.compile, CUDA memory management, multi-GPU setups
Familiarity with streaming/real-time ML systems
Working StyleWe operate like a research lab: empirical, fast iteration, incomplete docs, self-directed testing. You’ll need to be comfortable designing your own experiments and pushing for more engineering structure as you go.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.