Skip to main content

Machine Learning Engineer – Distributed Training & Research Infrastructure

Posted today

fintal partnersBrooklyn (NY)

SENIORITY

Senior

Apply (opens in a new tab)

About the role

We’re partnering with a leading global trading firm to hire a Machine Learning Engineer to build the systems their researchers use to train models at scale. You’ll own the infrastructure that turns a research idea into a trained, tested model, and you’ll be measured on how fast and how cheaply that happens.
What you’ll do:
  • Build and scale distributed training pipelines for large models and complex, high-volume datasets
  • Design the parallelism strategy, data loading and checkpointing for multi-node GPU training
  • Automate the research loop with quantitative researchers: experiment orchestration, hyperparameter tuning and model retraining
  • Work with HPC specialists to raise training throughput and GPU utilization, and to cut compute cost
  • Extend the internals of PyTorch and other open-source ML tools when off-the-shelf isn’t fast enough
  • Evaluate third-party tools and decide when to build versus buy
  • What you’ll bring 3+ years in machine learning, focused on training systems
Hands-on experience scaling training across many GPUs and nodes with PyTorch, JAX or TensorFlow (e.g. FSDP, DeepSpeed, Megatron, DDP)A strong grasp of ML fundamentals, including transformer and attention internals, beyond high-level APIsProduction-quality Python, with testing discipline and ownership of the systems you ship A habit of measuring before optimizing: profiling, benchmarking and finding the real bottleneck Clear communication, so you can explain your technical decisions to researchers and adapt when the evidence says toNice to have Experience with HPC schedulers (Slurm) and cloud or orchestration platforms (Kubernetes, Ray) Contributions to open-source ML or distributed-systems projects Familiarity with NCCL, collective communication or interconnect tuning No trading experience is required. What matters is depth in large-scale training systems and the ability to move fast among very smart people.

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this