# Senior ML Systems Engineer, Frameworks & Tooling

Hiring organization: [Cohere](https://career.thegoodapps.co/organizations/cohere)

Canonical page: https://career.thegoodapps.co/jobs/307114a4-5487-405e-a0a7-17c851214313

Listed on Cohere's own careers site. Applications go to them directly.

- Employment type: full time
- Seniority: Senior
- Location: New York, NY
- Remote: yes
- Salary: 250000 – 535000 CAD per year

## Summary

Cohere seeks a senior engineer to design and maintain the training framework for frontier-scale language models, focusing on distributed systems, HPC infrastructure, and the tooling that connects research to thousands of GPUs. This role suits someone with deep experience in large-scale distributed training who enjoys working across the full ML systems stack to solve performance and reliability challenges.

_Our summary, not Cohere's wording._

## Skills named

CUDA, Docker, JAX, Kubernetes, PyTorch, V-Ray

## Required

- Strong engineering experience in large-scale distributed training or HPC systems
- Deep familiarity with JAX internals, distributed training libraries, or custom kernels/fused ops
- Experience with multi-node cluster orchestration (Slurm, Ray, Kubernetes, or similar)
- Ability to debug performance issues across CUDA/NCCL, networking, IO, and data pipelines
- Experience with containerized environments (Docker, Singularity/Apptainer)
- Track record of building tools that increase developer velocity for ML teams
- Sound judgment balancing performance vs complexity and research velocity vs maintainability
- Strong collaboration skills across infrastructure, research, and deployment teams

## Nice to have

- Experience training LLMs or other large transformer architectures
- Contributions to ML frameworks (PyTorch, JAX, DeepSpeed, Megatron, xFormers)
- Familiarity with evaluation and serving frameworks (vLLM, TensorRT-LLM, custom KV caches)
- Experience with data pipeline optimization, sharded datasets, or caching strategies
- Background in performance engineering, profiling, or low-level systems
- Publication record at top-tier venues (NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)

Apply on Cohere's site: https://jobs.ashbyhq.com/cohere/c99e61c9-ed92-426d-9711-188dfc0f729f/application