# Member of Technical Staff, ML Platform

Hiring organization: [Runway](https://career.thegoodapps.co/organizations/runway)

Canonical page: https://career.thegoodapps.co/jobs/c013c6ad-ce97-415c-826d-012a22d334c0

Listed on Runway's own careers site. Applications go to them directly.

- Seniority: Senior
- Location: Remote
- Remote: yes
- Salary: 240000 – 290000 USD per year

## Summary

This role leads the design and development of an end-to-end model evaluation platform that will measure the quality of frontier AI models across the organization. You'll work embedded with research teams to build the infrastructure, metrics, and tooling that determines which checkpoints to train, which models to ship, and how to scientifically validate improvements.

_Our summary, not Runway's wording._

## Skills named

Data Pipelines, Kubernetes, Python, PyTorch

## Required

- 5+ years building ML infrastructure or data platforms in production, including work on evaluation, experimentation, or benchmarking systems
- Strong Python and PyTorch with hands-on experience running large batch GPU workloads on Kubernetes
- Experience designing data pipelines and storage for large volumes of media or model outputs with attention to versioning and reproducibility
- Understanding of experimental statistics: paired comparisons, confidence intervals, multiple-comparison pitfalls, inter-rater agreement
- Ability to build internal tools end to end from CLI to browser
- Can lead a broad technical area by gathering requirements, setting direction, making tradeoffs, and driving roadmaps
- Familiarity with the full model development lifecycle: data, training, evaluation, serving
- Self-starter who works embedded with research teams and moves fast
- Strong systems thinking and pragmatic approach to production reliability

## Nice to have

- Track record of building evaluation suites for generative models
- Hands-on work with LLM- or VLM-as-judge pipelines
- Experience with online experimentation platforms and connecting offline metrics to product outcomes
- Prior work evaluating agents or robotics policies

Apply on Runway's site: https://jobs.ashbyhq.com/runway-ml/9a5bef1b-cd54-4258-bb71-4dc7dfded54c/application