MP
$500,000 – $850,000
Listed on Menlo Ventures Portfolio’s own careers site. You apply with them directly — we never stand between you and the employer.
What this role is
This role involves designing and building the measurement systems that quantify what Claude, an advanced language model, can actually do. You'll create evaluation frameworks, maintain the infrastructure that runs them at scale during training, and work with researchers to turn fuzzy notions of capability into precise, actionable metrics.
Our summary, not Menlo Ventures Portfolio’s wording. The full posting is on their site.
Skills this role names
Log in to see which of these are already on your profile.
What they ask for
Required
- Strong Python programming skills for production or research code
- Experience building or operating distributed systems, data pipelines, or infrastructure at scale
- Clear written and verbal communication of technical results to non-specialists
- Ability to support production systems on-call during training runs
- Bachelor's degree or equivalent professional experience
Nice to have
- Hands-on experience with large language models like Claude
- Data visualization and dashboard design experience
- Track record developing evaluation metrics for language models
- Observability, monitoring, or experiment-tracking systems experience
- Statistics and experimental design background
- Large-scale dataset sourcing, curation, and processing experience
- ML training infrastructure experience
- Flexibility across team boundaries
- Enjoys pair programming