# Research Engineer, Model Evaluations

Hiring organization: [Menlo Ventures Portfolio](https://career.thegoodapps.co/organizations/menlo-ventures-portfolio)

Canonical page: https://career.thegoodapps.co/jobs/9574f1e0-b228-43bf-af37-5f26bc33397a

Listed on Menlo Ventures Portfolio's own careers site. Applications go to them directly.

- Employment type: full time
- Location: New York, NY
- Remote: yes
- Salary: 500000 – 850000 USD per year

## Summary

This role involves designing and building the measurement systems that quantify what Claude, an advanced language model, can actually do. You'll create evaluation frameworks, maintain the infrastructure that runs them at scale during training, and work with researchers to turn fuzzy notions of capability into precise, actionable metrics.

_Our summary, not Menlo Ventures Portfolio's wording._

## Skills named

Data Pipelines, Python, Statistics

## Required

- Strong Python programming skills for production or research code
- Experience building or operating distributed systems, data pipelines, or infrastructure at scale
- Clear written and verbal communication of technical results to non-specialists
- Ability to support production systems on-call during training runs
- Bachelor's degree or equivalent professional experience

## Nice to have

- Hands-on experience with large language models like Claude
- Data visualization and dashboard design experience
- Track record developing evaluation metrics for language models
- Observability, monitoring, or experiment-tracking systems experience
- Statistics and experimental design background
- Large-scale dataset sourcing, curation, and processing experience
- ML training infrastructure experience
- Flexibility across team boundaries
- Enjoys pair programming

Apply on Menlo Ventures Portfolio's site: https://job-boards.greenhouse.io/anthropic/jobs/5198255008