# Product Designer, Evals & Prompts

Hiring organization: [Menlo Ventures Portfolio](https://career.thegoodapps.co/organizations/menlo-ventures-portfolio)

Canonical page: https://career.thegoodapps.co/jobs/ad1c058f-06d5-4166-aae9-46ce1a9ebef0

Listed on Menlo Ventures Portfolio's own careers site. Applications go to them directly.

- Employment type: full time
- Location: San Francisco, CA
- Salary: 305000 – 385000 USD per year

## Summary

This role builds the evaluation systems that test whether Claude's prompts and features work as intended across product surfaces and model launches. It combines eval infrastructure, prompt engineering support, and internal tooling to help product designers systematically test and refine AI behavior without writing code.

_Our summary, not Menlo Ventures Portfolio's wording._

## Skills named

A/B Testing, Prompt Engineering, Python, Regression Testing

## Required

- Production-quality Python code
- Experience building evaluation pipelines for LLM products including graders and rubrics
- Experience building internal tools with UI for non-technical users
- Experience setting up test harnesses and sandboxing tool calls
- Experience shipping prompts or working closely with prompt engineers
- Ability to read and analyze transcripts, not just interpret scores

## Nice to have

- Worked inside a model-launch cycle
- A/B testing experience and connecting offline evals to online results
- Front-end or notebook-to-app experience with opinions on eval result visualization
- Experience converting product rubrics into training signal for model improvement
- Focus on how Claude behaves for users, beyond metric optimization

Apply on Menlo Ventures Portfolio's site: https://job-boards.greenhouse.io/anthropic/jobs/5411318008