# Staff + Senior Software Engineer, Inference Deployment

Hiring organization: [Menlo Ventures Portfolio](https://career.thegoodapps.co/organizations/menlo-ventures-portfolio)

Canonical page: https://career.thegoodapps.co/jobs/59f22f15-3cea-4819-a9bb-58ab967892f6

Listed on Menlo Ventures Portfolio's own careers site. Applications go to them directly.

- Employment type: full time
- Seniority: Senior
- Location: New York, NY
- Salary: 320000 – 485000 USD per year

## Summary

This role leads the design and operation of distributed inference systems that serve Claude to millions of users, handling request routing, load balancing, and fleet orchestration across diverse AI accelerators. It suits experienced distributed systems engineers who want to work on high-performance infrastructure challenges at scale while supporting both production services and cutting-edge ML research.

_Our summary, not Menlo Ventures Portfolio's wording._

## Skills named

Amazon Web Services (AWS), Kubernetes, Load Balancing, Microsoft Azure, Python, Rust

## Required

- Significant software engineering experience with distributed systems
- Results-oriented approach with bias towards flexibility and impact
- Willingness to work outside job description as needed
- Interest in learning about machine learning systems and infrastructure
- Drive for technical excellence
- Concern for societal impact of AI work

## Nice to have

- Experience with high-performance, large-scale distributed systems
- Experience deploying machine learning systems at scale
- Experience with load balancing, request routing, or traffic management
- Knowledge of LLM inference optimization, batching, and caching
- Kubernetes and cloud infrastructure experience
- Python or Rust proficiency

Apply on Menlo Ventures Portfolio's site: https://job-boards.greenhouse.io/anthropic/jobs/5285557008