MP
Staff + Senior Software Engineer, Inference Deployment
Menlo Ventures PortfolioNew York, NY · full time · Senior
$320,000 – $485,000
Listed on Menlo Ventures Portfolio’s own careers site. You apply with them directly — we never stand between you and the employer.
What this role is
This role leads the design and operation of distributed inference systems that serve Claude to millions of users, handling request routing, load balancing, and fleet orchestration across diverse AI accelerators. It suits experienced distributed systems engineers who want to work on high-performance infrastructure challenges at scale while supporting both production services and cutting-edge ML research.
Our summary, not Menlo Ventures Portfolio’s wording. The full posting is on their site.
Skills this role names
Log in to see which of these are already on your profile.
What they ask for
Required
- Significant software engineering experience with distributed systems
- Results-oriented approach with bias towards flexibility and impact
- Willingness to work outside job description as needed
- Interest in learning about machine learning systems and infrastructure
- Drive for technical excellence
- Concern for societal impact of AI work
Nice to have
- Experience with high-performance, large-scale distributed systems
- Experience deploying machine learning systems at scale
- Experience with load balancing, request routing, or traffic management
- Knowledge of LLM inference optimization, batching, and caching
- Kubernetes and cloud infrastructure experience
- Python or Rust proficiency