# Engineering Manager, LLM Inference & Deployment at Scale

Hiring organization: [NVIDIA](https://career.thegoodapps.co/organizations/nvidia)

Canonical page: https://career.thegoodapps.co/jobs/dcba84b4-18c9-411d-af1c-285218505d80

Listed on NVIDIA's own careers site. Applications go to them directly.

- Employment type: full time
- Seniority: Senior
- Location: Santa Clara, CA
- Salary: 224000 – 431250 USD per year

## Summary

Lead a team building an agentic platform to monitor, debug, and optimize large-scale generative AI models in production. This role suits experienced engineering managers who understand deep learning systems, observability platforms, and distributed inference infrastructure.

_Our summary, not NVIDIA's wording._

## Skills named

Agentic systems, ClickHouse, Elastic Stack (ELK), Grafana, OpenTelemetry, Prometheus

## Required

- BSc, MS, or PhD in Computer Science, Computer Engineering, or equivalent
- 8+ years software engineering experience
- 3+ years engineering management or technical leadership
- Led teams building large-scale distributed systems, observability, ML infrastructure, or production AI systems
- Understanding of LLM/VLM inference systems and production deployment patterns
- Experience with logs, metrics, traces, profiling, alerting, dashboards, or debugging workflows
- Strong programming and performance analysis skills
- Cross-organization collaboration and communication

## Nice to have

- GPU performance analysis background
- Distributed inference or model serving optimization experience
- Reliability engineering background
- Built observability or telemetry platforms for AI, ML, cloud, or distributed infrastructure
- Experience with OpenTelemetry, Prometheus, Grafana, Jaeger, ClickHouse, or Elastic
- Built agentic systems reasoning over logs, traces, performance data, or operational workflows
- Hands-on production GenAI serving experience with metrics like TTFT, TPOT, throughput, KV cache pressure, and cost per token

Apply on NVIDIA's site: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Manager--Deep-Learning-Algorithms_JR2020646