TA
[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering
Together AISan Francisco, CA · full time
$160,000 – $280,000
Listed on Together AI’s own careers site. You apply with them directly — we never stand between you and the employer.
What this role is
Build the Kubernetes control plane and state machine that provisions and manages a GPU inference fleet, creating a declarative API so inference teams request resources without understanding the underlying complexity. You'll own the scheduling, defragmentation, and self-healing systems that keep GPU utilization high and the platform reliable in production.
Our summary, not Together AI’s wording. The full posting is on their site.
Skills this role names
Log in to see which of these are already on your profile.
What they ask for
Required
- Strong software engineering background in Go, Python, Rust, or similar language
- Experience with durable workflow orchestration tools such as Temporal or Cadence
- Experience building software control planes or orchestration systems that model and reconcile state over time
- Experience with event-driven systems and message queues or pub/sub platforms
- Product mindset with experience building internal platforms or APIs consumed by other engineering teams
Nice to have
- Exposure to bare-metal provisioning (PXE, iPXE, Redfish, IPMI, BMC)
- Networking fundamentals (VLANs, BGP, fabric design)
- GPU or accelerator infrastructure experience
- Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand, RoCE)
- Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization
- Systems programming in Rust or Go