# Staff Software Engineer, Inference Infrastructure

Hiring organization: [Cohere](https://career.thegoodapps.co/organizations/cohere)

Canonical page: https://career.thegoodapps.co/jobs/9c7767ea-5354-438a-ac01-1a636235709b

Listed on Cohere's own careers site. Applications go to them directly.

- Employment type: full time
- Seniority: Staff
- Location: New York, NY
- Remote: yes
- Salary: 190000 – 325000 USD per year

## Summary

Cohere seeks a Staff Software Engineer to build and operate the inference infrastructure powering their enterprise AI models, focusing on high-performance distributed systems and GPU workload orchestration. This role suits experienced infrastructure engineers who excel at designing large-scale Kubernetes deployments and optimizing systems for low-latency, high-throughput production environments.

_Our summary, not Cohere's wording._

## Skills named

Amazon Web Services (AWS), C++, Kubernetes, Linux, Microsoft Azure

## Required

- 5+ years of production infrastructure engineering at scale
- Kubernetes design and production experience
- Large distributed systems architecture
- GPU workload cluster management
- Linux-based computing environment design and troubleshooting
- Compute, storage, and network resource optimization

## Nice to have

- Experience with multiple cloud platforms (GCP, Azure, AWS, OCI)
- Hybrid or on-prem infrastructure serving
- Familiarity with GPU, TPU, or custom accelerators
- Knowledge of inference latency and throughput optimization
- Golang or C++ experience for high-performance servers

Apply on Cohere's site: https://jobs.ashbyhq.com/cohere/41f23dad-9da2-451a-bd1e-a1800437cb64/application