Senior Site Reliability Engineer - AI Compute

Job summary

The Site Reliability Engineer operates and improves large-scale Kubernetes compute platforms supporting product, data, and AI/ML workloads across heterogeneous CPU and GPU resources. The role covers production incident response, cluster and node lifecycle management, workload scheduling, capacity and performance optimization, and SLOs, observability, and automation. The platform includes Kubernetes control planes, container runtimes, GPU clusters, and distributed AI/ML workloads.

Qualifications

• At least 3 years of experience in Site Reliability Engineering, Production Engineering, Systems Engineering, or a similar infrastructure role.
• Hands-on experience operating Kubernetes in production, including troubleshooting clusters, nodes, pods, controllers, and workload scheduling.
• Experience managing CPU and GPU compute resources, including capacity, allocation, isolation, utilization, and failure recovery.
• Strong knowledge of Linux systems, container runtimes, cgroups, namespaces, and networking fundamentals.
• Proficiency in Go, Python, Bash, or another programming language used for automation and infrastructure tooling.
• Preferred experience with large multi-cluster or multi-region Kubernetes fleets, GPU clusters, and AI infrastructure.
• Preferred knowledge of Kubernetes architecture, including the API server, etcd, scheduler, controller manager, kubelet, admission control, and custom controllers or operators.
• Preferred experience with distributed AI/ML training or inference workloads and GPU scheduling and troubleshooting.
• Preferred experience with infrastructure-as-code, configuration management, and Kubernetes deployment tooling.

Responsibilities

• Own reliability of large-scale Kubernetes and GPU clusters.
• Lead incident response, root-cause analysis, and corrective actions across control planes, nodes, container runtimes, scheduling, and host dependencies.
• Manage CPU and GPU cluster provisioning, upgrades, scaling, configuration changes, repairs, replacements, and decommissioning.
• Improve placement, utilization, isolation, and availability for online, batch, data, and AI/ML workloads.
• Plan compute capacity, forecast resource needs, and tune performance across clusters and failure domains.
• Define SLOs and error budgets for compute and AI infrastructure services.
• Improve monitoring, alerting, runbooks, production readiness, and failure recovery.
• Build automation and remediation tooling to reduce operational toil and improve deployment safety.

Skills

Degrees

Work schedule

Industry

Relocation