Sr SRE & Automation Engineer (Customer Facing)

Job summary

Own end-to-end reliability for a customer-facing GPU cloud serving mission-critical AI training, fine-tuning, and inference workloads. Operate Kubernetes clusters at scale, manage tenant onboarding and isolation, define service-level objectives, and lead incident response and recovery. Build observability and automation for GPU scheduling, bare-metal provisioning, and fault remediation using technologies including NVIDIA GPU Operator, Prometheus, Grafana, Terraform, Helm, and GitOps.

Qualifications

• At least 5 years of SRE or cloud operations experience, including 2 or more years operating GPU workloads at scale.
• Deep knowledge of Kubernetes operations and GPU workload management, including GPU Operator, device plugins, MIG, time-slicing, and GPU scheduling.
• Experience with topology-aware scheduling and GPU-specific resource management.
• Hands-on experience building multi-tenant cloud platforms with strong isolation guarantees.
• Experience operating customer-facing cloud services, defining SLAs and SLOs, and handling tenant incidents and communications.
• Experience with bare-metal provisioning and lifecycle automation, such as Ironic, MAAS, or custom solutions.
• Proficiency with Terraform, Helm, and GitOps workflows such as ArgoCD or Flux.
• Strong programming skills in Go or Python for automation or operator development.
• Experience with SRE practices including error budgets, incident management, and capacity planning.
• Experience with Prometheus, Grafana, and large-scale alerting.

Responsibilities

• Own availability, job completion, provisioning latency, and tenant experience for the customer-facing GPU cloud.
• Operate production Kubernetes clusters optimized for GPU workloads at scale.
• Manage tenant onboarding, quotas, isolation, offboarding, and resource reclamation.
• Define and publish customer-facing SLIs, SLOs, and SLAs for availability, job completion, provisioning latency, and API availability.
• Lead incident response, customer updates, runbook automation, escalation, and post-incident reviews.
• Automate bare-metal provisioning, tenant handoff, lifecycle management, and reclamation.
• Build tenant-aware monitoring, alerting, dashboards, and self-service observability.
• Use infrastructure-as-code and GitOps to manage GPU clusters.
• Automate GPU node failure detection, draining, and workload rescheduling.

Skills

Languages

Industry

Relocation