Sr Staff/Principal DevOps Engineer, Machine Learning Infrastructure and Performance

Job summary

This role builds and operates infrastructure and workflows supporting machine-learning development and performance measurement, including local server racks, on-site devices, and Linux self-hosted GitHub Actions runners. Responsibilities include creating CI/CD pipelines and reproducible Docker environments, managing Git LFS model storage, orchestrating traceable hardware measurements, and improving performance-data ingestion and analysis with Python, FastAPI, and PostgreSQL.

Qualifications

• Strong experience with Linux, CI/CD, Docker, and Python.
• Familiarity with self-hosted GitHub Actions runners, systemd, NFS, and resource monitoring.
• Familiarity with GitHub Actions reusable workflows, Docker, and release automation; Jenkins experience is a plus.
• Familiarity with Git LFS, shared caches, and large-model storage such as ONNX.
• Familiarity with pytest, pytest-xdist, integration testing, and performance reporting.
• Familiarity with ONNX and PyTorch; QAIRT SDK and Android device execution or profiling with adb are a plus.
• Familiarity with REST APIs, FastAPI, PostgreSQL, or similar technologies.
• Familiarity with batch scheduling, such as IBM Spectrum LSF, or cloud infrastructure for horizontally scaling automation.
• Related bachelor's degree and 8+ years, master's degree and 7+ years, or PhD and 6+ years of hardware, software, systems engineering, or related experience.

Responsibilities

• Maintain local server racks and on-site devices.
• Operate and improve Linux self-hosted GitHub Actions runners, including capacity, scheduling, storage, monitoring, recovery, access, and incident response.
• Build GitHub Actions pipelines and reproducible Docker environments for builds, tests, model benchmarks, artifacts, and releases.
• Maintain the Git LFS-backed model zoo and shared caches.
• Develop Python workflow orchestration to make hardware measurements traceable, repeatable, and actionable.
• Maintain performance-data ingestion and analysis services and clients using FastAPI and PostgreSQL.
• Improve test integration, regression detection, reporting, triage, and release promotion.
• Plan infrastructure scaling by isolating workloads, improving cache and artifact lifecycles, and evaluating cloud or batch execution.
• Turn project-specific tooling into supported, reusable platform components.

Skills

Degrees

Industry

Relocation