Jobs / Val***
Senior DataOps Engineer — Data Platform Reliability
Val*** · London, ENG, United Kingdom
London, ENG, United KingdomFull timeHybrid
Remuneration
Equity, employer pension contributions, private health insurance, gym memberships, and company retreats and meetups
Location
London, ENG, United Kingdom
Visa sponsorship
Sponsors visa
Job summary
This role ensures data services and processing workloads are reproducible to deploy, observable in production, and safe to recover across customer-controlled environments. It builds CI/CD and GitOps workflows, develops orchestration and replay tooling, monitors pipeline reliability, and operates data services while improving incident response and recovery. The work uses Kubernetes, Linux, Python, SQL, and workflow orchestration systems such as Airflow, Dagster, or Prefect.
Benefits
EquityEmployer pension contributionsPrivate health insuranceGym membershipsCompany retreats and meetups
Qualifications
- Experience operating production data pipelines or platforms and owning reliability and incident resolution.
- Practical Kubernetes, container, and Linux skills, including troubleshooting workloads, networking, resource limits, and persistent storage.
- Experience with automated delivery workflows using Git, CI/CD, and declarative configuration; familiarity with Helm, GitOps, or equivalent approaches.
- Python and shell scripting skills, plus SQL for investigating processing failures and database performance.
- Experience with workflow orchestration systems such as Airflow, Dagster, Prefect, or equivalent.
- Understanding of reliable processing patterns, including idempotency, bounded retries, duplicate delivery, incremental processing, and safe replay.
- Experience implementing metrics, logs, dashboards, and alerting against service and data reliability targets.
- Practical experience with stateful systems, controlled migrations, backup and recovery, secrets management, and least-privilege access.
Responsibilities
- Build and maintain CI/CD and GitOps workflows for data applications, pipelines, configuration, and database changes, including validation, promotion, and rollback.
- Develop reusable tooling for scheduling, dependency management, retries, checkpoints, replay, and backfills.
- Monitor pipeline freshness, completeness, processing latency, failures, and resource usage; configure actionable alerts.
- Operate workload-owned data services and coordinate with platform owners for shared database, messaging, and storage services.
- Implement and test recovery procedures for backup restoration, interrupted processing, and failed upgrades.
- Integrate data workloads with identity, secrets, and network controls while preserving enclave isolation and authorized access.
- Investigate production incidents, improve runbooks, and automate recurring operational work.
Skills
AirflowArgo CDBashGitGrafanaHelmKubernetesLinuxOpenShiftOpenTelemetryPostgreSQLPrometheusPythonTerraformDocker
Languages
PythonSQLShell
Industry
Artificial intelligenceSecurity
Relocation
No