Senior Data Platform Engineer

Job summary

MY.GAMES is seeking a Senior Data Platform Engineer to drive the evolution of their Data Platform and lead migration to a new on-premise stack. The role involves building, maintaining, and migrating ETL/ELT pipelines, working across a full toolchain including Python, SQL, Spark, Airflow, HDFS, Kafka, Trino, Iceberg, and dbt. This position requires designing and implementing DataOps practices, setting up observability, and automating recurring team operations.

Benefits

Qualifications

• Hands-on experience as a Data Engineer, Data Platform Engineer, DataOps Engineer, or similar role
• Proven track record building and maintaining ETL/ELT pipelines
• Strong command of Python and SQL
• Practical experience with Spark or other distributed processing frameworks
• Deep knowledge of Airflow internals: scheduler, workers, executors, DAG lifecycle, retries, sensors, pools, queues, SLAs, logs, alerts, and callbacks/listeners
• Experience running data workflows in production environments
• Ability to set up monitoring, alerting, and dashboards (e.g., Grafana/Prometheus-style observability stacks)
• Solid understanding of monitoring Airflow, Spark, HDFS, and Kafka for failures, lag, resource usage, data freshness, completeness, queue time, and scheduler health
• Experience automating engineering processes: scripts, CLIs, internal tools, deployment automation, and validation checks
• Familiarity with DWH, Data Lake, and Lakehouse architectures
• Experience working with legacy pipelines and migrating them to a modern stack
• Kubernetes proficiency at the application level: containers/images, deployments, jobs, cronjobs, configs, secrets, and resource requests/limits
• Ability to drive a technical task end-to-end, from problem scoping to a production-ready solution
• Consider reliability, observability, maintainability, and lifecycle as part of the definition of done

Responsibilities

• Drive the evolution of the Data Platform and lead migration to a new on-premise stack
• Build, maintain, and migrate ETL/ELT pipelines from legacy to new stack
• Work across the full toolchain including Python, SQL, Spark, Airflow, HDFS, Kafka, Trino, Iceberg, and dbt
• Own Airflow as a production orchestration layer, managing DAGs, deployment, retries, sensors, pools, queues, callbacks/listeners, backfills, and reruns
• Design and implement DataOps practices: monitoring, alerting, SLA/SLO tracking, runbooks, incident diagnostics, and postmortems
• Set up observability across Airflow, Spark, HDFS, Kafka, and other platform components
• Build custom listeners, exporters, checkers, and internal tooling for platform health diagnostics
• Automate recurring team operations: DAG deployment, pipeline migrations, backfill/retry/recovery flows, and pre-release validation
• Advance CI/CD and production-readiness standards for data workflows
• Contribute to Data Governance at the engineering level, including ownership, naming conventions, metadata, lineage, access patterns, auditability, and privacy-by-design
• Work with Kubernetes at the application level: updating images, configuring deployments/jobs/cronjobs, migrating services, and managing configs, secrets, and environment variables
• Help the team reduce manual operations, recurring failures, and operational noise

Skills

Industry

Relocation