Senior ML Data Infrastructure Engineer

Job summary

The Senior ML Data Infrastructure Engineer role focuses on building the ML data infrastructure platform to enhance data preparation, feature engineering, and dataset management for machine learning. Key responsibilities include designing scalable data processing pipelines, maintaining feature stores, and developing data quality monitoring frameworks, utilizing technologies such as GCP, BigQuery, Spark, and Python.

Benefits

Qualifications

• Strong expertise in GCP's data and ML infrastructure
• Deep expertise in data processing frameworks such as Spark, Beam, and Flink
• Experience with feature stores (Feast, Tecton) and data versioning tools
• Proficiency in Python and SQL, along with experience in data quality and testing frameworks and pipeline orchestration tools such as Airflow or Dagster

Responsibilities

• Design and implement scalable data processing pipelines for ML training and validation
• Build and maintain feature stores with support for both batch and real-time features
• Develop data quality monitoring, validation, and testing frameworks
• Create systems for dataset versioning, lineage tracking, and reproducibility
• Partner with data scientists to optimize data preparation workflows

Skills

Relocation