Senior Data & AI Platform Engineer Remote

Job summary

Build and operate a production data platform and Microsoft Fabric lakehouse supporting executive reporting, merchant insights, finance automation, and AI-driven data products for an international restaurant platform. The role includes developing tested pipelines with Fabric, Python, and PySpark; migrating legacy systems; improving production reliability and data quality; and delivering AI capabilities such as RAG, Azure OpenAI, agents, and applied machine-learning models.

Qualifications

• At least 6 years of relevant data experience, including recent hands-on data engineering or data-platform engineering in production.
• Ability to take data-engineering problems from requirements and design through implementation, deployment, monitoring, and ongoing operation.
• Hands-on Microsoft Fabric experience, or strong Synapse, Databricks, or Delta experience with a clear path to Fabric.
• Strong knowledge of lakehouse architecture and Delta engineering, including partitioning, schema evolution, file optimisation, and performance.
• Expert SQL and T-SQL skills, including complex modelling and tuning.
• Production-grade Python or PySpark experience.
• Experience with incremental ingestion patterns such as CDC, change tracking, or watermarks, and reliable merge/upsert, replay, and idempotent processing.
• Experience with analytical and semantic data modelling, performance optimisation, governance, and reusable business metrics.
• Working knowledge of Azure SQL, ADLS/OneLake, Entra ID, managed identities, RBAC, and Key Vault.
• Track record of leading a legacy-to-modern migration.
• Hands-on AI engineering beyond out-of-the-box Copilot.
• Delivered data science or machine-learning models with demonstrated results.
• Strong Power BI or Fabric experience, including Direct Lake, DAX, semantic model tuning, incremental refresh, and capacity considerations, is preferred.
• Infrastructure as Code experience with Terraform, Bicep, or equivalent tooling is preferred.

Responsibilities

• Design, build, and operate production data solutions across Microsoft Fabric Lakehouse, Warehouse, notebooks, and pipelines.
• Develop tested, maintainable production pipelines using Fabric, Python, and PySpark.
• Design scalable ingestion and transformation patterns across source systems and business domains.
• Manage Fabric capacity and costs, and retire redundant legacy layers.
• Lead legacy-to-modern platform migrations with metric parity, stakeholder sign-off, change control, and safe decommissioning.
• Operate the platform through monitoring, alerting, incident triage, root-cause analysis, data-quality checks, lineage, and runbooks.
• Build AI capabilities using RAG, Azure OpenAI, and agents, including prompt and version management.
• Deliver applied data science models, such as churn, recommendations, personalisation, fraud, or risk models.
• Diagnose and resolve production issues across SQL, Spark, pipelines, and semantic models.
• Evolve governed semantic models and reusable metrics with analytics and business stakeholders.

Skills

Travel

Industry

Relocation