Senior Data & AI Platform Engineer Remote (m/f/d)

Job summary

Build and operate a production data platform and Microsoft Fabric lakehouse supporting executive reporting, merchant insights, finance automation, and AI-driven data products. Design scalable pipelines with Fabric, Python, and PySpark; lead migrations from legacy platforms; and improve data quality, reliability, and cost management. The role also develops AI capabilities using RAG and Azure OpenAI and delivers applied data science models such as churn or recommendations.

Qualifications

• At least 6 years of relevant data experience, including recent hands-on production data engineering or data-platform engineering.
• Ability to take data-engineering work from requirements and design through implementation, deployment, monitoring, and ongoing operation.
• Hands-on Microsoft Fabric experience, or strong Synapse, Databricks, or Delta experience and ability to develop Fabric expertise.
• Strong knowledge of lakehouse architecture and Delta engineering, including partitioning, schema evolution, file optimization, and performance.
• Expert SQL and T-SQL skills, including complex modeling and tuning.
• Production-grade Python or PySpark experience.
• Experience implementing incremental ingestion using CDC, change tracking, or watermarks, with reliable merge/upsert, replay, and idempotent processing.
• Experience with analytical and semantic modeling, ideally Power BI or Fabric semantic models, including performance, governance, and reusable metrics.
• Working knowledge of Azure SQL, ADLS or OneLake, Entra ID, managed identities, RBAC, and Key Vault.
• Track record leading a legacy-to-modern platform migration.
• Hands-on AI engineering beyond out-of-the-box Copilot.
• Delivered data science or machine learning models with measurable results.

Responsibilities

• Design, build, and operate production data solutions across Microsoft Fabric lakehouses, warehouses, notebooks, and pipelines.
• Develop tested, maintainable production pipelines using Fabric, Python, and PySpark.
• Design scalable ingestion and transformation patterns across source systems and business domains.
• Manage Fabric capacity and costs, and retire redundant legacy platform layers.
• Lead platform migrations with metric parity, stakeholder sign-off, change control, and safe decommissioning.
• Operate production systems through monitoring, alerting, incident triage, root-cause analysis, data-quality checks, lineage, and runbooks.
• Troubleshoot SQL, Spark, pipeline, and semantic-model performance or reliability issues.
• Evolve governed semantic models and reusable business metrics with analytics and business stakeholders.
• Build AI capabilities using RAG, Azure OpenAI, and agents, including prompt and version management.
• Deliver applied data science models, such as churn, recommendations, personalization, fraud, or risk models.

Skills

Travel

Industry

Relocation