Jobs / CAPITOLE CONSULTING
Hibrido BCN - Senior Data Platform Engineer
CAPITOLE CONSULTING · Barcelona, CT, Spain
Barcelona, CT, SpainExp: 6+ yrsHybrid
Remuneration
Not specified
Location
Barcelona, CT, Spain
Visa sponsorship
Not specified
Job summary
The Senior Data Platform Engineer evolves and operates an Azure-based data platform and Lakehouse, providing reliable foundations for data product teams. The role leads migration from Azure Data Factory to Apache Airflow, manages Azure resources and Terraform infrastructure, and operates Databricks and Unity Catalog. It also advances CI/CD, governance, observability, security, and cost efficiency while contributing technical leadership and production support.
Benefits
1,200 annual professional development budgetFully covered private health insuranceMeal, transport, and childcare vouchersWellhub subscriptionBrand discountsRegular feedback sessionsTeam-building eventsSports teams and technical communities
Qualifications
- At least 6 years of data engineering experience, including significant platform or infrastructure work
- Advanced Python and SQL skills, including PySpark and Delta Lake semantics
- Production experience designing, deploying, operating, and upgrading Apache Airflow
- Hands-on Azure resource management experience, including Key Vault, managed identities, service principals, access controls, and Azure Data Factory
- Experience with Terraform, Azure DevOps, CI/CD, Databricks, and Unity Catalog
- Understanding of Lakehouse, Medallion, and metadata-driven architectures
- Experience with observability, monitoring, alerting, and incident management
- Strong problem-solving, analytical, communication, and collaboration skills
- Experience leading an Airflow implementation or migration, building reusable orchestration frameworks, or operating Airflow on Kubernetes is desirable
Responsibilities
- Own and evolve the Apache Airflow platform, including environments, deployments, upgrades, configuration, and DAG standards
- Lead migration from Azure Data Factory to Airflow and develop metadata-driven orchestration frameworks
- Manage Azure resources and Terraform infrastructure, including storage, networking, secrets, and identities
- Manage Azure RBAC, Entra ID groups, Unity Catalog permissions, and access reviews
- Own Azure DevOps repositories, CI/CD pipelines, branch policies, and multi-environment deployments
- Operate Databricks and Unity Catalog, including compute and job policies, ingestion patterns, governance, and optimization
- Improve data catalog, data quality, lineage, observability, and platform cost efficiency
- Contribute to architecture decisions, engineering standards, code reviews, and developer tooling
- Support production operations, resolve incidents, lead root cause analysis, and maintain documentation and runbooks
Skills
AirflowAKSAzureAzure DevOpsAzure Key VaultDatabricksKubernetesPythonTerraformVaultIAMS3Spark
Languages
PythonSQL
Industry
Data engineeringCloud computing
Relocation
No