Databricks Platform Engineer (m/w/d)

Job summary

Union Investment is seeking a Databricks Platform Engineer to design, build, and optimize their enterprise-wide data platform on Databricks.

Qualifications

• Several years of experience in software development and system operation with modern tech stacks and CI/CD pipelines on Microsoft Azure
• Proven knowledge in building and maintaining reusable Infrastructure-as-Code (IaC) templates and components (e.g., Terraform)
• Strong scripting skills (e.g., Python, Bash) are advantageous
• Practical experience with Databricks, Delta Lake, and Spark

Responsibilities

• Manage Databricks Platform ensuring availability, security, and scalability
• Provision and configure Databricks workspaces using DAB, Terraform CLI, and SDK
• Manage workspace settings such as clusters, libraries, compute policies, and access controls
• Define and maintain catalogs, schemas, and tables across workspaces with Unity Catalog
• Utilize Infrastructure-as-Code (IaC) tools like Terraform and Terragrunt for infrastructure provisioning and configuration
• Implement, operate, and improve CI/CD pipelines using GitLab CI/CD for efficient software delivery

Skills

Degrees

Relocation