Jobs / Exp***
Lead Data Platform Engineer - AWS, Spark
Exp*** · Nottingham, ENG, United Kingdom
Nottingham, ENG, United KingdomHybrid
Remuneration
Compensation package, discretionary bonus plan, pension, Bupa healthcare, sharesave scheme, annual leave, bank holidays, volunteering days, and option to purchase additional annual leave
Location
Nottingham, ENG, United Kingdom
Visa sponsorship
Sponsors visa
Job summary
Lead a technical team delivering a fraud data capability that connects fraud-related assets to a Data Analytics Platform. The role combines hands-on engineering with team leadership, guiding AWS data pipelines and data lake components, maintaining Spark-based pipelines, publishing metadata to a data catalogue, and applying DevSecOps, CI/CD, and Infrastructure as Code practices. It also includes mentoring engineers and collaborating with platform, security, compliance, and data stakeholders to deliver secure, scalable solutions.
Benefits
PensionBupa healthcareSharesave scheme25 days annual leave8 bank holidays3 volunteering daysOption to purchase additional annual leaveDiscretionary bonus plan
Qualifications
- Strong experience building data pipelines, data lakes, or data platforms.
- Strong experience with AWS services for data engineering and platform delivery.
- Strong experience with DevOps and Infrastructure as Code tools, including Terraform, CI/CD, Git, and Jenkins.
- Strong experience developing with Apache Spark or Hadoop.
- Significant programming experience with Scala or Python.
- Experience with Configuration as Code principles and API integration.
- Strong understanding of cloud security, data privacy, access control, and data governance.
- Desirable: experience with automated testing tools such as pytest or ScalaTest.
- Desirable: familiarity with SonarQube, Veracode, RESTful APIs, and microservice architectures.
- Desirable: experience with sensitive, governed, or regulated data environments and data catalogue, metadata management, data marketplace, or federated data mesh concepts.
Responsibilities
- Lead an agile technical team delivering fraud data platform solutions.
- Guide delivery of AWS data pipelines, data lake components, and producer-side platform capabilities.
- Take ownership of, maintain, and enhance pipelines and cloud components developed by the central platform team.
- Build, test, and maintain data pipelines using sources such as Snowflake and internal data platforms.
- Prepare and publish metadata for discovery through the data catalogue.
- Apply DevSecOps, CI/CD, Infrastructure as Code, and cloud-native practices.
- Mentor and coach engineers and promote collaboration and learning.
- Promote automation and AI-assisted development where appropriate.
Skills
AWSCloudFormationEventBridgeGitAWS GlueHadoopIAMJenkinsKubernetesAWS LambdaOpenShiftPythonRESTS3ScalaSnowflakeSonarQubeSparkSQSTerraform
Certifications
AWS Developer Certification or another relevant AWS certification
Relocation
No