Senior Data Platform Engineer

Job summary

The Senior Data Platform Engineer role is centered on developing a production-grade platform that facilitates the deployment of AI tools and data products. Responsibilities include designing identity and access controls using Okta and Entra ID, maintaining Infrastructure-as-Code with Terraform, and enhancing CI/CD practices in GitHub Actions. Collaboration with data engineers and business stakeholders in finance and asset management is essential for success.

Qualifications

• Proven experience designing and implementing enterprise-grade federated identity, token exchange, permission propagation, SCIM provisioning, and access lifecycle management.
• Strong hands-on experience with IaC (Terraform) and production CI/CD pipelines.
• Experience hardening API gateway patterns.
• Technical comfort working across Azure, AWS, or Databricks platform ecosystems.
• Highly autonomous engineer comfortable in a lean team.

Responsibilities

• Build and maintain Infrastructure-as-Code (IaC) using Terraform across development, test, and production environments.
• Strengthen CI/CD practices in GitHub Actions, establishing repeatable deployment patterns, environment promotions, automated rollbacks, and release controls.
• Design and implement federated identity architecture across Okta, Entra ID, and ZScaler.
• Build identity, token exchange, and permission propagation models for human users and non-human AI agents.
• Implement SCIM provisioning/deprovisioning and access models with single-revoke capability and full auditability.
• Harden existing AI gateway and agent orchestration layers into secure, production-ready capabilities.
• Implement API gateway controls: authentication, routing, rate limiting, token budgets, policy-as-code, and action logging.
• Implement AI system observability, including prompt logging, output monitoring, quality scoring, and drift detection.
• Deploy vector database and retrieval infrastructure to support Document AI and RAG use cases.

Skills

Relocation