Senior AI Platform Engineer LLM Gateway

Job summary

Industrialize a corporate LLM Gateway that provides secure, governed, observable access to foundation models for AI applications and agents. Implement and evolve the platform, integrate corporate identity and secrets management, configure access policies and usage controls, and ensure resilience and observability. Track consumption and costs by application, project, and model, and automate deployments using Infrastructure as Code and CI/CD.

Qualifications

• At least 6 years of experience in platform engineering, cloud engineering, API management, or distributed systems integration.
• At least 1 year of experience integrating AI Gateways.
• Experience with AWS identity, secrets, and networking services.
• Experience with Python and Terraform, CloudFormation, or CDK.
• Knowledge of API security using OAuth2, OIDC, and JWT.
• Production experience with generative AI APIs and high concurrency.
• Experience with observability and service operations, including SLI/SLOs.

Responsibilities

• Implement and evolve the LLM Gateway in line with the platform architecture and standards.
• Integrate corporate identity, access controls, and secrets management.
• Configure access policies, rate limits, quotas, guardrails, and traceability.
• Ensure service resilience and observability through metrics, logs, traces, and alerts.
• Measure consumption and costs by application, project, and model.
• Automate deployments using Infrastructure as Code and CI/CD.

Skills

Work schedule

Industry

Relocation