Senior AI Platform Engineer – LLM Gateway

Job summary

The Senior AI Platform Engineer will implement and evolve an enterprise LLM Gateway that provides secure, governed, observable access to generative AI models for applications, services, and AI agents. The role covers API integration and security, AWS networking and IAM, resilience, observability, cost optimization, and deployment automation using Python, Terraform, and CI/CD, in collaboration with architecture, security, cloud, SRE, and development teams.

Qualifications

• At least 6 years of experience in platform engineering, cloud engineering, distributed systems integration, API management, or a related field.
• At least 1 year of demonstrable experience with AI Gateways.
• Experience implementing and operating APIs, shared platforms, or enterprise services.
• Production experience integrating inference APIs and generative AI models, including streaming, concurrency, latency, token usage, and error handling.
• Strong knowledge of API gateways, access policies, quotas, rate limiting, and observability.
• Hands-on AWS experience, particularly with IAM, secrets management, networking, observability, and infrastructure as code.
• Knowledge of cloud networking and security, including VPC, proxies, DNS, TLS, certificates, and controlled egress.
• Experience with observability, alerting, and SRE practices.
• Python development experience for APIs, integrations, middleware, or platform components.
• Experience with Terraform, CloudFormation, or CDK, CI/CD, and version-controlled configuration.
• Ability to independently diagnose and resolve complex technical incidents.

Responsibilities

• Implement and evolve LLM Gateway components to meet enterprise architecture and standards.
• Integrate applications, services, and AI agents with inference endpoints.
• Configure OAuth2, OIDC, and JWT authentication and authorization.
• Manage secrets, credentials, permissions, and least-privilege access controls.
• Implement API policies for rate limits, quotas, routing, guardrails, and traceability.
• Configure secure provider connectivity, including DNS, proxies, TLS, certificates, and controlled network egress.
• Build resilience with timeouts, retries, circuit breakers, fallbacks, and load testing.
• Implement metrics, structured logs, distributed tracing, alerts, and dashboards.
• Automate deployments, configuration, and tests with infrastructure as code and CI/CD pipelines.
• Measure and optimize costs by application, agent, project, and model.
• Collaborate with architecture, IAM, cybersecurity, networking, cloud, SRE, and development teams.

Skills

Languages

Industry

Relocation