Platform Engineer (m/w/d) LLMOps/AI Infrastructure

Job summary

Cornelsen is seeking a Platform Engineer (m/w/d) for LLMOps/AI Infrastructure in Berlin. The role involves developing and operating a LiteLLM-based LLM proxy, ensuring data protection compliance, and providing support to product teams in integrating LLMs.

Qualifications

• Experience operating an LLM proxy in production
• Solid Kubernetes knowledge
• Familiarity with at least one cloud platform (Azure, AWS, or GCP)
• Familiarity with Azure OpenAI, AWS Bedrock, or similar model providers
• Consultative mindset
• Ability to communicate technical considerations clearly
• Strong documentation and knowledge transfer skills
• Experience with observability tooling, ideally Langfuse or similar
• Fluent in German and English

Responsibilities

• Develop and operate LiteLLM-based LLM proxy
• Provide reliable access to LLM providers for product teams
• Manage gateway as a production-critical service
• Define SLOs, provider failover, and rate-limiting
• Lead incident response
• Ensure data protection compliance at the gateway
• Advise product teams on model selection and integration patterns
• Create clear developer documentation and best practice examples
• Support teams in onboarding and daily platform usage
• Establish observability for LLM and agent applications

Skills

Relocation