AI Infrastructure Engineer

Job summary

The AI Infrastructure Engineer role involves designing, building, and operating an AWS-based, GPU-accelerated inference and streaming serving platform, focusing on low latency and high concurrency. Responsibilities include optimizing real-time inference systems, supporting the research team's training environment, and enhancing ML/LLMOps capabilities using technologies such as vLLM, TensorRT, and EKS.

Qualifications

• 3+ years of experience building and operating AI/ML infrastructure or distributed systems
• Hands-on experience with AWS, particularly EKS, EC2 GPU workloads, ECR, S3, IAM/KMS
• Experience with at least one inference stack, such as vLLM, SGLang, or TensorRT-LLM
• Experience operating ML/LLM systems in production, including model serving and performance benchmarking
• Familiarity with GPU infrastructure and serving stacks like vLLM or Triton
• Understanding of LLM Ops practices, including evaluation and versioning

Responsibilities

• Design, deploy, and operate low-latency streaming inference systems on AWS
• Support and evolve the research team’s AWS-based training environment
• Build model and artifact registries, lineage and versioning, automated evaluation gates, deployment pipelines
• Partner with researchers and device teams for model optimization and deployment workflows
• Own capacity planning, production readiness, incident response, and secure operation of the AI platform

Skills

Relocation