Platform Engineer - CI/CD Gate & Online Evaluation

Job summary

Intellias is seeking a Platform Engineer to establish automated quality controls for enterprise AI platforms. The role involves integrating evaluation workflows into deployment pipelines and collaborating with various teams to ensure AI agents meet quality standards.

Qualifications

• 4+ years platform or DevOps engineering experience.
• CI/CD pipeline integration for quality gates.
• CloudWatch dashboard and alarm design experience.

Responsibilities

• Design and implement CI/CD quality gates using AWS AgentCore Evaluation.
• Integrate evaluation execution into GitHub Actions and deployment pipelines.
• Configure and maintain on-demand evaluation workflows for quality validation.
• Implement and manage online evaluation capabilities and scheduling.
• Build CloudWatch dashboards for evaluation scores and trends.
• Configure CloudWatch alarms for quality degradation and performance issues.
• Collaborate with AI engineers on evaluation thresholds and quality standards.
• Support observability by integrating evaluation results into reporting workflows.
• Analyze evaluation outcomes and identify quality regressions.
• Develop automation for continuous validation of AI systems.
• Partner with teams to improve deployment safety and governance controls.
• Contribute to AI quality engineering practices and monitoring frameworks.

Skills

Relocation