Senior Kafka Platform Engineer

Job summary

Charles Schwab is seeking a Senior Kafka Platform Engineer to lead the evolution of its enterprise event-streaming platform. This role involves driving enterprise-scale event streaming, Confluent Cloud adoption, platform efficiency, and AI-enabled SDLC transformation. The ideal candidate will provide technical leadership across Confluent Platform and Confluent Cloud, focusing on cloud modernization, platform resiliency, and building automation.

Benefits

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 10+ years of hands-on software engineering experience with proven technical leadership.
• 5+ years of expertise in large-scale distributed systems and mission-critical infrastructure.
• Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering.
• Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency.
• Deep expertise in Confluent Kafka (capacity planning, administration, internals, performance tuning).
• Strong experience with Kafka clusters, replication, disaster recovery, and data pipelines.
• Hands-on experience with Kubernetes, Confluent Operator, and cloud-native modernization (GCP).
• Strong development background on Linux/Unix platforms.
• Experience with data modeling and streaming technologies (Avro, Protobuf, JSON Schema).
• Strong automation mindset with hands-on expertise in Python, Ansible, and CI/CD pipelines (Jenkins, DevOps tooling) to drive operational efficiency and reduce manual effort.
• Solid foundation in security engineering (encryption, TLS, certificate and key management).
• Strong focus on observability, reliability, automation, and production readiness with proactive risk mitigation.
• Experience leading architecture and execution across cross-functional, large-scale programs.
• Strong understanding of networking, load balancing, OS fundamentals, and performance optimization.
• Experience in regulated environments (financial services preferred).
• Strong collaboration skills across engineering, architecture, infrastructure, and business teams.
• Experience working with globally distributed/offshore teams.
• Strong Agile delivery experience (Scrum/Kanban/Scaled Agile).
• Strong leadership and soft skills including strategic thinking, decision-making, mentoring/coaching, and driving engineering excellence.

Responsibilities

• Design and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale.
• Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
• Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges.
• Act as the senior technical authority for Confluent Platform and Confluent Cloud, providing architecture, design, and implementation guidance to solve complex enterprise integration challenges at scale.
• Lead solution architecture across cross-functional initiatives, ensuring alignment with cloud modernization, resiliency, security, and AI-driven engineering priorities.
• Perform deep-dive analysis of complex production issues, drive root cause identification, and implement sustainable remediation strategies with full documentation.
• Design and build automation frameworks and self-service tools to streamline Kafka operations, reduce manual effort, and improve developer productivity and platform scalability.
• Lead seamless migration of on-prem Kafka workloads to Confluent Cloud using cluster linking, schema portability, and hybrid streaming patterns, ensuring zero business disruption.
• Drive execution across Kafka, infrastructure, security, and application teams to remove dependencies, mitigate risks, and ensure on-time delivery of committed milestones.
• Contribute to enterprise architecture forums, defining and enforcing best practices for event streaming, integration patterns, and platform standardization.
• Establish best practices for testing, reliability, observability, and operational readiness, ensuring high availability and scalable platform performance.
• Identify systemic and architectural risks early, leading mitigation efforts to prevent client impact and ensure platform stability.
• Implement enterprise-grade security controls (mTLS, RBAC, encryption) and drive continuous remediation of vulnerabilities, ensuring compliance with regulatory and audit requirements.
• Interpret technical requirements to build and enhance Kafka configurations, message flows, and integration solutions supporting business-critical applications.
• Partner with distributed teams (including offshore engineering) to ensure consistent standards, efficient delivery, and 24x7 operational support across time zones.

Skills

Certifications

Degrees

Industry

Relocation