SME - AWS IAC, Terraform,Python

Job summary

The Senior Platform Engineer role involves designing, developing, and supporting scalable data pipelines and distributed systems utilizing Big Data technologies and AWS Cloud. Responsibilities include leading data platform migration initiatives, managing ETL/ELT processes with tools like SSIS and Pentaho, and implementing AWS services such as EMR, S3, and Redshift while collaborating with cross-functional teams.

Responsibilities

• Design, develop, maintain and support scalable data pipelines using Big Data and AWS technologies
• Lead and support data platform migration initiatives (On-Prem to AWS Cloud)
• Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar
• Implement and manage AWS services such as EMR, S3, EC2, Redshift, Glue, and Airflow
• Build and optimize data workflows and orchestration pipelines using Airflow
• Work with real-time streaming technologies such as Kafka and Spark Streaming
• Perform data ingestion, transformation, and validation from multiple data sources
• Optimize SQL queries for performance and scalability
• Monitor system performance, troubleshoot issues, and ensure system reliability
• Collaborate with cross-functional teams including developers, architects, and infrastructure teams

Skills

Relocation