Spark EKS Engineer (AWS Big Data Platform) - Remote

Overview

The Spark EKS Engineer will be responsible for administering and scaling multi-petabyte Spark EKS clusters and related services on AWS, focusing on performance enhancement, troubleshooting, and managing large-scale global deployments.

In Short

• Administer and scale multi-petabyte Spark EKS clusters on AWS.
• Perform provisioning, capacity planning, monitoring, and management of Spark EKS platforms.
• Enhance the performance of applications and middleware.
• Troubleshoot and resolve Spark-related issues.
• Implement and maintain Spark, Flink, and Trino infrastructure.
• Improve scalability, service reliability, capacity, and performance.
• Triage production issues in collaboration with teams.
• Conduct ongoing maintenance for global deployments.
• Write automation code for managing Big Data clusters.
• Participate in on-call rotations.

Requirements

• 5+ years of experience managing large-scale Spark production clusters.
• Strong expertise in AWS services (EMR, S3, EKS).
• Hands-on experience with Hadoop and Spark ecosystems.
• Proficiency in Python for development and automation.
• Experience with configuration management and automation tools.
• Deep understanding of Spark ecosystem and troubleshooting techniques.
• Ability to analyze logs and debug issues.
• Strong organizational skills focused on delivering results.
• Excellent communication and collaboration skills.

Benefits

• Work for a top IT service provider recognized for sustainability.
• Opportunity to work with next-generation technologies.
• Be part of a large, global team.
• Competitive salary and benefits.
• Remote work flexibility.