Sr Big Data Engineer Airflow and Oozie (GCP) - Remote

Overview

We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively.

In Short

• Develop scalable and robust code for batch processing systems.
• Manage and optimize data workflows using Oozie and Airflow.
• Leverage GCP for scalable big data processing and storage solutions.
• Implement automation/DevOps best practices for CI/CD, IaC, etc.

Requirements

• Bachelor's degree in Computer Science or related field.
• Experience with managed cloud services and cloud-based batch processing systems.
• Proficiency in Oozie, Airflow, Map Reduce, Java.
• Strong programming skills with Java, Python, Pig, and SQL.
• Expertise in public cloud services, particularly GCP.
• Proficiency in the Apache Hadoop ecosystem.
• Familiarity with BigTable and Redis.
• Experience in Infrastructure and Applied DevOps principles.
• Proven experience in engineering batch processing systems at scale.

Benefits

• Competitive salary range based on location and experience.
• Variable compensation in the form of bonuses or commissions.
• Comprehensive benefits package.