Lead Java + PySpark Developer - Remote

Overview

Incedo is seeking a skilled and experienced Java + PySpark Developer to join its data engineering team, working on scalable data pipeline development and backend microservices.

In Short

• Design, develop, and maintain scalable data processing pipelines using PySpark.
• Build and enhance Java-based microservices/APIs for data ingestion and transformation.
• Collaborate with data scientists, analysts, and architects.
• Optimize data workflows and troubleshoot performance bottlenecks.
• Participate in code reviews and provide technical leadership.
• Implement data validation and error-handling mechanisms.
• Ensure data security and compliance standards.
• 6+ years of experience in Java, 3+ years in PySpark.
• Experience with Hadoop ecosystem and SQL.
• Familiarity with CI/CD tools and Agile methodology.

Requirements

• 6+ years of hands-on experience in Java (Spring Boot preferred).
• 3+ years of experience with PySpark in distributed data processing.
• Experience with Hadoop ecosystem, Hive, HDFS, and Spark SQL.
• Solid understanding of RESTful APIs, JSON/XML, and integration patterns.
• Strong knowledge of SQL and data modeling.
• Familiarity with CI/CD tools, Git, and Agile methodology.

Benefits

• Opportunity to work on cutting-edge technology in data engineering.
• Collaborative and innovative work environment.
• Professional development and career growth opportunities.
• Competitive salary and benefits package.