Data Engineer – Databricks - Remote

Overview

The Data Engineer will design, build, and maintain scalable data pipelines using Databricks and Apache Spark, integrating data from various sources into data lakes or data warehouses while ensuring data quality and performance.

In Short

• Design and maintain scalable data pipelines using Databricks and Apache Spark.
• Integrate data from various sources into data lakes or data warehouses.
• Implement and manage Delta Lake architecture.
• Ensure data quality, performance, and reliability.
• Collaborate with data analysts and stakeholders.
• Automate workflows and manage job scheduling.
• Maintain documentation of data workflows.
• Work on Databricks-based AI/ML solutions.

Requirements

• 3+ years in data engineering with exposure to Databricks and AI/ML.
• Proficient in Python or Scala for ETL development.
• Strong understanding of Apache Spark, Delta Lake, and Databricks SQL.
• Experience with AWS, Azure, or GCP.
• Familiarity with data lakehouse concepts and dimensional modeling.
• Comfortable using Git and pipeline automation tools.
• Strong problem-solving abilities and attention to detail.

Benefits

• Opportunity to work with leading technologies in data engineering.
• Collaborative work environment with skilled professionals.
• Potential for career growth and development.
• Access to training and certifications in relevant technologies.