Data Engineer - Remote

Overview

As a Data Engineer at Definitive Healthcare, you will design and develop scalable data pipelines, integrate data from various sources, and ensure data quality and governance to support healthcare commercial intelligence.

In Short

• Build and maintain scalable data pipelines using Python, Spark, and Databricks.
• Implement data workflows and ETL processes using Apache Airflow.
• Integrate data from various sources into a unified data warehouse.
• Ensure data quality and integrity through effective data cleansing practices.
• Manage and optimize data storage solutions for high availability.
• Implement and manage Unity Catalog for metadata management.
• Optimize Spark jobs for performance and efficiency.
• Work closely with data scientists and analysts to deliver solutions.
• Implement CI/CD pipelines for data infrastructure.
• Stay updated with industry trends to improve data engineering practices.

Requirements

• Hands-on experience with Python or Scala programming.
• Strong experience with Apache Spark and Databricks.
• Experience with Apache Airflow or similar tools.
• Knowledge of data cleansing and curation techniques.
• Familiarity with Unity Catalog or metadata management tools.
• Experience with cloud platforms (AWS and GCP).
• Proficiency in CI/CD tools and practices.
• Experience with JVM tuning and Spark job performance investigation.
• Familiarity with containerization.
• Excellent problem-solving and analytical skills.

Benefits

• Industry leading products.
• Flexible and dynamic culture.
• Competitive benefits package including healthcare benefits and 401(k) match.
• Opportunities for community service through DefinitiveCares.
• Recognition as a best place to work in Massachusetts.
• Support for employee affinity groups.
• Commitment to diversity and inclusion.
• Limitless opportunity for growth.
• Collaborative work environment.
• Focus on work-life balance.