Senior Data Engineer - Remote

Overview

The Senior Data Engineer drives the design, development, and operational excellence of our data platform built on Databricks and the Delta Lakehouse Architecture. This role requires deep expertise in scalable ETL/ELT, Spark optimization, and modern data governance using Unity Catalog. The Senior Data Engineer serves as a technical leader, establishing best practices for code quality, performance, and automated deployments across all environments.

In Short

• Design and implement scalable data pipelines using Delta Lake.
• Optimize large-scale Spark jobs (PySpark/SQL).
• Build resilient data scheduling via Databricks Workflows.
• Migrate data and models from relational databases to Databricks.
• Monitor data pipelines performance.
• Partner with Data Management and Full Stack development engineers.
• Support existing data pipelines to ensure business continuity.
• Stay updated with the latest trends in data engineering.
• Establish automated CI/CD pipelines.
• Perform other related duties as assigned.

Requirements

• Bachelor’s degree is required.
• 5+ years of experience in Data Engineering.
• 3+ years of hands-on experience with Databricks.
• Expertise in PySpark and high-performance SQL.
• Deep understanding of Delta Lake architecture.
• Experience with cloud platforms (AWS preferred).
• Solid experience implementing CI/CD for Databricks notebooks.
• Healthcare experience is preferred.

Benefits

• No Travel.
• #LI-Remote.