Data Architect - Remote

Overview

We are seeking a Senior Data Engineer with hands-on experience in big data processing, data quality, and distributed querying. You will be responsible for designing and building robust, scalable data pipelines using modern tools such as Databricks, Apache Spark, PySpark, Deequ, and Trino. You’ll play a key role in enabling reliable, fast, and clean data delivery to support analytics, reporting, and data science use cases.

In Short

• Design, develop, and optimize large-scale data pipelines using Apache Spark and Databricks.
• Write scalable PySpark code to process structured and semi-structured data.
• Use Trino to query data across various sources in a federated manner.
• Implement data validation and quality checks using Deequ.
• Collaborate with data scientists, analysts, and other engineers to ensure high-quality data delivery.
• Tune Spark jobs for performance and cost efficiency in a cloud environment.
• Contribute to building a modern data platform with a focus on automation, reliability, and scalability.

Requirements

• 15+ years of IT experience
• Databricks (highly preferred)
• PySpark

Benefits

• Exceptional benefits package.
• Flexibility to select benefits based on personal preference.
• Excellent 401k plan with employer match.
• 100% vested immediately in the 401k plan.