Data Engineer 2 - Remote

Overview

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer 2 in India.

In Short

• Design, develop, and maintain ETL/ELT pipelines to support AI/ML workflows and analytics.
• Build and optimize data ingestion and transformation logic using Python, PySpark, and SQL.
• Implement and manage DataOps pipelines with data quality checks, logging, monitoring, and automated testing.
• Schedule and orchestrate pipelines using tools like Airflow, Databricks Workflows, or Azure Data Factory.
• Optimize SQL queries and Spark jobs to ensure performance and cost efficiency.
• Collaborate with data scientists to provide clean, well-structured data for model development and deployment.
• Ensure adherence to data governance, security, and compliance policies, particularly in healthcare environments.

Requirements

• Strong experience in data engineering and ETL processes.
• Proficiency in Python, PySpark, and SQL.
• Experience with DataOps practices and tools.
• Familiarity with scheduling and orchestration tools like Airflow.
• Understanding of data governance and compliance, especially in healthcare.
• Ability to collaborate effectively with data scientists and analysts.

Benefits

• Opportunity to work in a fast-growing, innovative environment.
• Engagement with challenging problems in healthcare and financial data systems.
• Contribute to impactful AI solutions.