Data Engineer - Remote

Overview

As a Data Engineer at First Advantage, you will join a team that is dedicated to building innovative data analytics solutions in the Azure cloud. You will work collaboratively while also taking initiative on projects to improve processes and enhance user experiences.

In Short

• Develop reusable, metadata-driven data pipelines.
• Automate and optimize data platform processes.
• Build integrations with data sources and consumers.
• Add data transformation methods to ETL libraries.
• Write unit tests to ensure code quality.
• Develop solutions for Databricks monitoring.
• Resolve performance or quality issues in ETL.
• Collaborate with infrastructure engineering on cloud resources.
• Contribute to data platform documentation.
• Initiate improvements to data platform architecture.

Requirements

• Strong programming skills in Python/PySpark and SQL.
• Experience with Databricks Spark for data pipelines.
• Ability to handle large and complex datasets.
• Knowledge of building data transformation libraries.
• Familiarity with Databricks Delta optimization techniques.
• Experience in developing CI/CD pipelines.
• Experience with event brokers like Kafka.
• Understanding of basic networking concepts.
• Familiarity with Agile methodologies.

Benefits

• Competitive benefits package including health care and life insurance.
• Challenging projects to work on.
• Flexibility and possibility to work remotely.
• Opportunities for personal development.