Azure Data Engineer - Remote

Overview

The Data Engineer participates in the design and build of modern data products that comprise of raw data stores (data lakes) and cleansed data repositories, populated by batch or streaming data pipelines. The Data Engineer works with a team to create a robust, sustainable and flexible design and leads the technical delivery using Agile delivery frameworks like Scrum or Kanban.

In Short

• Work closely with stakeholders across departments to architect, build and deploy various data acquisition initiatives across multiple-tenants.
• Participate in all stages of data pipeline development - from early brainstorming to coding and bug fixing.
• Design, develop, deploy and maintain data services and/or pipelines to AWS.
• Develop best practices and approaches to support continuous process automation for data ingestion and data pipeline workflows.
• Perform multiple tasks simultaneously under changing requirements and deadlines.
• Prepare and present proof of concept, solution evaluation and recommendation to various stakeholders including executives.

Requirements

• 5+ years relevant professional work experience.
• Design and develop SQL or python data pipelines that power our data lake and data warehouse.
• Design and develop big data pipelines with both structured and unstructured data.
• Proficiency with Microsoft Azure data services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Databricks.
• Comfortable with modern data orchestration tools like DBT, AWS Glue, Apache NiFi and Airflow.
• Design and develop strategies to acquire data as product.
• Experience with test-driven code development practices.
• Experience with GitLab code development practices.
• Comfortable to develop infrastructure as code such as CloudFormation or Terraform.

Benefits

• Top of industry medical, dental, and vision benefits with multiple options.
• 8 weeks of Parental Leave and discounted pet insurance.
• Starting day one, every employee is bonus eligible and receives 15 days of paid vacation.
• Annual Internet Reimbursement benefit of $25 per month.
• 3 days of paid professional development every year.
• Employee wellness events and mental health coverage.