Senior Data Engineer - Remote

Overview

The Senior Data Engineer will architect and build scalable data pipelines and workflows, ensuring data is available for machine learning and analytics in a remote setting.

In Short

• Architect and build scalable data pipelines using Dagster.
• Design and optimize storage solutions for industrial and vision data.
• Develop robust data ingestion frameworks for live production data.
• Collaborate with ML engineers on data structuring for model training.
• Work with Kubernetes to orchestrate data processing jobs.
• Enhance CI/CD for data workflows using GitLab CI/CD.
• Maintain AWS-based data infrastructure using Terraform.
• Implement data governance best practices.
• Optimize batch and real-time processing frameworks.
• Act as a technical leader in data engineering.

Requirements

• 5+ years of experience in data engineering.
• Strong Python skills with emphasis on type safety.
• Experience with data processing frameworks like Pandas or Dask.
• Expertise in data orchestration tools, preferably Dagster.
• Experience with streaming and event-driven architectures.
• Hands-on experience with Kubernetes.
• Experience deploying infrastructure via Terraform.
• Proficiency in Cloud Services, preferably AWS.
• Strong database skills including SQL and NoSQL.
• Experience with strongly-typed ORMs.

Benefits

• Opportunity to shape the data infrastructure of a growing AI company.
• Collaborative, fast-paced environment with cutting-edge technology.
• Competitive salary and annual incentive plan.
• Opportunities for growth and leadership as the data team scales.