Staff Software Engineer, Data Ingestion - Remote

Overview

The Staff Software Engineer, Data Ingestion will be a critical individual contributor responsible for designing collection strategies, developing, and maintaining robust and scalable data pipelines. This role is at the heart of our data ecosystem, delivering new analytical software solutions to access timely, accurate, and complete data for insights, products, and operational efficiency.

In Short

• Design, develop, and maintain high-performance data ingestion pipelines using Python.
• Integrate with diverse data sources including databases, APIs, and cloud storage.
• Implement data transformation and cleansing logic during ingestion.
• Monitor and troubleshoot data ingestion pipelines.
• Collaborate with database engineers to optimize data models.
• Evaluate and propose new technologies for ingestion efficiency.
• Develop self-healing mechanisms for data pipelines.
• Define SLAs and SLOs for data quality.
• Participate in on-call rotation for critical issues.

Requirements

• 6+ years of experience in software development.
• Extensive experience in Python for production-grade applications.
• Proven experience in data collection from various sources.
• Strong understanding of distributed systems and scalability.
• Experience with major cloud providers like AWS or GCP.
• Solid understanding of relational databases.
• Experience with monitoring tools like Prometheus and Grafana.
• Proficiency with Git.
• Experience with Docker and Kubernetes is a plus.
• Experience with streaming technologies like Kafka is a plus.

Benefits

• Opportunity to work on cutting-edge technologies.
• Collaborative and innovative work environment.
• Professional development opportunities.
• Competitive salary and benefits package.
• Flexible work hours.