Senior Data Engineer - Remote

Overview

Intuition Machines uses AI/ML to build enterprise security products. We apply our research to systems that serve hundreds of millions of people, with a team distributed around the world. You are probably familiar with our best-known product, the hCaptcha security suite. Our approach is simple: low overhead, small teams, and rapid iteration.

In Short

• Maintain, extend, and improve existing data/ML workflows, and implement new ones to handle high-velocity data.
• Provide interfaces and systems that enable ML engineers and researchers to build datasets on demand.
• Influence data storage and processing strategies.
• Collaborate with the ML team, as well as frontend and backend teams, to build out our data platform.
• Reduce time-to-deployment for dashboards and ML models.
• Establish best practices and develop pipelines that enable ML engineers and researchers to efficiently build and use datasets.
• Work with large datasets under performance constraints comparable to those at the largest companies.
• Iterate quickly, with a focus on shipping early and often, ensuring that new products or features can be deployed to millions of users.

Requirements

• Thoughtful, conscientious, and self-directed.
• Experience working with data engineering services on major cloud providers.
• Minimum of 3 years of experience in a data role involving designing and building data stores, feature engineering, and building reliable data pipelines that handle high loads.
• Proven ability to make independent decisions regarding data processing strategy and architecture.
• At least 2 years of professional software development experience in a role other than data engineering.
• Significant experience coding and developing in Python.
• Experience in building and maintaining distributed data pipelines.
• Experience working with Kafka infrastructure and applications.
• Deep understanding of SQL and NoSQL databases (preferably Clickhouse).
• Familiarity with public cloud providers (AWS or Azure).
• Experience with CI/CD and orchestration platforms: Kubernetes, containerization, and microservice design. Familiarity with distributed systems and architectures.

Benefits

• Fully remote position with flexible working hours.
• An inspiring team of colleagues spread all over the world.
• Pleasant, modern development and deployment workflows: ship early, ship often.
• High impact: lots of users, happy customers, high growth, and cutting-edge R&D.
• Flat organization, direct interaction with customer teams.