Remote Otter LogoRemoteOtter

Data Pipeline Engineer - Remote

Posted 2 weeks ago

Overview

At Near Space Labs we design, build and operate a fleet of stratospheric robots to image the earth. We launch our proprietary, balloon-based imaging robots to heights between 40,000 and 60,000 feet. From this vantage point, we capture petabytes of imagery for a variety of use cases.

We seek a proactive Data Engineer to drive the evolution of our petabyte-scale geospatial imagery pipeline. This role will be instrumental in ensuring the seamless flow of data, collaborating closely with data and software engineers. You will provide ongoing support by actively monitoring and troubleshooting data pipeline performance and reliability. Your contributions will directly impact our ability to deliver high-quality imagery to customers with optimal speed and reliability.

In Short

  • Build and maintain our proprietary data pipeline for batch geospatial data processing.
  • Monitor and support data pipeline performance and reliability.
  • Develop new data processing tasks to support evolving data requirements.
  • Design and implement data storage solutions using cloud-based storage services.
  • Develop and optimize data workflows using distributed systems.
  • Rapid generation of functional prototypes.

Requirements

  • Proficient in Python: 5+ years of experience developing robust data pipelines and applications, with a strong understanding of data structures, algorithms and geospatial data.
  • Data Pipeline Expertise: Proven experience designing, building, and maintaining scalable data pipelines using distributed processing frameworks (e.g., Airflow, Spark, etc).
  • Containerization and Kubernetes: Demonstrated ability to interact and manage containerized data applications using Docker and Kubernetes.
  • Cloud Storage and Data Warehousing: Experience working with cloud-based storage solutions (e.g., AWS S3, Google Cloud Storage, Azure Blob Storage) and data warehousing technologies.
  • Understanding of distributed systems: Knowledge of how distributed systems work and the challenges that they present.
  • Problem-Solving and Troubleshooting: Strong ability to diagnose and resolve complex data infrastructure and pipeline issues.
  • Self-Starter and Collaborative: Ability to work independently and collaboratively in a fast-paced environment, managing projects and deliverables effectively.

Benefits

  • An exciting startup culture where you will have the opportunity to play a critical role in building and scaling a one-of-a-kind technology and organization.
  • You will be part of an enthusiastic, international and motivated team of professionals who are committed to building unique technologies, being rigorous, and finding novel solutions to interesting problems.
  • A diverse and inclusive workplace where we welcome people of different backgrounds, experiences and perspectives.
  • A commitment that you will never be bored.

Similar Jobs:

N.S.L

Data Pipeline Engineer - Remote

Near Space Labs

2 weeks ago

Join Near Space Labs as a Data Pipeline Engineer to enhance our geospatial imagery pipeline and ensure data reliability.

Python
Data Pipeline
Distributed Systems
Kubernetes
USA
Full-time
Software Development
$120,000 - $220,000/year
Software Mind logo

Data Pipeline Engineer - Remote

Software Mind

5 weeks ago

Join Software Mind as a Data Pipeline Engineer to design and optimize data processing pipelines using Python.

Python
AWS
PostgreSQL
Data Processing
Argentina
Full-time
Data Analysis
Dune logo

Data Pipeline Engineer - Remote

Dune

13 weeks ago

Join Dune as a Data Pipeline Engineer to build and manage robust data pipelines for blockchain analytics.

Data Pipelines
Orchestration Tools
DBT
Prefect
USA
Full-time
Software Development
HCLTech Hungary logo

Data Pipeline Engineer - Remote

HCLTech Hungary

27 weeks ago

Join Starschema as a Data Pipeline Engineer to develop data pipelines and collaborate in a dynamic team environment.

SQL
DBT
Airflow
Snowflake
Worldwide
Full-time
Data Analysis
Motional logo

Senior Data Pipeline Engineer - Remote

Motional

11 weeks ago

Join Motional as a Senior Data Pipeline Engineer to design and implement large scale data processing pipelines.

Data Pipeline Engineering
Cloud Infrastructure
Distributed Data Processing
Spark
USA
Full-time
Software Development
$155,300 - $207,000 USD