Software Engineer - Remote

Overview

itD is seeking a Software Engineer to design and scale the data pipelines that power next-generation foundation models for machine-generated data, including time series, logs, and large-scale event streams. This role contributes directly to the success of model training and production systems by enabling reliable, high-performance data infrastructure at scale. The ideal candidate will bring deep experience in distributed systems and data engineering, along with a proven track record of delivering scalable, production-ready data pipelines that support machine learning workflows.

In Short

• Build and scale distributed data pipelines for large-scale time series, log data, and high-volume event streams.
• Design and maintain reliable, high-performance Spark and Python workflows to support model training datasets.
• Analyze and resolve performance bottlenecks related to latency, memory utilization, data skew, and throughput.
• Improve data quality, validation processes, and reproducibility for machine learning workloads.
• Partner with machine learning engineers and researchers to accelerate foundation model development.
• Measure and optimize application and transaction performance in production data systems.
• Collaborate cross-functionally to ensure data infrastructure aligns with evolving research and product needs.

Requirements

• 5+ years of software engineering experience.
• Strong proficiency in Python.
• Hands-on experience with Apache Spark (PySpark or Scala).
• Experience building large-scale data pipelines in distributed environments.
• Experience working with time series data, logs, or high-volume event streams.
• Strong debugging skills and experience with performance optimization in distributed systems.

Benefits

• Comprehensive medical benefits.
• 401(k) plan.
• Paid holidays.
• Opportunities for career growth.
• Dynamic environment in a culture of respect, empowerment, and recognition.