Scala Spark Application Developer - Remote

Overview

We're looking for a talented Scala Spark Application Developer to join our early stage startup team and drive the core development of our platform's dataplane, focusing on building a sophisticated product platform.

In Short

• Design and implement the core dataplane architecture for our Spark-based analytics engine.
• Develop sophisticated statistical algorithms and inductive learning models for automated data quality inference.
• Build high-performance, concurrent data processing pipelines using Scala and Spark.
• Implement asynchronous processing patterns for real-time anomaly detection.
• Collaborate with CTO & customers to embed statistical models into the Spark execution engine.
• Optimize query performance across distributed computing clusters.
• Design advanced SQL generation for complex analytical workloads.
• Build robust error handling and fault tolerance mechanisms.
• Establish best practices for functional programming and mathematical computation accuracy.
• Mentor team members on Scala and advanced Spark optimization techniques.

Requirements

• 3+ years of experience with Scala development and distributed systems.
• 2+ years of hands-on Apache Spark development experience.
• Strong background in statistics and mathematical analysis.
• Advanced SQL skills with experience in query optimization.
• Proficiency in asynchronous and concurrent programming patterns in Scala.
• Deep understanding of functional programming concepts.
• Experience with large-scale data processing and ETL pipelines.
• Familiarity with multiple datastores (PostgreSQL, Snowflake, etc.).
• Proficiency with Git and collaborative development workflows.

Benefits

• Competitive salary and equity.
• Unlimited vacation policy.
• Comprehensive medical, dental, and vision insurance.
• Work with a passionate and dedicated team.
• Fun environment with like-minded individuals.