Data Analytics Intermediate Engineer - Remote

Overview

The Data Analytics Intermediate Engineer is a hands-on technical contributor who designs, builds, and maintains scalable data pipelines and infrastructure within large enterprise data environments.

In Short

• Design and implement data ingestion, transformation, and cleansing pipelines using PySpark, SQL, and Python/Java.
• Work on structured and unstructured datasets stored in HDFS, Hive, Parquet, or cloud-based storage.
• Optimize existing data workflows and jobs for performance, scalability, and reliability.
• Support batch and streaming data processing frameworks across Big Data platforms (e.g., Hadoop, Spark, Hive, Kafka).
• Integrate and process data from multiple sources including APIs, flat files, relational databases, and cloud-native services.
• Apply data modeling, partitioning, and file format best practices for efficient storage and querying.
• Implement monitoring, logging, and alerting for production pipelines.
• Document pipeline logic, data lineage, and schema changes.
• Collaborate with data analysts, data scientists, and product owners.
• Assist in proof-of-concept efforts for new technologies and data integration strategies.

Requirements

• 2–5 years of experience in a data engineering, ETL development, or big data role.
• Strong programming experience in Python (or Java) for data manipulation and automation.
• Advanced proficiency in SQL (window functions, joins, CTEs, optimization techniques).
• Experience working with Apache Spark (PySpark) in a distributed environment.
• Hands-on with Hadoop ecosystem tools (Hive, HDFS, Oozie, etc.).
• Familiarity with Git, Jenkins, Airflow, or other CI/CD and orchestration tools.
• Exposure to cloud platforms (AWS Glue/EMR, Azure Data Factory, GCP Dataflow) is a plus.
• Knowledge of basic ML workflows is desirable but not mandatory.
• Strong verbal and written communication skills.
• Proactive in identifying data quality issues and bottlenecks.

Benefits

• Competitive employee benefits, including medical, dental & vision coverage.
• 401(k) plan.
• Life, accident, and disability insurance.
• Wellness programs.
• Paid time off packages including vacation and sick leave.