Senior Data Engineer - Remote

Overview

The company is looking for an experienced Data Engineer with strong expertise in Python and SQL to build and optimize data pipelines, integrate various data sources, and ensure efficient data processing and storage solutions.

In Short

• Design, develop, and maintain scalable data pipelines and ETL processes.
• Work with AWS services, including Lambda, S3, DynamoDB, RDS, Athena, and Secrets Manager.
• Implement and optimize distributed processing systems using PySpark/Spark and Databricks.
• Develop data workflows using Prefect.
• Manage and automate infrastructure with DevOps tools such as Jenkins, Terraform, and Rundeck.
• Implement distributed messaging solutions using SQS and SNS.
• Utilize boto3, troposphere, and other AWS Python libraries for automation and deployment.
• Ensure data quality, governance, and security best practices.
• Collaborate with data scientists, analysts, and engineers to support analytical and operational use cases.

Requirements

• Strong experience in Python and SQL for data processing and analysis.
• Hands-on experience with AWS services, including Lambda, S3, DynamoDB, RDS, Athena, Secrets Manager, and SQS.
• Experience with big data technologies such as Spark, PySpark, and Databricks.
• Familiarity with AWS Python libraries like boto3 and troposphere for automation and deployment.
• Knowledge of DevOps and infrastructure automation tools such as Jenkins, Terraform, and Rundeck.
• Familiarity with distributed messaging systems like SQS and SNS.
• Strong problem-solving and troubleshooting skills.
• Ability to work effectively in an Agile/Scrum environment.

Benefits

• 100% Remote Work: Enjoy the freedom to work from the location that helps you thrive.
• Highly Competitive USD Pay: Earn an excellent, market-leading compensation in USD.
• Paid Time Off: Our paid time off policies ensure you have the chance to unwind and recharge.
• Work with Autonomy: Enjoy the freedom to manage your time as long as the work gets done.
• Work with Top American Companies: Grow your expertise working on innovative, high-impact projects.