Data Pipeline Infrastructure Engineer - Remote

Overview

This role involves maintaining and improving the Data Pipeline infrastructure and supporting the migration to containerized services.

In Short

• Maintain and improve the Data Pipeline infrastructure consisting of Confluent Kafka, Kubernetes clusters, Cloud Composer, CloudSQL, and BigQuery.
• Introduce Terraform to automate the management of Data Pipeline infrastructure.
• Support the migration of the Data Pipeline to golden-path containerized services.
• Manage cloud hosted VMs in Google Cloud Platform.
• Automate repetitive manual processes.
• Maintain up to date VMs.
• Document technical architecture and user features.
• Educate engineers on tool usage and possible process improvements.
• Assist in defining the team’s roadmap.
• Work closely with Engineers to gather requirements.
• Create and monitor statistics to observe tooling health and functionality.
• Work and collaborate with distributed teams across the world.

Requirements

• At least 5 years experience as an Infrastructure or Software Engineer.
• Completed either a comprehensive software engineering boot camp, CS, CE, SW, MIS or equivalent degree, or has relevant work experience.
• Knowledge of configuration management systems such as Puppet, Chef, or Ansible (we use Puppet).
• Experienced with Linux systems administration, including solid scripting skills (Ruby) and Bash.
• A solid understanding of modern programming and design principles.
• Ability to write tested, maintainable code.
• A good communicator who works well with geographically distributed teams.
• Excited to explore and engage in new languages, tools, and technologies.
• Experienced in leading a project initiative.
• Comfortable working on several projects.
• Desire to work in a collaborative, open environment on an Agile team as a contributor and mentor.
• Highly proactive and results-oriented with excellent critical thinking skills.

Benefits

• Familiarity with data pipeline concepts and technologies (bonus points).