Data Manager - Remote

Overview

Allata is a fast-growing technology strategy and data development consulting firm delivering scalable solutions to our enterprise clients. Our mission is to inspire our clients to achieve their most strategic goals through uncompromised delivery, active listening, and personal accountability. We are a group who thrive in fast-paced environments, working on complex problems, continually learning, and working alongside colleagues to be better together.

In Short

• Provide technical leadership in designing, building, and maintaining reusable data products using Data Bricks, Python, and SQL.
• Lead the migration of existing data pipelines to modern frameworks and ensure scalability and efficiency.
• Establish and enforce data quality standards, leveraging automated testing and reporting to ensure data accuracy, consistency, and compliance.
• Develop and oversee the data infrastructure, pipeline architecture, and integration solutions while actively contributing to hands-on implementation.
• Collaborate with business and technical stakeholders to define requirements and ensure data solutions align with business goals.
• Build and maintain scalable, efficient data processing pipelines and solutions for data-driven applications.
• Develop comprehensive documentation of data engineering processes and provide mentorship to junior team members.
• Monitor and ensure adherence to data security, privacy regulations, and compliance standards.
• Troubleshoot and resolve complex data-related challenges and incidents in a timely manner.
• Stay at the forefront of emerging trends and technologies in data engineering and advocate for their integration when relevant.

Requirements

• Proven expertise in DataBricks, SQL, and Python for processing and managing large data volumes.
• Strong experience in designing and migrating existing Databricks workspaces to unity catalog workspaces.
• Familiarity with batch and streaming data processing techniques.
• Familiarity with architectural best practices for building data lakes.
• Knowledge of SSIS/DacPacs.
• Demonstrated experience applying DevOps principles (e.g., Git, CI/CD) to data engineering projects, Pulumi in Typescript.
• Experience with Azure Functions.
• Knowledge of ASP.Net and .Net Core.
• Knowledge of observability and monitoring frameworks for data solutions.
• Experience with Databricks Unity Catalog, i.e. configuring and managing data governance and access controls in a Lakehouse environment.

Benefits

• Demonstrated ability to lead technical discussions and guide teams in solving complex data challenges.
• Strong collaboration and stakeholder management skills in distributed teams.
• Proactive problem-solving attitude with excellent organizational and analytical capabilities.
• Fluent English communication skills, both written and verbal.
• Familiarity with Agile methodologies and SDLC practices.