At Lunavi, Lead Data Engineers collaborate to create modern data estates and robust business intelligence reporting dashboards. We are seeking an individual with excellent soft skills who can lead a team of developers as a tech lead. This individual must be versed in Agile methodologies and have experience with development in Azure.

In Short

Lead data engagements including defining architecture, creating databases structures and diagrams, optimizing speed and resiliency, and intelligent data strategies
Implement and manage data solutions using Microsoft Fabric, Databricks and other data platforms, ensuring optimal performance and scalability
Implement machine learning models for automated data classification and tagging, automated data quality validation, predictive insights and intelligent anomaly detection in data streams
Create and deploy resources using Infrastructure as Code and DevOps pipelines
Design, develop, and maintain ETL pipelines and notebooks to ensure efficient data flow and integration across various systems and incorporate predictive analytics and automated insights
Apply architecture frameworks and methodologies (Medallion, Lambda, data mesh, etc.) to organize data into layers for efficient processing and analysis
Design and implement resilient disaster recovery and high availability strategies that optimize system performance, minimize downtime, and balance operational efficiency with cost-effectiveness
Develop and maintain reports and dashboards and reports to provide actionable insights and key performance indicators to stakeholders using PowerBI, PowerApps and other data visualization platforms
Communicate effectively and translate complex technical concepts to technical and non-technical stakeholders and within cross-functional internal and client teams
Provide leadership and mentoring for developers, data engineers and data analysts on your project teams

Requirements

5+ years of experience working with enterprise data warehouses/lakehouses with a recent focus on modern cloud native data architecture and development
4+ years of experience working with a diverse range of data platforms including Fabric, Databricks, Microsoft SQL, Oracle, Postgres, or other data platforms
4+ years of experience with PowerBI Pro or Premium datasets and developing strong visualizations
Provide leadership on implementation of engineering practices such as automated CI/CD processes, ETL pipelines, notebooks and other modern data transformation processes and modern data architecture patterns
Proficiency in programming languages commonly used in data analytics and AI/ML projects (e.g., Python, R, SQL)
Experience with AI and ML frameworks, tools, and libraries such as TensorFlow, PyTorch, and scikit-learn
Proficiency in data governance and security strategies including row level security, access controls using Purview, Unity Catalog, Active Directory, etc
4+ years of experience in customer-facing leadership roles
3+ years experience providing technical leadership on Agile data projects
Desire to learn, investigate, and provide guidance on new technology in a rapidly changing environment
The desire to bring out the best of the people working on the project to make a positive, shared impact

Benefits

Certifications in Data Engineering, Data Governance, and AI/ML technologies (Azure AI, Google Cloud AI, AWS Machine Learning)
Experience with advanced ML ops and MLflow implementations
Background in developing and tuning custom machine learning models for data engineering optimization