Databricks Solutions Architect - Remote

Overview

As a Databricks Solutions Architect at Data Intellect, you will work on a variety of impactful customer technical projects, including designing and building scalable reference architectures and productionalising customer use cases within Capital Markets.

In Short

• Define end-to-end data architectures, ensuring best practices for data governance, access control, and cost optimisation using Unity Catalog and Delta Lake.
• Optimise query performance, storage and compute efficiency for various workloads.
• Develop and deploy ETL/ELT pipelines, enabling real-time and batch data processing.
• Implement best practices for data ingestion and transformation in financial market data environments.
• Guide strategic customers in adopting Databricks Lakehouse as a unified data platform for structured and unstructured market data.
• Enhance and grow your knowledge among subject matter experts in our learning and development ecosystem.

Requirements

• Proficient in data engineering, data platforms and analytics.
• Deep experience with Python, SQL and/or Scala.
• Knowledge of two or more common Cloud ecosystems (Azure, AWS, GCP) with expertise in at least one.
• Deep experience with distributed computing with Apache Spark.
• Working knowledge CI/CD for production deployments.
• Working knowledge of MLOps.
• Familiarity with designing and deploying performant end-to-end data architectures.
• Experience with technical project delivery, managing scope and timelines.
• Able to communicate complex technical concepts to business stakeholders.
• Adaptable to evolving technologies and projects.
• At least one Databricks Certification, Data or ML Engineering.

Benefits

• Flexible working – we offer hybrid working so our people can achieve that elusive work/life balance.
• Professional development – we offer extensive training, ranging from leadership to specific technical skills.
• Progression opportunities - we run a biannual promotion process. Monthly 121s with your People Leader provides support to guide you and your career in the right direction.
• International travel opportunities – we offer the opportunity to work internationally, with teams in Belfast, London, New York, Hong Kong & Singapore.
• Healthcare cover – provider is dependent on region, UK is provided by Benenden Health, including 24/7 GP Service & Mental Health Helpline to give you peace of mind when it comes to your health.
• Generous referral scheme – we love to see referrals and referring a friend means cash for you!
• Regular social events, prizes and giveaways – our talented social committee work hard all year round to provide exciting events across all regions to promote our value of togetherness.