Senior Data Engineer - Databricks (3rd Party Contractor)

to Position:         Senior Data Engineer (Databricks & Cloud Data Platforms) Reporting to:  Practice Head: Data and Automation ROLE OVERVIEW  As a Senior Data Engineer, you will play a critical consulting role in designing, building, and modernising enterprise data platforms, with a strong focus on Databricks lakehouse implementations across cloud environments. You will work closely with solution architects, analysts, data scientists, and client stakeholders to deliver secure, scalable, and high-performing analytics platforms that drive real business impact. This is a hands-on technical role suited to someone passionate about solving complex data challenges and delivering best-in-class cloud-native solutions. responsibilities RESPONSIBILITIES - Design and implement scalable Databricks-based data platforms - Build robust ETL / ELT pipelines for batch and streaming workloads - Lead data migration and modernisation initiatives from legacy platforms to cloud-based lakehouse architectures - Develop high-performance data processing solutions using PySpark, Python, and SQL - Implement Delta Lake, Unity Catalog, and modern governance frameworks including Purview - Design enterprise-grade lakehouse and medallion architectures - Optimise Spark workloads for performance, scalability, and cost efficiency - Build orchestration solutions using Databricks Workflows, ADF, Airflow, or similar tools in Azure/AWS - Collaborate with technical and business stakeholders to define solution requirements - Mentor junior engineers and contribute to technical best practices within the Data & AI practice - Produce technical documentation, architecture artefacts, and reusable delivery assets IDEALLY YOU ARE: - A strong technical leader who can own complex delivery outcomes - Comfortable engaging directly with clients and stakeholders - A proactive problem-solver with excellent analytical skills - Able to clearly communicate technical concepts to both technical and non-technical audiences - Passionate about mentoring and uplifting engineering teams - Delivery-focused, quality-driven, and adaptable in consulting environments MINIMUM EXPERIENCE - 8+ years in Data Engineering projects - 3+ years of hands-on Databricks delivery experience - Deep expertise in Python and PySpark - Strong understanding of the Medallion Architecture and Lakehouse Architecture - Hands-on experience building and maintaining live Databricks AI/BI Governance Dashboards using SQL - Strong experience with Azure data services including Synapse, ADF, OneLake, Purview - Practical exposure to AWS data platforms including Glue, S3, Redshift - Proven track record delivering data migration and modernisation projects - Experience working in consulting or client-facing environments - Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience TECHNICAL REQUIREMENTS - Databricks Expertise (Essential) - Cloud Data Platform Experience (Azure Fabric, AWS) (Essential)