Data Engineer - Azure Databricks & Purview

Data Engineer | Azure Databricks & Microsoft Purview

Contract: 6 weeks

Rate: 30-32k AED Per Month

Start: Immediate

Location: Abu Dhabi, UAE

Travel: Candidates must be able to work onsite in Abu Dhabi for the first 4 weeks

Openings: 3

Right to work: Candidates must already hold the right to work in the UAE. Sponsorship is not available.

We are seeking three experienced Data Engineers to join an urgent data governance and lakehouse programme in Abu Dhabi. The team is building and operating authoritative data registers across eight business verticals, creating trusted, governed records for core subjects including employees, suppliers, government services and projects.

Each register will have a stable business key, a named owner and full governance in Microsoft Purview. You will work closely with the Data Modeller and Product/Programme Delivery lead to turn register specifications into production-ready data products.

Key Responsibilities

• Build and operate bronze, silver and gold data pipelines in Azure Databricks using PySpark, Spark SQL, Delta Lake and Unity Catalog.
• Develop gold-layer register views, including current-record and SCD Type 2 logic, deduplication, stable business keys and source-key mapping tables.
• Design, implement and test data-quality rules in Microsoft Purview, covering uniqueness, completeness and validity, then investigate and resolve failures.
• Catalogue each data register in Purview, including business definitions, named owners, Abu Dhabi data classification and lineage from Unity Catalog system tables.
• Validate and evidence end-to-end source-to-gold lineage where it is currently assumed or incomplete.
• Publish governed registers as data products through Purview Unified Catalog, partnering with each vertical's Data Steward.
• Work from register specifications covering grain, keys and minimum data fields, ensuring data products meet agreed governance and modelling standards.
• Use Git and CI/CD to deploy and maintain Databricks jobs and pipelines.

Essential Skills & Experience

• At least 4 years' Data Engineering experience, including 2+ years working hands-on with Azure Databricks.
• Strong practical experience with PySpark, Spark SQL, Delta Lake and Unity Catalog.
• Proven delivery of production Medallion Architecture data platforms across bronze, silver and gold layers.
• Advanced SQL skills, including window functions, joins, deduplication and SCD Type 2 processing.
• Experience designing, implementing and testing data-quality rules.
• Experience integrating at least one of the following source systems: Oracle Fusion HCM/ERP, SQL Server or ServiceNow.
• Strong understanding of Git-based development and CI/CD for Databricks workloads.
• Available to start immediately and able to commit to four weeks onsite in Abu Dhabi.
• Existing UAE right to work. Unfortunately, sponsorship cannot be provided.

Desirable Experience

• Master Data Management, reference data or matching records across multiple source systems.
• Exposure to UAE government data environments, UAE PDPL or Abu Dhabi data-classification standards.
• Ability to read Arabic, particularly for names, code lists and source-data interpretation.

This role suits a hands-on Data Engineer who can rapidly build reliable, governed datasets and work effectively within a high-profile data transformation programme.