Data Engineer

XPT Software Australia Pty Ltd Contract

Sydney, Australia Posted on 10/30/2026

• XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
  
• XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
  
• We have 120+technocrats in Australia working at our clientlocations.
  
• XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
  
• We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
  

Job Description Role Overview

We are looking for a Senior Data Engineerwith strong hands-on experience in Azure Databricks and the Microsoft Azuredata ecosystem. The role will design, build, optimise, and support scalabledata pipelines and lakehouse solutions that enable analytics, reporting, datascience, and enterprise decision-making.

What You'll Work On

• Design,build, and optimise scalable batch and streaming data pipelines using AzureDatabricks, PySpark, Python, SQL, and Delta Lake.
• Developreusable ingestion, transformation, validation, and publishing frameworks forlarge-scale enterprise datasets.
• Implementmedallion or layered lakehouse patterns across raw, curated, andconsumption-ready data zones.
• IntegrateAzure Databricks with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and related Azure services.
• Developproduction-grade Databricks notebooks, jobs, workflows, libraries, and clusterconfigurations.
• Createcomplex SQL and PySpark transformations, data-quality checks, reconciliationlogic, and error-handling routines.
• OptimiseSpark workloads, storage layouts, partitioning, file sizes, caching, and queryperformance while controlling platform cost.
• ImplementDelta Lake capabilities such as ACID transactions, schema enforcement, schemaevolution, time travel, and merge operations.
• Buildautomated CI/CD pipelines for code promotion, configuration management,testing, and deployment across environments.
• Applydata security and governance controls using role-based access, secretsmanagement, data masking, auditability, and controlled access patterns.
• Monitordata pipelines, troubleshoot failures, perform root-cause analysis, and deliversustainable production fixes.
• Partnerwith Architects, Business Analysts, Data Analysts, Product Owners, and businessstakeholders to convert requirements into reliable data products.
• Providetechnical leadership, code reviews, engineering standards, and mentoring toother data engineers.

KeySkills and Experience

• 8+years of experience in data engineering, data integration, ETL/ELT, orlarge-scale data platform delivery.
• Stronghands-on experience with Azure Databricks, including notebooks, jobs orworkflows, cluster configuration, debugging, and performance tuning.
• Advancedproficiency in Python, PySpark, and SQL for production-grade data engineering.
• Strongexperience with Delta Lake and lakehouse architecture, including reliableincremental processing and data-quality controls.
• Practicalexperience with Azure Data Lake Storage Gen2 and Azure Data Factory.
• Experienceimplementing CI/CD using Azure DevOps, Git, branching strategies, automatedtesting, and environment-based deployment controls.
• Strongunderstanding of dimensional modelling, data warehousing, data lake design,metadata, lineage, partitioning, and file formats such as Parquet.
• Experiencewith orchestration, monitoring, alerting, operational support, incidentanalysis, and service-level expectations.
• Understandingof Azure identity, Key Vault, managed identities, networking, privateendpoints, and secure data-platform patterns.
• Strongcommunication, documentation, problem-solving, code-review, and stakeholdercollaboration skills.

Good to Have

• Experiencewith Microsoft Purview, Unity Catalog, Azure Synapse Analytics, Event Hubs, orStructured Streaming.
• Knowledgeof infrastructure as code using Terraform or Bicep.
• Experiencewith dbt, Airflow, Kafka, Power BI, or machine-learning data pipelines.
• Experiencein telecommunications or another high-volume enterprise environment.