Lead Data Engineer

Canberra, Australia

HybridMost of the section is located at Agriculture House in Canberra CBD. Standard expectation is three (3) working days in the office each week. Flexible arrangements may be considered. In-person attendance at important Agile ceremonies and other key team/section events is required.

Job Description

Our federal client is looking for a Data Engineer to join itsEnterprise Analytics and Technology Services (EATS) section to work across itsdata and analytics platforms. We are seeking candidates with strong experiencein developing Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT)processes, and /or the development of data products and complex datavisualisations.

The role will be responsiblefor design, development and testing activities across several data movement,data transformation, and data visualisation processes within DAFF. The datamovement and transformation processes focus on the preparation of data foruseid decision making processes across the department, utilising modern cloudtechnology (Azure & Databricks) to enable operational analytics use cases.

To be successful in the role,you must have a strong work ethic including taking ownership, providingleadership, having a high level of productivity, and working with amulti-disciplinary team in an Agile development environment. You must havesuitable qualifications or training in data engineering techniques and at least5 years' experience working as a data engineer.

Requirements

Responsibilities include, but not limited to:

• Be responsive,flexible, and work collaboratively as part of an agile team.
  
• Strongrelationship building, and negotiation skills.
  
• Strong written andoral communication skills.
  
• Creating andmaintaining automated ingest and transformation patterns and frameworks.
  
• Designing,building and maintaining data ingest and transformation solutions to meetcurrent and emerging needs.
  
• Assisting projectteams achieve objectives that align with departmental, divisional, andprogram priorities.
  
• Supporting dataengineers in delivery teams, through regular quality reviews andconstructive feedback on utilising data assets to produce quality dataproducts.
  

The successful candidate willrequire experience with the following techniques and technologies:

• DataIntegration - Data Factory, SQL Server Integration Services and/orDatabricks
  
• DataStore - SQL Server and/or Data Lake Storage
  
• Development tools- DevOps, Visual Studio
  
• Data technologysolutions - sourcing (Oracle, Ingres, Azure, SQL Server), automatedingestion
  
• Data Preparation
  
• Transformationof data into formats tailored for advanced analytics and AI use cases -Parquet and/or Delta
  
• Analyseand interpret complex data sets, and to identify trends and patterns.
  
• Designprinciples to create appropriate visualisations for target audience.