Senior Data Engineer

Company Description

Showtime Consulting is a leading provider of Shielded Cloud Solutions. Based in Australia and New Zealand, it specialises in secure cloud deployments and extending the cloud to the intelligent edge. Our expertise spans Azure, Azure Stack, AWS, OpenStack, and Red Hat Linux, serving both commercial and government environments, including government classified clouds. Our mission is to forge the future with secure, innovative technologies that protect and uplift communities, while our vision is to revolutionise seamless collaboration across domains on an ultra secure platform. We have delivered outcomes on programs such as Azure Stack Hub, Azure Protected, OpenStack Protected, Secret Cloud, and major national security system uplifts, and continue to play a pivotal role in evolving sovereign cloud capability.

The Role

We are seeking an experienced Senior AWS Data Engineer to design, build, and support scalable data pipelines and curated datasets on our modern AWS Data Platform. In this role you will work closely with cross functional teams to ingest, transform, and serve high quality data for analytics, reporting, and downstream applications. You will be hands on with AWS native data services, SQL, Python, and modern data engineering practices, playing a critical role in ensuring reliable, secure, and high performing data products across enterprise and mission critical environments.

Key Responsibilities

• Design, develop, and maintain end to end data pipelines on the AWS Data Platform.
• Build and manage ETL/ELT workflows using AWS services such as AWS Glue, Amazon S3, Redshift, Athena, and EMR, as well as orchestration tools.
• Implement scalable data ingestion patterns for databases, APIs, files, and event streams, delivering reliable raw, cleansed, and curated data layers.
• Develop transformation logic using SQL, Python, and PySpark for data cleansing, enrichment, and standardisation.
• Implement robust data quality frameworks, reconciliation routines, monitoring, alerting, and fault tolerant recovery processes.
• Collaborate with data analysts, data scientists, and business stakeholders.
• Contribute to DataOps/DevOps practices including version control, CI/CD pipelines, automated testing, and release management.
• Produce and maintain technical documentation including data flows, mapping specifications, job schedules, and operational runbooks.
• Optimise data pipeline performance, query execution, and workflow orchestration.
• Support production deployments, operational troubleshooting, and continuous improvement activities.

What You'll Need

• Experience as a Data Engineer in enterprise scale environments.
• Advanced SQL capability, including complex joins, window functions, and performance tuning.
• Hands on experience working with Teradata and Siebel CRM datasets.
• Strong practical experience with core AWS data services.
• Strong programming skills in Python and data transformation experience with PySpark (preferred) or Spark.
• Experience with workflow orchestration platforms such as Airflow.
• Solid understanding of data warehousing concepts: dimensional modelling, partitioning, incremental loads, and CDC patterns.
• Experience implementing monitoring, logging, alerting, and operational support frameworks.
• Excellent communication skills and ability to translate requirements into technical data solutions.
• Telecommunications industry experience is highly desirable.