Why This Role Stands Out
This hybrid GCP Data Engineer role at Tata Consultancy Services offers exciting opportunities to build and optimize scalable data solutions using cutting-edge Google Cloud technologies, perfect for those eager to deepen their expertise in data warehousing and ETL/ELT. You'll thrive here if you possess strong Python and SQL skills and enjoy collaborating with diverse teams to drive data-informed insights. Embrace this chance to grow your career with a globally recognized leader and contribute to impactful projects.
Quick Overview
Seniority
Mid Senior
Employment type
Full Time
Work mode
Hybrid
Location
Sydney, Sydney, Australia
DockerGCPSQLETLLookerAgileAirflowApacheBigQueryGoogle CloudKafkaKubernetesPythonStakeholder Management
Job Description
- Design, develop, and maintain scalable ETL/ELT pipelinesusing Python, SQL, and GCP tools
- Build batch and real-time data processing workflows
- Automate data pipelines using Apache Airflow (CloudComposer) for orchestration
- Develop and manage data solutions using BigQuery, CloudStorage, Dataflow, and Pub/Sub
- Implement scalable data warehousing and lakehouse architectures
- Monitor and optimize cloud infrastructure performance,reliability, and cost
- Design and implement logical and physical data models
- Perform data cleansing, transformation, and aggregationfor analytics readiness
- Ensure high data quality, integrity, and consistency acrosssystems expertia.ai
- Enable business insights through dashboards using LookerStudio / LookML
- Collaborate with analysts and business teams to definereporting requirements
- Work closely with data scientists, analysts, and businessstakeholders to gather requirements
- Provide data platform support and resolve data-related issues
- Communicate technical solutions effectively to non-technicalstakeholders
- Implement data governance, access control, and security bestpractices
- Ensure compliance with data privacy regulations and enterprisepolicies
- Identify and implement performance optimizations acrosspipelines and databases
- Introduce automation and best practices for DataOps and CI/CD
- Stay updated with evolving GCP and data engineeringtechnologies
- Strong programming expertise in Python and SQL
- Hands-on experience with Google Cloud Platform (BigQuery,Dataflow, Pub/Sub, Cloud Storage)
- Workflow orchestration using Apache Airflow / Cloud Composer
- Experience with ETL/ELT pipeline design and data warehousing
- Visualization and reporting using Looker Studio / LookML
- Strong understanding of data lifecycle (ingestion transformation analytics)
- Expertise in data modeling, data integration, and pipelineoptimization
- Familiarity with Agile / DevOps practices in dataengineering
- Strong analytical and problem-solving capabilities
- Excellent communication and stakeholder management skills
- Ability to work in cross-functional, global teams
- Bachelor's / Master's degree in Computer Science, DataEngineering, or related field
- 5+ years of experience in dataengineering or similar roles
- Proven experience working with GCP-based data platforms
- Hands-on experience in Python, SQL, Airflow orchestration
- Experience with PySpark / Spark / Kafka (streamingpipelines)
- Knowledge of CI/CD, Docker, Kubernetes
- Exposure to AI/ML pipelines and advanced analytics
- Certifications such as Google Professional Data Engineer
- Pipeline performance and reliability (uptime, latency)
- Data quality and accuracy metrics
- Time-to-deliver data for business insights
- Cost optimization on cloud data infrastructure
- Adoption and usability of dashboards (Looker Studio)
Similar jobs
- MG
AI-Enabled Data Engineer - Hybrid & Impact
NewMantel Group
Brisbane, Queensland🇦🇺Hybrid2 hours agoTechnology - GF
Interim Lead Data Engineer
NewGo Fractional
Sydney🇦🇺Hybrid2 hours agoOracleSQLSnowflake+3Technology - MG
AI-Enabled Data Engineer - Hybrid & Impact
NewMantel Group
Melbourne, Victoria🇦🇺Hybrid2 hours agoTechnology - CO
Senior Snowflake Data Engineer - Cloud Data Solutions
NewCognizant
Sydney🇦🇺Hybrid2 hours agoSnowflakeTechnology - KG
Senior Data Engineer
NewKinetic Group
Melbourne, Victoria🇦🇺Hybrid7 hours agoSQLETLMachine Learning+1Technology - EI
Data Engineer
NewEightcap
Melbourne, Victoria🇦🇺Hybrid7 hours agoSQLAWSETL+3Technology