Google Cloud Platform Data Engineer

Required Qualifications (Must-Haves):

● Professional Experience: 5+ years of professional experience in a data engineering role, with a proven track record of building and maintaining large-scale data systems.

● Framework Design & Development: Demonstrable experience designing, building, and promoting the adoption of reusable data engineering frameworks such as Ingestion, Transformation, Validation etc

● On-Premise Data Warehouse/Big Data Expertise: Deep, hands-on experience with at least one of the following on-premise ecosystems:

○ Teradata: Strong understanding of the Teradata architecture, utilities (BTEQ, TPT, FastLoad/MultiLoad), and advanced SQL/Stored Procedure development.

○ Hadoop: Experience with the Hadoop ecosystem (HDFS, YARN, Hive) and hands-on proficiency with PySpark for large-scale data processing.

○ Enterprise ETL: Demonstrable experience designing and building complex workflows in an enterprise ETL tool like Informatica PowerCenter.

● Google Cloud Platform (Google Cloud Platform) Proficiency: Demonstrable hands-on experience designing, building, and operating solutions with a comprehensive set of Google Cloud Platform data services, including:

○ Core Data Processing & Warehousing: Google BigQuery (including data modeling, performance tuning, cost management), Google Cloud Storage (GCS), Cloud Dataflow, and Cloud Dataproc.

○ Orchestration & Event-Driven Architecture: Cloud Composer (Managed Airflow) for complex workflow orchestration, and Pub/Sub and Cloud Functions for building streaming and event-driven data pipelines.

○ Data Governance & Management: Practical experience using Dataplex for unified data management, security, and governance across data lakes and warehouses.

● Core Engineering & Migration Skills:

○ Expert-level proficiency in SQL, including complex joins, window functions, and performance tuning across different database engines.

○ Strong programming skills in Python, applying software engineering best practices.

○ Hands-on experience with Google's migration assessment tools (e.g., BigQuery Migration Service, Database Migration Service) to analyze on-premise workloads and accelerate migration.

○ Deep understanding of data warehousing concepts, ETL/ELT patterns, data modeling, and database design.

Preferred Qualifications (Nice-to-Haves):

● Proven Migration Experience: Direct, hands-on experience successfully completing at least one large-scale on-premise (Teradata, Hadoop, etc.) to Google Cloud Platform.