Google Data Platform Engineer

4-8 years data engineering, including 2+ years hands-on Google Cloud Platform. Reports to the Senior Platform Engineer.

A build role against a defined architecture. Implements assigned pipelines and models to the standard set by the architect and senior engineer. Streaming is the default on this platform, not an occasional requirement.

Responsibilities

• Build streaming Dataflow pipelines in Apache Beam consuming Pub/Sub events into the BigQuery bronze layer.

• Implement deduplication, idempotent writes, event ordering, and late-arriving event handling the logic most likely to fail silently if done carelessly.

• Implement schemas as data contracts and handle schema evolution without dropping or corrupting events.

• Implement DLQ routing, message archival, and the replay path, and test recovery under realistic failure rather than happy-path only.

• Build Dataform models across conformed and mart layers with meaningful assertions, plus business-friendly table and column documentation as part of the build.

• Apply BigQuery performance and cost practices in code: partitioning, clustering, incremental materializations.

• Build reconciliation checks against the system of record and produce sign-off evidence.

• Register datasets in Dataplex and apply policy tags and row-level security to the required granularity.

• Build one-time historical migration loads from files and database extracts, reconciled against the streaming path at cutover.

• Contribute Terraform modules and CI/CD; write tests including replay and duplicate-event scenarios.

• Emit structured logs and metrics from every pipeline so the platform's operations layer can monitor it; write runbooks; support UAT, cutover, and hypercare.

Required

• Hands-on streaming experience Pub/Sub and Dataflow, or Kafka / Flink / Kinesis with real exposure to deduplication, ordering, and replay.

• Strong Python and advanced SQL: window functions, CTEs, incremental merge patterns, query tuning.

• Apache Beam, or demonstrable ability to ramp quickly from another streaming framework.

• Hands-on BigQuery: partitioning, clustering, cost-aware query design.

• Dataform or dbt including tests or assertions and dependency management.

• Working knowledge of dimensional modeling.

• Git workflow and CI/CD; Terraform, or willingness to ramp quickly.

• Exposure to a major SaaS platform as a data source and its change-event mechanisms.

• Comfort building to an architecture someone else defined, raising concerns through the right channel rather than deviating quietly.

Preferred

• Google Cloud Platform Professional Data Engineer certification; Dataplex and DLP; Analytics Hub or Looker familiarity.