Sr. Data Platform Engineer

Snowflake Platform Engineering | DataOps | DevOps

Remote (Work on EST)

W2 Contract

About the role

We are seeking a Senior Snowflake Data Platform Engineer to design, build, secure, optimize, and operate an enterprise data platform in a regulated financial services environment. This role combines hands-on Snowflake engineering with Azure Data Factory operations, vendor integrations, cloud networking, CI/CD, observability, data quality, and database migration responsibilities.

This is an end-to-end ownership role. The engineer will be accountable for reliable nightly processing, governed data delivery, production-ready deployment practices, proactive monitoring, and sustainable operating standards across Snowflake and Azure.

What you'll own

1. Snowflake platform engineering

• Design and implement Snowflake databases, schemas, virtual warehouses, resource monitors, storage integrations, network policies, and secure data-sharing patterns.

• Develop and maintain tables, views, materialized views, streams, tasks, Snowpipe, Dynamic Tables, stored procedures, Snowpark components, and Time Travel or zero-copy cloning patterns.

• Optimize query performance, warehouse sizing, clustering strategies, caching behavior, concurrency, storage, and compute consumption.

• Implement workload isolation, tagging, chargeback or show-back reporting, budget controls, and cost alerts.

• Establish platform standards for naming, object lifecycle, release management, and operational support.

2. Azure Data Factory at enterprise scale

• Operate, tune, and extend a production ADF estate supporting large-scale ingestion, transformation, and data distribution workloads.

• Design reusable, metadata-driven pipelines using parameterized datasets, configuration tables, shared templates, and restartable processing patterns.

• Manage Azure, Self-Hosted, and SSIS Integration Runtimes, including availability, upgrades, capacity planning, and performance troubleshooting.

• Own triggers, dependencies, concurrency, retry, idempotency, watermarking, and recovery behavior.

• Diagnose throttling, Integration Runtime saturation, cascading failures, poison files, schema drift, and redundant data movement.

3. Data integration: SFTP, APIs, batch, CDC, and streaming

• Onboard and operate inbound and outbound SFTP feeds with varying file formats, naming standards, delivery windows, and service-level expectations.

• Manage SSH keys, PGP encryption, IP allow-listing, certificates, and secrets through Azure Key Vault and managed identities.

• Integrate REST and SOAP APIs using OAuth 2.0, client credentials, mTLS, API keys, and token-refresh patterns through ADF, Azure Functions, or Logic Apps.

• Handle pagination, rate limiting, backoff, partial failures, incremental extraction, CDC, and micro-batch patterns.

• Implement file-arrival controls, late or missing file alerts, checksums, archival, retention, duplicate detection, and quarantine handling.

4. Data quality, reconciliation, and modeling

• Implement row-count reconciliation, source-to-target validation, duplicate detection, referential integrity, data profiling, freshness controls, and exception reporting.

• Build automated, metadata-driven data quality rules with thresholds, alerting, audit trails, and evidence retention.

• Design Raw, Curated, and Consumption layers and scalable dimensional, star, snowflake, and canonical data models.

• Support analytics-ready data products, semantic layers, and reporting solutions for Power BI, Sisense, Tableau, and similar tools.

• Collaborate with business, risk, finance, compliance, and data governance teams to define trusted data and acceptance criteria.

5. Security, governance, and regulatory controls

• Implement role-based access control, least privilege, segregation of duties, masking policies, row access policies, encryption, tagging, and secure views.

• Support PCI DSS, SOX, OCC, privacy, audit evidence, change control, retention, and internal security requirements.

• Enable data lineage, classification, cataloging, ownership, and policy enforcement using Microsoft Purview or comparable tooling.

• Ensure production changes are traceable, reviewed, tested, approved, and supported by rollback procedures.

6. Networking, CI/CD, and infrastructure as code

• Design Azure DevOps YAML pipelines for Snowflake objects, ADF artifacts, database changes, Azure Functions, and infrastructure deployments.

• Implement DEV-to-UAT-to-PROD promotion with parameterization, approval gates, artifact versioning, source control, and rollback.

• Troubleshoot VNets, subnets, NSGs, private endpoints, Private Link, service endpoints, DNS, private DNS zones, VPN or ExpressRoute, firewalls, and Self-Hosted IR network paths.

• Build reproducible environments using Bicep, ARM, Terraform, Snowflake CLI, schemachange, Flyway, Liquibase, dbt, or comparable deployment tools.

7. Database and warehouse migrations

• Plan and execute migrations from SQL Server, Oracle, PostgreSQL, DB2, Sybase, and legacy data warehouses into Snowflake and Azure targets.

• Use assessment, backup and restore, replication, CDC, staged file loading, and reconciliation approaches appropriate to cutoff windows and downtime tolerance.

• Address linked servers, SQL Agent jobs, SSIS packages, stored procedure incompatibilities, data type differences, collation, historical loads, and undocumented ETL.

• Rehearse cutovers with validation, business acceptance criteria, audit evidence, and tested rollback plans.

8. Observability and production operations

• Build monitoring with Snowsight, Access History, Account Usage, Event Tables, Azure Monitor, Log Analytics, KQL, alert rules, and action groups.

• Develop operational dashboards for pipeline health, SLA attainment, file arrival, warehouse utilization, cost trends, data quality, and Integration Runtime health.

• Reduce alert noise through actionable thresholds, dependency-aware notifications, and automated recovery where appropriate.

• Participate in on-call support, incident response, root cause analysis, blameless postmortems, problem management, and permanent remediation.

• Maintain runbooks, support procedures, architecture documentation, and vendor-facing technical communications.

Required qualifications

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related discipline, or equivalent practical experience.

• 7+ years of experience in data engineering, platform engineering, database engineering, or DevOps, including 3+ years of hands-on Snowflake production experience.

• 3+ years of hands-on Azure Data Factory experience in a meaningful production environment and demonstrated experience operating a large pipeline estate.

• Advanced SQL and strong knowledge of Snowflake architecture, query optimization, workload management, security, and cost governance.

• Strong Azure DevOps skills, including YAML pipelines, repositories, branching strategies, pull requests, approvals, service connections, and deployment troubleshooting.

• Solid Azure networking knowledge, including VNets, NSGs, private endpoints, DNS, hybrid connectivity, firewalls, and independent connectivity troubleshooting.

• Strong T-SQL and SQL Server knowledge, including cloud migration and production troubleshooting experience.

• Hands-on experience integrating external sources through SFTP and authenticated APIs.

• Practical experience with Azure Monitor, Log Analytics, KQL, and dashboards in Grafana, Power BI, or equivalent.

• Python and/or PowerShell scripting for automation, validation, and operational tooling.

• Infrastructure as code experience using Bicep, ARM, or Terraform.

• Experience implementing data quality, reconciliation, lineage, auditability, and secure data handling controls.

• Clear written and verbal communication, including runbooks, incident reports, architecture documentation, and stakeholder communications.

Preferred qualifications

• Financial services, banking, or another regulated-industry background with change control, SOX, PCI DSS, privacy, audit evidence, and segregation-of-duties experience.

• SnowPro Core or SnowPro Advanced certification; Microsoft Azure certifications such as DP-203, AZ-104, or AZ-400.

• Experience migrating enterprise SQL Server data warehouses and historical data to Snowflake.

• Experience with dbt, Snowpark, Databricks, Synapse, Microsoft Fabric, Kafka, or event-driven architectures.

• Experience with Microsoft Purview or another data catalog and lineage platform.

• Experience supporting AI/ML use cases, feature pipelines, or governed AI-ready data products.

• Experience with post-merger data integration, portfolio onboarding, or platform consolidation.

First 90 days

Success measures

• Reliable, secure, and scalable delivery of Snowflake and Azure data platform services.

• Measurable improvement in pipeline reliability, SLA attainment, data quality, incident detection, and recovery.

• Reduced compute and data movement costs through sizing, optimization, automation, and workload governance.

• Faster source onboarding through reusable metadata-driven patterns and governed self-service capabilities.

• Audit-ready change, reconciliation, security, lineage, and operational evidence.

Technology stack

Snowflake  |  Snowpipe  |  Streams & Tasks  |  Dynamic Tables  |  Snowpark  |  Azure Data Factory  |  Azure DevOps  |  Azure SQL / SQL Server  |  ADLS Gen2  |  Key Vault  |  Azure Monitor / Log Analytics  |  KQL  |  Grafana / Power BI  |  Purview  |  dbt  |  Bicep / Terraform  |  PowerShell  |  Python

How we work

We value engineers who automate repeatable work, document what they learn, raise delivery risks early, and treat observability, security, data quality, and rollback as part of the product. We favor sustainable engineering practices over preventable production heroics.