Back to Jobs
Recently postedTechnology

Sr Data Platform Engineer / Platform Lead - Reading PA, Onsite. Need strong experience with Python and PySpark, SQL skills, Apache Airflow, Snowflake, PostgreSQL, S3, EMR/Spark, AWS, streaming or change-data-capture (CDC), Machine Learning, SnowPro Core,

KeylentReading, PA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationReading, PA, United States
DockerSQLGitApache SparkAWSELKSnowflakeAirflowApacheBambooConfluenceGrafanaKafkaPrometheusPythonPostgreSQLKubernetesData AnalysisCI/CD

Job description

Sr Data Platform Engineer / Platform Lead

Position Summary

As a Data Platform Engineer/Platform Lead within Digital Products, you'll help build and run the data platform behind our fleet and telematics products from ingesting raw data through to the datasets and views that power our customer-facing applications. You'll work closely with Data Science and product teams, balancing hands-on pipeline development with platform-level decisions on cost, reliability, and scale.

Responsibilities

Design, build, and maintain data pipelines and orchestration workflows that move data from ingestion through transformation to serving layers

Write and optimize SQL and PySpark for large-scale data processing and transformation

Tune cloud data warehouse performance and cost (query optimization, refresh/scheduling strategy, resource sizing) as data volume and usage scale

Build and maintain data quality checks and pipeline monitoring/alerting to catch issues before they reach downstream consumers

Partner with Data Science to support feature engineering and analytics pipelines feeding ML/analytics workloads

Collaborate with product and engineering stakeholders to translate business needs into reliable, scalable pipeline design

Support incident response, root-cause analysis, and continuous improvement of the data platform

Document architecture and pipeline design to support onboarding of new team members

Contribute to platform-level decisions on architecture, tooling, and technical direction as the team and data volume scale

Core Skill Set Required

Strong SQL skills, including query optimization

Proficient in Python and PySpark for large-scale/distributed data processing

Hands-on experience with a workflow orchestration tool (e.g., Apache Airflow)

Experience with a cloud data warehouse (e.g., Snowflake)

Experience with relational databases (e.g., PostgreSQL)

Experience with cloud data infrastructure (AWS), including cloud storage and distributed processing (e.g., S3, EMR/Spark)

8+ years in data engineering with hands-on ownership of production data pipelines

Strong communication skills; able to explain technical tradeoffs to both technical and non-technical stakeholders

Good to Have

Familiarity with streaming or change-data-capture (CDC) ingestion patterns (e.g., Apache Kafka)

Experience building data quality frameworks and pipeline observability/alerting

Exposure to ML/analytics pipeline support (feature engineering, model-serving data prep)

Familiarity with monitoring and logging tools, such as Dynatrace, Prometheus, Grafana, or the ELK stack

Strong knowledge of DevOps principles, continuous integration, continuous delivery, and infrastructure automation

Hands-on experience with containerization technologies like Docker and container orchestration platforms like Kubernetes or OpenShift

Experience with CI/CD and collaboration tooling such as Bamboo, Bitbucket, and Confluence

Relevant certifications (e.g., SnowPro Core, SnowPro Advanced: Data Engineer, AWS Data Analytics)

Experience building or supporting SaaS products

Specialized experience with secure data sharing in Snowflake

Similar jobs