Sr. Data Engineer

Role Title – Sr. Data Engineer
Duration: 6+ months
Location: 100% Remote -50% travel to client site (Indianapolis)
Job ID- DLTJP00063915
W2 Candidates only

Role Overview

Senior Data Engineer (8+ years) will be responsible for designing, building, and maintaining data pipelines that extract data from source systems, transform it through the medallion

architecture layers, and prepare it for consumption by the analytics layer. This role will work closely with the Power BI Developer to ensure data is properly structured, documented, and accessible for reporting and analytics.

Good Manufacturing Practice GMP experience is a preferred.

Key Responsibilities

Phase 1 Deliverables

• Following EDB standard, design and establish AWS S3 bucket structure for data lake (Bronze/Silver/Gold zones) with Red CCI security controls

• Build new data pipelines to extract data from the selected SaaS-based HSE systems

• Enhance/extend data pipelines using the dataset from AWS EDB Asset related data products and digital solutions

• Build data ingestion pipeline for Manufacturing Execution System via REST API (Lambda-based extraction).

• Data pipeline to SaaS based tools to integrate data into the data warehouse

• Implement CDC pipeline for LabVantage LIMS using AWS DMS from Oracle database (or Azure Data Factory)

• Develop Bronze-to-Silver transformations using AWS Glue or Azure Data Factory depending on data domains

• Configure AWS Glue Data Catalog with appropriate metadata and Red CCI classification tags

Phase 2 Deliverables

• Connect to EDB marketplace for enterprise reference data

• Build Silver-to-Gold transformations creating batch-centric data products

• Implement data quality checks and monitoring dashboards

• Develop orchestration workflows using AWS Step Functions

• Support Power BI Developer in validating OneLake shortcut connectivity to Gold zone

• Document data lineage, schema definitions, and pipeline architecture

Ongoing Responsibilities

• Monitor pipeline health, troubleshoot failures, and optimize performance

• Collaborate with Power BI Developer on data model requirements and data quality issues

• Support data governance and security compliance reviews

• Respond to ad-hoc data requests from engineers

• Coordinate with enterprise EDB team on data sharing agreements and standards