Quick Overview
Job Description
Job Summary:
We are looking for an experienced Lead Data Engineer to support a large-scale Data Warehouse modernization and migration initiative. The ideal candidate will analyze existing ETL processes built on SSIS, Informatica, SQL Server Stored Procedures, and Oracle PL/SQL packages, understand the underlying business logic, and migrate them to Snowflake SQL and Python-based data processing frameworks.
This role requires strong expertise in Snowflake, Python, AWS, Control-M, GitHub, Jenkins, SSIS, Informatica, Oracle PL/SQL, SQL Server.. In addition to migration activities, the candidate will be responsible for data validation, report adoption support, and decommissioning legacy ETL processes.
Key Responsibilities:
- Analyze and understand existing ETL and database processes built using:
- SSIS
- Informatica
- SQL Server Stored Procedures
- Oracle PL/SQL Packages
- Convert and optimize existing business logic into Snowflake SQL.
- Develop and maintain Python-based data ingestion and orchestration processes.
- Build scalable data pipelines and batch processing solutions.
- Configure, schedule, monitor, and troubleshoot jobs using Control-M.
- Work with AWS services including EC2, S3, AWS Batch, Secrets Manager, and CloudWatch/Log Monitoring.
- Perform Snowflake performance tuning and query optimization.
- Execute data validation and reconciliation between legacy systems and Snowflake.
- Partner with Reporting and Analytics teams to validate reports and ensure successful adoption of migrated data.
- Support UAT, parallel runs, and production cutovers.
- Decommission legacy ETL jobs, scheduling processes, and obsolete data platforms after successful migration.
- Manage source code using GitHub and support CI/CD deployments through Jenkins.
- Collaborate with business and technical teams throughout the migration lifecycle.
Required Skills:
Must Have:
- 10+ years of Data Engineering, ETL, and Data Warehouse experience.
- Strong Snowflake SQL development and performance tuning experience.
- Experience migrating ETL processes from SSIS, Informatica, SQL Server, or Oracle environments.
- Strong Python development skills.
- Hands-on experience with Control-M scheduling and monitoring.
- AWS experience with:
- EC2
- S3
- AWS Batch
- Secrets Manager
- CloudWatch
- GitHub and Jenkins CI/CD experience.
- Experience with data comparison, reconciliation, and migration validation.
Similar jobs
- RA
Database Administrator with Security Clearance
Red Arch Solutions
San Antonio, TX🇺🇸Hybrid6 weeks agoOracleSQLSQL ServerTechnology - LG
Data Engineer - Clearance Required
LMI Government Consulting (Logistics Management In
United States🇺🇸$102.0k - $170.2k/yrHybrid6 weeks agoData PipelineTechnology - AS
Postgres Database Administrator (HYBRID) with Security Clearance
American Systems Corporation
Mechanicsburg, PA🇺🇸$125k - $155k/yrHybrid4 weeks agoOracleAWSPostgreSQL+1Technology - PS
Database Engineer (MongoDB) with Security Clearance
Power3 Solutions
Linthicum Heights, MD🇺🇸Hybrid5 weeks agoDjangoDockerFlask+11Technology - AS
Data Engineer - Mid with Security Clearance
American Systems Corporation
Quantico, VA🇺🇸$135k - $155k/yrHybrid6 weeks agoGCPOracleSQL+8Technology - AS
Data Engineer - Junior with Security Clearance
American Systems Corporation
Quantico, VA🇺🇸$110k - $125k/yrHybrid6 weeks agoSQLData PipelinePythonTechnology