Why this role stands out
This Lead Data Engineer role offers a fantastic opportunity to shape enterprise data strategy and build scalable, impactful solutions within a well-established midstream company, with the added flexibility of a hybrid work model. You'll thrive here if you possess extensive cloud and Databricks experience, strong Python and SQL skills, and a passion for driving innovation in data engineering. Apply today to leverage your expertise and advance your career!
Quick overview
Job description
We are seeking an experienced Lead Data Engineer with strong expertise in the Oil & Gas industry, specifically in liquids and gas systems, to lead the design, development, and delivery of modern data products that enable operational excellence, engineering analytics, and business decision-making.
The ideal candidate combines deep domain knowledge with exceptional software engineering skills. This role is responsible for integrating data from diverse operational and enterprise systems, designing scalable data pipelines, and building high-quality data products that support engineering, operations, commercial, and analytics teams.
As the technical lead, you will establish engineering best practices, mentor other developers, and collaborate closely with business stakeholders, engineers, data scientists, and solution architects to transform complex operational data into trusted, consumable assets.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines that ingest data from multiple operational and enterprise systems.
- Build modern, reusable data products that provide clean, governed, and analytics-ready datasets.
- Integrate structured and semi-structured data from a variety of sources including:
- SCADA systems
- Historians (OSI PI, Canary, etc.)
- Pipeline management systems
- Measurement systems
- ERP platforms
- Asset management systems
- Laboratory systems
- IoT and sensor data
- APIs, flat files, and cloud storage
- Develop robust data models that support operational reporting, engineering analysis, forecasting, optimization, and advanced analytics.
- Write high-quality, maintainable, and well-tested Python and SQL code following software engineering best practices.
- Design orchestration workflows and automated data pipelines using modern data engineering frameworks.
- Optimize pipeline performance, reliability, scalability, and monitoring.
- Collaborate with business users to translate operational requirements into scalable technical solutions.
- Partner with data scientists and analytics teams to deliver production-ready datasets for machine learning and AI initiatives.
- Define data quality standards, validation rules, lineage, and governance practices.
- Mentor and provide technical leadership to other data engineers.
- Participate in architecture decisions and establish data engineering standards across the organization.
Required Qualifications
- Bachelor''''s degree in Computer Science, Engineering, Information Systems, or related technical field.
- 8+ years of professional data engineering or software engineering experience.
- 3+ years leading technical teams or serving as a senior technical contributor.
- Demonstrated experience within the Oil & Gas industry, with strong knowledge of liquids and natural gas operations, pipeline systems, gathering systems, production, transportation, or midstream operations.
- Strong understanding of operational and engineering data used in oil and gas environments.
Required Technical Skills
Programming
- Expert-level Python
- Advanced SQL
- Object-oriented programming
- REST API integration
- Git version control
Data Engineering
- ETL/ELT pipeline development
- Data modeling (dimensional and normalized)
- Data warehousing
- Data lake architectures
- Data transformation frameworks
- Workflow orchestration
- Data quality and validation
- Metadata management
- Data governance principles
Cloud & Platforms
- Microsoft Azure
- AWS
- Google Cloud Platform
Experience with technologies such as:
- Databricks
- Snowflake
- Microsoft Fabric
- Azure Data Factory
- Delta Lake
Databases
- SQL Server
- PostgreSQL
- Oracle
- Azure SQL
- Snowflake
- NoSQL databases (preferred)
Preferred Oil & Gas Domain Experience
Candidates should have experience working with data related to:
- Liquids pipeline operations
- Natural gas transportation
- Midstream operations
- Pipeline integrity
- Measurement and metering
- Flow and pressure monitoring
- Compression systems
- Storage facilities
- Scheduling and nominations
- Production accounting
- Asset performance
- Regulatory reporting
- Operational KPIs
Experience integrating data from systems such as SCADA, PI Historian, measurement systems, GIS, ERP, and engineering applications is highly desirable.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- KS

SAS EngineerRecently posted
Keypixel Software Solutions · United States · Hybrid · Posted 18 Nov 2026 🇺🇸
Data AnalysisEngineeringSalary not listed - CG
Sr. Data Engineer with Security ClearanceRecently posted
Contact Government Services, LLC · Albany, NY · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLShellGit+10Technology$144.8k - $209.1k/yr - SJ

Senior Data EngineerRecently posted
Selby Jennings · Westport, CT · Hybrid · Posted 18 Nov 2026 🇺🇸
Neo4jSQLGit+8TechnologySalary not listed - PE

Sr. Data Engineer (OBI Advanced Analytic Method Augmentation) with Security ClearanceRecently posted
Prescient Edge · Washington, DC · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerSQLETL+5Technology$160k - $180k/yr - AP

Sr Snowflake Data EngineerRecently posted
Aptino · United States · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLApache SparkETL+3TechnologySalary not listed - AT
Azure Data EngineerRecently posted
Arbor Tek Systems · Cincinnati, OH · On-site · Posted 18 Nov 2026 🇺🇸
GitApache SparkETL+7TechnologySalary not listed - DT
Data Engineer -Scala Expert /Remote/10+yrs onlyRecently posted
Digipulse Technologies, Inc · United States · Remote · Posted 18 Nov 2026 🇺🇸
SQLScalaGit+4TechnologySalary not listed - KT

Data Engineer - Mid LevelRecently posted
Kforce Technology Staffing · Irving, TX · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLScalaETL+2TechnologySalary not listed - MT
Senior Kafka Engineer / AdministratorRecently posted
Microgreen Technologies LLC · St. Louis, MO · On-site · Posted 18 Nov 2026 🇺🇸
EngineeringSalary not listed - CT
Senior Kafka Engineer / AdministratorRecently posted
Chabez Tech LLC · St. Louis, MO · On-site · Posted 18 Nov 2026 🇺🇸
EngineeringSalary not listed - AT
Salesforce Data Migration Lead
Atika Tech · Holmdel, NJ · On-site · Posted 7 Nov 2026 🇺🇸
Stakeholder ManagementMySQLOracle+7TechnologySalary not listed - PB
Software Data Engineer (SWD6) with Security ClearanceRecently posted
Point Blank Technologies · Laurel, MD · Hybrid · Posted 18 Nov 2026 🇺🇸
AWSPowerShellPython+3TechnologySalary not listed
