Lead Data Engineer

job summary:

Are you a Data Engineer with over 10 years of experience and expertise working with Databricks on AWS?

Overview:

We are investing in the next evolution of our data, analytics, and artificial intelligence ecosystem. We are looking for a Lead Data Engineer who's ready to do more than just build pipelines on Databricks and AWS.

You will help shape enterprise analytics strategy, influence architecture, and partner directly with the business to drive smarter decisions.

Why This Role?

This is a high-impact, high visibility role where you will:

Lead the design and implementation of a modern AWS + Databricks Lakehouse platform

Partner across IT, analytics, and business teams to turn data into decisions

Drive scalable, governed, high-performance data solutions

Influence how data powers analytics, reporting, and AI across the company

Our transportation client is looking for a seasoned Data Engineer to lead the design and implementation of their modern AWS and Databricks Lakehouse platform.

You will design and lead enterprise-grade pipelines, own the ETL/ELT lifecycle, drive data engineering standards and architecture, and translate business requirements into scalable, high-performing solutions.

Apply if you are a good fit and local to the Chicagoland area, interviews will be onsite in Chicago.

location: Chicago, Illinois
job type: Contract
salary: $70 - 83 per hour
work hours: 9am to 5pm
education: Bachelors
responsibilities:

Why This Role?

This is a high-impact, high visibility role where you will:

Lead the design and implementation of a modern AWS + Databricks Lakehouse platform

Partner across IT, analytics, and business teams to turn data into decisions

Drive scalable, governed, high-performance data solutions

Influence how data powers analytics, reporting, and AI across the company

What You'll Be Doing

Design and lead enterprise-grade pipelines, supporting both batch and streaming data processing

Own the full ETL/ELT lifecycle, from design through production support

Drive data engineering standards, best practices, and architecture across the data ecosystem

Translate business requirements into scalable, high-performance data solutions

Mentor a small team while remaining hands-on

Ensure data reliability, performance, governance, and cost optimization

qualifications:

Qualifications:

10+ years of IT experience, including deep expertise in data engineering & ETL

4-6+ years of recent hands-on experience in designing, building, and scaling data pipelines

Strong hands-on experience with Databricks (Spark, Delta Lake, Unity Catalog) and AWS (S3, IAM, networking)

Highly proficient in Databricks' core architectural pattern, which combines the best elements of data lakes and data warehouses.

Familiar with advanced features like Unity Catalog (for governance), Delta Live Tables (DLT for pipelines), Databricks SQL, and workflow orchestration.

Expertise in SQL, Python, data modeling, and distributed processing.

Experience with CI/CD, DevOps, and modern data architecture (lakehouse, medallion, etc.).

Experience working in an Agile analytics environment where close collaboration with key stakeholders to address evolving requirements is the norm.

Strong communication skills and the ability to translate complex technical concepts into solutions that deliver business outcomes.

A passion for continued learning and an interest in staying current with the rapidly evolving data and analytics space.

What Sets You Apart

You're analytical, curious, and solutions-oriented

You thrive in environments where you balance priorities and drive outcomes

You're comfortable with ambiguity and can quickly learn the business

You build strong partnerships and bring a "get things done" mindset

skills:

Agile,S3,AWS,Spark,AI,database platform,CI/CD,modern data architecture,Delta Lake,data lakes,data pipelines,streaming data,data solutions,data warehouses,Databricks,Databricks SQL,data ecosystem,DevOps,distributed processing,ETL,IAM,Data Engineering,IT experience,AZURE,Python,SQL,Unity,workflow orchestration,Flexibility,analytical,Strong communication skills,curious,innovation mindset,networking,passion,high-performance,solutions-oriented,architecture,partnerships,data modeling,hands-on design,governance,production support,quickly learn

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.
At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact
Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).
This posting is open for thirty (30) days.