Senior Data Engineer

Senior Data Quality Engineer

Job ID: P00002343
Location: Chicago, IL
Duration: 12+Months

Job Summary

Seeking a Senior Data Quality Engineer with 5+ years of experience in data engineering/data quality engineering. The ideal candidate will have strong hands-on experience with Databricks, Spark/PySpark, Delta Lake, advanced SQL, cloud platforms, and data quality frameworks.

Key Responsibilities

• Design and implement enterprise data quality frameworks for Lakehouse Bronze, Silver, and Gold layers.
• Develop automated data validation, reconciliation, profiling, and quality checks within Databricks/ETL/ELT pipelines.
• Implement data observability, monitoring, alerts, dashboards, and quality scorecards.
• Perform RCA and remediation of data quality issues.
• Support data governance, metadata, lineage, auditability, SLAs, and data contracts.
• Integrate quality controls into batch and streaming pipelines.
• Implement CI/CD and automated testing for data quality processes.
• Collaborate with Data Engineers, Architects, BI, Governance, and business teams.
• Provide technical leadership and mentorship on data quality best practices.

Required Skills

• 5+ years in Data Engineering/Data Quality Engineering.
• Databricks, Spark/PySpark, Delta Lake
• Advanced SQL and data profiling/validation
• AWS or Azure
• ETL/ELT and orchestration tools such as Airflow or Control-M
• Data governance and lifecycle management
• Data observability/quality tools such as Great Expectations, Monte Carlo, or Deequ

Preferred

• Financial services or regulated-environment experience
• Collibra
• Real-time/streaming data quality
• Regulatory data controls/reporting
• Databricks or cloud certifications