Lead Engineer

Lead Engineer

Duration - 6 months contract to hire

Location – onsite Tue, Wed, Thu – Chicago – downtown / Irving, TX

Qualifications

• 15+ years of professional data engineering experience, with proven technical leadership.
• Deep understanding of Azure tech stack (Databricks, ADF, Key Vault, ASQL, ADLS Gen2, Service Principles etc.)
• Deep experience with Databricks (design, optimization, notebooks, stored procedures, UDFs, etc).
• Hands-on experience with ETL tools such as SSIS and Azure Data Factory.
• Experience working with API calls and Ingesting data from external source systems
• Experience with SQL, Python and PySpark
• Experience with building dynamic and scalable solutions for data ingestion and curation
• Experience in data engineering and MS SQL Server and T-SQL development.
• Strong understanding of data lake/delta lake architectures and modern data pipelines.
• Strong understanding of medallion architectures and design patterns.
• Strong understanding of data modeling techniques.
• Experience handling large-scale data volumes (millions of records).
• Proficiency in building and maintaining CI/CD pipelines.
• Experience with IaC technologies such as Terraform or Pulumi.
• Exposure to big data tools (Spark, Hadoop, Kafka) preferred.
• Experience with reporting tools such as Power BI or Tableau a plus.
• Knowledge of .NET development beneficial for cross-team collaboration.