Sr. Data Engineer / Lead (SQL / Databricks)

Introduction: As a Sr. Data Engineer / Lead specializing in SQL and Databricks, you will play a key role in our team, working on data transformation and integration projects. You will be responsible for designing, developing, and maintaining data pipelines to support our organization''s data needs.

Responsibilities:

• Utilize SQL Server and Azure SQL to develop and optimize database solutions
• Work with Databricks and PySpark for data processing and analysis
• Use Python for scripting and automation tasks
• Implement and manage Azure Data Factory (ADF) for data integration
• Perform Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes
• Work with Azure services such as ADLS Gen2 and Azure SQL
• Implement Continuous Integration and Continuous Delivery (CI/CD) pipelines using Git and Jenkins
• Take ownership of projects and lead a team of data engineers

Requirements:

Required Skills:

• Experience with SQL Server and Azure SQL
• Proficiency in Databricks and PySpark
• Strong knowledge of Python programming language
• Hands-on experience with Azure Data Factory (ADF)
• Experience with ETL/ELT processes
• Familiarity with Azure services like ADLS Gen2 and Azure SQL
• Experience with CI/CD practices, Git, and Jenkins
• Prior experience in a lead or ownership role

Preferred:

• Knowledge of Pulumi for Infrastructure as Code (IaC)
• Background in Healthcare or Clinical Domain

Location: Irving, TX (Hybrid – 3 Days Onsite)

Duration: 6+ Months

Experience: 8+ Years