Job Title: Senior Azure Data Architect / Lead Data Engineer (15+ Years)

Location: Irvine, CA (Hybrid/Onsite)

Duration: Long-Term Contract

Job Summary

We are seeking a highly experienced Azure Data Architect / Lead Data Engineer with 15+ years of experience in designing and implementing enterprise-scale data platforms. The ideal candidate will have deep expertise in Azure Databricks, Azure Data Factory (ADF), Azure Data Lake, Azure DevOps, AWS, PySpark, SQL, and cloud-based data engineering. The candidate will lead architecture, mentor engineering teams, and deliver scalable, secure, and high-performance data solutions.

Required Experience

• 15+ years of experience in Data Engineering, Data Warehousing, and Big Data.
• 8+ years of hands-on experience with Microsoft Azure.
• 5+ years of experience with Azure Databricks and PySpark.
• Strong expertise in Azure Data Factory (ADF).
• Experience working with AWS cloud services.
• Experience leading enterprise-scale cloud migration and modernization initiatives.
• Experience leading technical teams and collaborating with stakeholders.

Required Technical Skills

• Azure Databricks
• Azure Data Factory (ADF)
• Azure Data Lake Storage Gen2 (ADLS)
• PySpark
• Spark SQL
• Python
• SQL Server
• Azure Synapse Analytics
• Delta Lake
• Medallion Architecture (Bronze, Silver, Gold)
• Azure DevOps
• Git
• CI/CD Pipelines
• AWS S3
• AWS Glue
• AWS Lambda
• AWS EC2
• AWS IAM
• AWS Redshift
• Data Modeling
• ETL/ELT
• Data Warehousing
• Performance Tuning
• REST APIs
• JSON
• Parquet
• Power BI
• Terraform (preferred)
• Docker & Kubernetes (preferred)

Responsibilities

• Design enterprise-scale cloud data architecture on Azure.
• Develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
• Build high-performance data processing solutions using PySpark and Spark SQL.
• Design and implement Lakehouse architecture using Delta Lake.
• Integrate Azure and AWS platforms for hybrid cloud data solutions.
• Build CI/CD pipelines using Azure DevOps.
• Lead cloud migration from on-premises data platforms to Azure.
• Optimize Spark jobs and data pipelines for performance and cost.
• Implement data governance, security, and monitoring best practices.
• Mentor junior engineers and provide technical leadership.
• Collaborate with architects, business stakeholders, and product teams.
• Drive solution design, code reviews, and architectural decisions.

Qualifications

• Bachelor''s or Master''s degree in Computer Science, Information Systems, Engineering, or a related field.
• Microsoft Azure certifications (preferred):
  • Azure Data Engineer Associate (DP-203)
  • Azure Solutions Architect Expert (AZ-305)
• Databricks Certified Data Engineer (preferred)
• AWS Certified Solutions Architect or AWS Data Analytics certification (preferred)

Preferred Skills

• Microsoft Fabric
• Snowflake
• Kafka
• Event Hubs
• Airflow
• Streaming data processing
• Data governance tools
• Healthcare, Banking, Insurance, or Retail domain experience

Soft Skills

• Excellent communication and stakeholder management.
• Strong leadership and mentoring skills.
• Experience leading globally distributed teams.
• Ability to translate business requirements into scalable technical solutions.
• Strong analytical and problem-solving skills.