Lead Data Engineer

Location: Tustin, CA Salary: $180,000.00 USD Annually - $190,000.00 USD Annually Description:
Lead Data Engineer (EDO Platform Engineering Lead)

Location: Irvine, CA
Salary: $180,000-$190,000

About the Role

We are seeking an experienced Lead Data Engineer / EDO Platform Engineering Lead to architect, build, and manage enterprise-scale cloud-native data platforms supporting Asset Management business functions. This role will lead platform engineering, infrastructure automation, DevOps enablement, platform reliability, security, and operational excellence across the Enterprise Data Office (EDO).

The ideal candidate will possess deep expertise in Databricks, dbt, Airflow, AWS Cloud, Terraform, Python, DevOps, Platform Engineering, and Data Operations, along with a strong background supporting Asset Management and Investment Management organizations.

This position will be responsible for establishing and operating a secure, scalable, highly available enterprise data platform supporting analytics, reporting, regulatory compliance, portfolio management, risk management, and investment operations.
Responsibilities
Platform Engineering & Architecture

• Lead the design, implementation, and management of enterprise-scale data platforms.
• Define platform engineering strategy aligned with Enterprise Data Office (EDO) and enterprise architecture standards.
• Establish reusable platform frameworks, accelerators, reference architectures, and engineering best practices.
• Drive cloud-native modernization initiatives across data and analytics ecosystems.
• Own platform lifecycle management, scalability, reliability, availability, and performance.

Databricks Platform Leadership

• Lead Databricks platform architecture, administration, and governance.
• Design and implement modern Lakehouse architectures utilizing Delta Lake.
• Manage and optimize:
  • Databricks Workflows
  • Delta Live Tables (DLT)
  • Unity Catalog
  • MLflow
  • Structured Streaming
  • Cluster Policies
  • Security & Access Controls
• Optimize platform performance, scalability, and cost efficiency.
• Establish governance controls across development, testing, and production environments.

Cloud Engineering & Infrastructure

• Lead cloud engineering initiatives utilizing AWS services including:
  • Amazon S3
  • ECS
  • EKS
  • AWS Lambda
  • AWS Glue
  • CloudWatch
• Design highly available and secure cloud-based data platform solutions.
• Ensure platform resiliency, security, compliance, and operational excellence.

Infrastructure as Code (IaC)

• Develop and maintain reusable Terraform modules.
• Automate environment provisioning and deployment processes.
• Implement infrastructure standardization and governance.
• Maintain infrastructure version control and change management.
• Enforce compliance through Infrastructure-as-Code best practices.

DevOps & Automation

• Design and implement CI/CD pipelines for platform and data engineering teams.
• Automate deployment, release management, and operational processes.
• Enable GitOps and Infrastructure-as-Code methodologies.
• Improve deployment speed, reliability, and operational efficiency.
• Support self-service platform capabilities for engineering teams.

Platform Reliability & Operations

• Establish and implement Site Reliability Engineering (SRE) practices.
• Define and monitor platform SLAs, SLOs, and key operational metrics.
• Lead platform monitoring, observability, and incident management strategies.
• Drive root cause analysis, remediation, and preventive action programs.
• Support API integrations and operational workflows.

Tools & Technologies

• Databricks
• Delta Lake
• dbt
• Apache Airflow
• AWS Cloud
• Terraform
• Python
• GitHub Actions
• GitLab CI/CD
• Jenkins
• Azure DevOps
• Docker
• Kubernetes

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10+ years of experience in Data Engineering, Platform Engineering, Cloud Engineering, or related disciplines.
• Strong experience designing and supporting enterprise data platforms.
• Hands-on expertise with:
  • Databricks
  • dbt
  • Apache Airflow
  • AWS Cloud Services
  • Terraform
  • Python
  • CI/CD pipelines
  • Docker & Kubernetes
• Experience implementing Infrastructure as Code (IaC) and DevOps practices.
• Strong understanding of data governance, security, and compliance controls.
• Experience supporting large-scale enterprise data environments.
• Excellent leadership, communication, and stakeholder management skills.

Preferred Qualifications

• Experience within Asset Management, Wealth Management, Investment Management, or Financial Services organizations.
• Expertise in Lakehouse Architecture and Data Platform Modernization.
• Experience implementing Site Reliability Engineering (SRE) practices.
• AWS Certifications preferred.
• Databricks Certifications preferred.
• Experience supporting enterprise-scale analytics and regulatory reporting platforms.

Must-Have Skills

• Databricks
• dbt
• Apache Airflow
• AWS Cloud
• Terraform
• Python
• Platform Engineering
• DevOps
• Infrastructure as Code (IaC)
• Asset Management Industry Experience

Preferred Skills

• Delta Live Tables (DLT)
• Unity Catalog
• MLflow
• Structured Streaming
• Kubernetes
• Docker
• GitOps
• Jenkins
• GitHub Actions
• GitLab CI/CD
• Site Reliability Engineering (SRE)

Please send me your resume and Linked IN profile for immediate consideration.

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact:
This job and many more are available through The Judge Group. Please apply with us today!