Quick overview
Job description
Lead Data Engineer / Architect: Full time
Location: Cary, NC
Work Model: Day 1 onsite; hybrid schedule, Tuesday–Thursday onsite
Experience: 10–15 years
Mandatory Skills – Must-Have:
- Python, PySpark, Scala and SQL – strong production-level data engineering skills.
- Azure Databricks – Delta Lake, Spark performance tuning, Jobs/Workflows and Unity Catalog.
- Azure data services – ADLS Gen2 and Azure Data Factory (ADF).
- Lakehouse / Medallion Architecture – proven experience implementing Bronze, Silver and Gold layers at enterprise scale.
- Data Pipeline Engineering – metadata-driven ingestion frameworks, data transformation, data quality and large-scale pipeline optimization.
- Production AI/LLM Experience – hands-on experience designing and deploying LLM-based solutions, including RAG, embeddings, retrieval and prompt engineering.
- LLM Frameworks – experience with LangChain, LlamaIndex or LangGraph, plus Azure OpenAI, OpenAI or Databricks Model Serving.
- Azure DevOps / CI-CD – experience automating data pipeline deployments; Terraform and Databricks Asset Bundles are highly relevant.
- Technical Leadership – ability to design solutions, write production code, review code and guide data engineering teams.
- Financial services or insurance data, especially financial reconciliation, general ledger or accounting data.
- AI-driven data quality, anomaly detection and automated reconciliation.
- Knowledge graphs, ontologies, semantic layers or enterprise Text-to-SQL solutions.
- Azure security, data governance, PII protection and fine-grained access controls.
- MLOps/LLMOps, model evaluation and production AI monitoring.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- RI

MDM Data EngineerRecently posted
RightTalents · United States · Hybrid · Posted 19 Nov 2026 🇺🇸
MySQLOracleSQL+16TechnologySalary not listed - EX
Snowflake Data EngineerRecently posted
Excelraise LLC · United States · Hybrid · Posted 19 Nov 2026 🇺🇸
GitETLSnowflake+4TechnologySalary not listed - AG

Data Migration ConsultantRecently posted
ASCII Group LLC · Houston, TX · On-site · Posted 19 Nov 2026 🇺🇸
MySQLSQLAWS+5$80/hr - AS
Data Integration EngineerRecently posted
Abacus Service Corporation · United States · Hybrid · Posted 19 Nov 2026 🇺🇸
ETLTechnologySalary not listed - VC

Senior Kafka Engineer / AdministratorRecently posted
VAK Consulting LLC · Denver, CO · On-site · Posted 19 Nov 2026 🇺🇸
EngineeringSalary not listed - CO

Entry-Level Data EngineerRecently posted
Collabera LLC · Tampa, FL · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLPythonTechnology$22 - $24/hr - CC
Sr. Data Engineer - SDE 26-17080Recently posted
Compu-Vision Consulting, Inc. · United States · Hybrid · Posted 19 Nov 2026 🇺🇸
MySQLOracleSQL+16TechnologySalary not listed - HU

Data EngineerRecently posted
HUMAC INC. · United States · On-site · Posted 19 Nov 2026 🇺🇸
GitETLSnowflake+4TechnologySalary not listed - CA

Senior Data Engineer - Databricks
Capgemini America, Inc. · NY · On-site · Posted 2 Nov 2026 🇺🇸
SQLGitApache Spark+11Technology$133k - $153k/yr - CG
Sr. Data Engineer with Security ClearanceRecently posted
Contact Government Services, LLC · Los Angeles, CA · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLShellGit+10Technology$144.8k - $209.1k/yr - CG
Senior Database Administrator with Security ClearanceRecently posted
Contact Government Services, LLC · Dallas, TX · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLAWSVMware+4Technology$89.3k - $121.2k/yr - CG
Senior Database Administrator with Security ClearanceRecently posted
Contact Government Services, LLC · New York, NY · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLAWSVMware+4Technology$89.3k - $121.2k/yr
