Quick overview
Job description
- Architecture & Lakehouse Design: Design and execute scalable, resilient Medallion Architecture (Bronze/Silver/Gold) on Azure Databricks / AWS Databricks utilizing Delta Lake and Delta Live Tables (DLT).
- Genie AI & Agentic AI Modernization: Architect and deploy Databricks Genie and Genie Agents to enable natural-language data analysis, automated querying, and self-service BI for business stakeholders. Integrate LLM tools, RAG architectures, and AI frameworks (LangChain, LangGraph, FAISS/vector stores) into data pipelines.
- Pipeline Engineering & Performance Tuning: Architect PySpark/Spark SQL streaming and batch pipelines (Auto Loader, Structured Streaming, CDC). Perform advanced Spark UI diagnostics, query optimization, Z-Ordering, partitioning, and table maintenance on large-scale datasets.
- Data Governance & Security: Architect fine-grained data security using Unity Catalog (catalog/schema hierarchies, row-level security, column masking, dynamic data masking, dynamic lineage, and ABAC). Support integrations with external governance tools such as Alation and Microsoft Purview.
- Infrastructure as Code & DevOps: Design automated deployment workflows using Terraform (Databricks provider), remote state locking, and CI/CD pipelines (Azure DevOps / GitHub Actions) across Dev, Staging, and Production environments.
- Cross-Cloud & Enterprise Integration: Lead cross-cloud data flows (ADLS Gen2, AWS S3, Kinesis, Event Hubs, DynamoDB Streams) and ensure seamless orchestration with enterprise schedulers (Airflow / Cloud Composer / Azure Data Factory).
- Technical Leadership & Mentorship: Collaborate with cross-functional business analysis, data engineering, and data science teams to translate business needs into technical Lakehouse architecture blueprints.
- Experience: 10+ years of progressive experience in Data Architecture, Enterprise Data Engineering, and Cloud Data Platform design, with at least 5–6+ years of daily, hands-on experience in Databricks and Delta Lake.
- Databricks Genie AI Expertise: Direct hands-on experience setting up, tuning, and modernizing platforms with Databricks Genie and Genie Agents for natural-language analysis.
- Generative AI / Agentic AI: Exposure to Agentic AI workflows, LangChain, RAG architectures, Vector Search, and AI-assisted tools (e.g., GitHub Copilot, Claude API, LLM tool-calling).
- Core Tech Stack:
- Databricks & Spark: PySpark, Spark SQL, Delta Live Tables (DLT), Auto Loader, Structured Streaming.
- Governance: Unity Catalog, Microsoft Purview, Alation.
- Cloud Platforms: Azure Databricks (ADLS Gen2, ADF) and/or AWS Databricks (S3, Glue, Athena, Kinesis).
- Data Modeling: Medallion Lakehouse, SCD Type 1 & 2, Star/Snowflake schemas, dbt on Databricks.
- IaC & CI/CD: Terraform (Databricks provider), Azure DevOps, GitHub, Docker.
- Certifications (Preferred):
- Databricks Certified Data Engineer Professional / Databricks Certified Machine Learning Professional
- Cloud Certifications (Azure Solutions Architect, AWS Certified Solutions Architect, or Azure Data Engineer Associate DP-203)
- Education: Bachelor's degree in computer science, Electrical Engineering, Information Technology, or related quantitative field.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- AV
Data warehouse ArchitectRecently posted
Avacend, Inc. · United States · On-site · Posted 18 Nov 2026 🇺🇸
TechnologySalary not listed - TE

Database ArchitectRecently posted
TechNix LLC · United States · Remote · Posted 18 Nov 2026 🇺🇸
WorkdayCAMERPSalary not listed - SN

Database ArchitectRecently posted
SnapCode Inc · United States · On-site · Posted 18 Nov 2026 🇺🇸
WorkdayERPSalary not listed - NS

Sr Data ArchitectRecently posted
Nava Software Solutions · Dallas, TX · On-site · Posted 18 Nov 2026 🇺🇸
Stakeholder ManagementMongoDBOracle+4TechnologySalary not listed - B&

Senior Database Architect with Security Clearance
Bart & Associates · Washington, DC · On-site · Posted 10 Nov 2026 🇺🇸
AdministrativeSalary not listed - DE
Principal AI & Data Architect (Snowflake, Semantic Modeling, Agentic AI)Recently posted
DTEL Engineering & Consultants Inc · United States · Hybrid · Posted 18 Nov 2026 🇺🇸
Stakeholder ManagementNeo4jOracle+23TechnologySalary not listed - IN
DATABASE ARCHITECTRecently posted
Infini-Tek · Athens, GA · On-site · Posted 18 Nov 2026 🇺🇸
WorkdaySalary not listed - GT
Principal AI & Data ArchitectRecently posted
Goldenpick Technologies LLC · United States · Remote · Posted 18 Nov 2026 🇺🇸
Stakeholder ManagementSQLGit+13TechnologySalary not listed - CS

Database ArchitectRecently posted
Conflux Systems Inc · Atlanta, GA · On-site · Posted 18 Nov 2026 🇺🇸
WorkdayCAM$100/hr - VS
Data Warehouse ArchitectRecently posted
Vision Square · United States · On-site · Posted 18 Nov 2026 🇺🇸
Technology$100/hr - HT
Data Architect _ Onsite in Irvine, California _ 6-12 months Contract to Hire C2H on W2 (USC/EAD)Recently posted
HDY Technologies LLC · Irvine, CA · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLETLSnowflake+2TechnologySalary not listed - AC

Data Warehouse ArchitectRecently posted
Accelyst, LLC · United States · On-site · Posted 18 Nov 2026 🇺🇸
Stakeholder ManagementTechnologySalary not listed