Back to Jobs
Recently postedOther

Data Bricks Architect

Arrowminds incSt. Louis, MO🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeOn Site
LocationSt. Louis, MO, United States
DockerDynamoDBSQLGitApache SparkAWSSnowflakeAirflowBusiness AnalysisDatabricksGitHub ActionsLLMTerraformUnitydbtGenerative AIAzureMachine LearningData AnalysisCI/CDAzure DevOpsAzure Data FactoryDelta LakeLangChainLangGraphRAG

Job description

Databricks Architect with Databricks Genie / Genie AI
Location: St. Louis, MO (Hybrid / On-site as per client requirement)
Role Type: Full-Time / Contract
Job Description & Position Overview
TIXY is seeking an experienced Databricks Architect with specialized hands-on expertise in Databricks Genie AI / Genie Agents to lead the architecture, design, and AI modernization of our enterprise data platform in St. Louis, MO.
In this role, you will lead the implementation of modern multi-cloud Lakehouse solutions (Azure/AWS), drive enterprise data governance using Unity Catalog, and pioneer AI-ready data capabilities. You will enable natural-language analytics across enterprise datasets by deploying Databricks Genie and integrating Agentic AI capabilities (LangChain, LangGraph, LLM integration) directly into modern data workflows.
Key Responsibilities
  • Architecture & Lakehouse Design: Design and execute scalable, resilient Medallion Architecture (Bronze/Silver/Gold) on Azure Databricks / AWS Databricks utilizing Delta Lake and Delta Live Tables (DLT).
  • Genie AI & Agentic AI Modernization: Architect and deploy Databricks Genie and Genie Agents to enable natural-language data analysis, automated querying, and self-service BI for business stakeholders. Integrate LLM tools, RAG architectures, and AI frameworks (LangChain, LangGraph, FAISS/vector stores) into data pipelines.
  • Pipeline Engineering & Performance Tuning: Architect PySpark/Spark SQL streaming and batch pipelines (Auto Loader, Structured Streaming, CDC). Perform advanced Spark UI diagnostics, query optimization, Z-Ordering, partitioning, and table maintenance on large-scale datasets.
  • Data Governance & Security: Architect fine-grained data security using Unity Catalog (catalog/schema hierarchies, row-level security, column masking, dynamic data masking, dynamic lineage, and ABAC). Support integrations with external governance tools such as Alation and Microsoft Purview.
  • Infrastructure as Code & DevOps: Design automated deployment workflows using Terraform (Databricks provider), remote state locking, and CI/CD pipelines (Azure DevOps / GitHub Actions) across Dev, Staging, and Production environments.
  • Cross-Cloud & Enterprise Integration: Lead cross-cloud data flows (ADLS Gen2, AWS S3, Kinesis, Event Hubs, DynamoDB Streams) and ensure seamless orchestration with enterprise schedulers (Airflow / Cloud Composer / Azure Data Factory).
  • Technical Leadership & Mentorship: Collaborate with cross-functional business analysis, data engineering, and data science teams to translate business needs into technical Lakehouse architecture blueprints.
Required Qualifications & Skills
  • Experience: 10+ years of progressive experience in Data Architecture, Enterprise Data Engineering, and Cloud Data Platform design, with at least 5–6+ years of daily, hands-on experience in Databricks and Delta Lake.
  • Databricks Genie AI Expertise: Direct hands-on experience setting up, tuning, and modernizing platforms with Databricks Genie and Genie Agents for natural-language analysis.
  • Generative AI / Agentic AI: Exposure to Agentic AI workflows, LangChain, RAG architectures, Vector Search, and AI-assisted tools (e.g., GitHub Copilot, Claude API, LLM tool-calling).
  • Core Tech Stack:
    • Databricks & Spark: PySpark, Spark SQL, Delta Live Tables (DLT), Auto Loader, Structured Streaming.
    • Governance: Unity Catalog, Microsoft Purview, Alation.
    • Cloud Platforms: Azure Databricks (ADLS Gen2, ADF) and/or AWS Databricks (S3, Glue, Athena, Kinesis).
    • Data Modeling: Medallion Lakehouse, SCD Type 1 & 2, Star/Snowflake schemas, dbt on Databricks.
    • IaC & CI/CD: Terraform (Databricks provider), Azure DevOps, GitHub, Docker.
  • Certifications (Preferred):
    • Databricks Certified Data Engineer Professional / Databricks Certified Machine Learning Professional
    • Cloud Certifications (Azure Solutions Architect, AWS Certified Solutions Architect, or Azure Data Engineer Associate DP-203)
  • Education: Bachelor's degree in computer science, Electrical Engineering, Information Technology, or related quantitative field.

Will your resume get past this employer's screening? Check your resume against this job, free

Similar jobs