AI Architect - Databricks

Job Description

We are looking for an experiencedAI Architectwith strong expertise inDatabricks, Data Engineering, Machine Learning, and Cloud-based AI solutionsto lead the design and implementation of scalable AI/ML platforms and intelligent data solutions. The ideal candidate will define enterprise AI architecture, drive modern data and AI transformation initiatives, and work closely with business, engineering, and analytics teams to deliver high-impact solutions.

This role requires a strong understanding ofend-to-end AI/ML lifecycle,Lakehouse architecture,Databricks ecosystem, and the ability to translate business use cases into production-ready AI solutions.

Key Responsibilities

• Design and implemententerprise AI/ML architectureusingDatabricks Lakehouse Platform.
• Define scalable solutions fordata ingestion, feature engineering, model training, deployment, monitoring, and governance.
• Architect and optimizeAI/ML pipelinesusingDatabricks, Spark, Python, SQL, and cloud-native services.
• Lead the setup ofMLOps / LLMOps frameworksfor model lifecycle management, CI/CD, model registry, and automated deployment.
• Work with business stakeholders to identify and prioritizeAI/ML use cases, including predictive analytics, NLP, recommendation engines, and generative AI.
• Build and guide architecture forLLM / Generative AI solutions, includingRAG, vector databases, prompt orchestration, and model integrationwhere applicable.
• Establish best practices fordata quality, security, compliance, observability, scalability, and responsible AI.
• Collaborate with Data Engineers, Data Scientists, Product Owners, and Cloud teams to ensure solution alignment with enterprise architecture standards.
• Provide technical leadership in selecting AI/ML tools, frameworks, and cloud services aligned to business and platform strategy.
• Support architecture reviews, technical design workshops, PoCs, and enterprise AI roadmap planning.

Required Skills & Experience

• 8- 15 yearsof experience inData / AI / Analytics architecture, with strong exposure to enterprise-scale implementations.
• Hands on experience withDatabricksincluding:
• Databricks Lakehouse
• Delta Lake
• Unity Catalog
• MLflow
• Databricks Workflows
• Model Serving / Feature Store(preferred)
• Strong programming experience inPython, PySpark, SQL, and AI/ML solution development.
• Solid experience in designing and deployingMachine Learning pipelinesin production.
• Good understanding ofMLOps / LLMOps, including model versioning, CI/CD, deployment, monitoring, and governance.
• Experience withcloud platforms such asAzure, AWS, or GCP, preferably with Databricks integration.
• Familiarity withGenerative AI / LLM ecosystems, such asOpenAI, Hugging Face, LangChain, vector stores, embeddings, and RAG architectures.
• Strong understanding ofdata engineering, ETL/ELT, distributed computing, and data platform modernization.
• Experience insolution architecture, technical governance, and stakeholder engagement.