AI Engineer (Databricks)

We are seeking an AI Engineer with hands-on experience designing, building, and deploying enterprise Generative AI solutions on Databricks.

The ideal candidate will have expertise in Retrieval-Augmented Generation (RAG), custom AI agents, document intelligence, vector search, evaluation frameworks, prompt engineering, and model serving.

The role involves working closely with business and technical stakeholders to develop scalable AI solutions that support real-world enterprise use cases.

Design and implement Generative AI and Agentic AI solutions.

Build and optimize RAG architectures and vector search solutions.

Develop custom AI agents and agent workflows.

Implement document parsing, information extraction, and document intelligence capabilities.

Create and maintain evaluation frameworks to assess AI solution quality, accuracy, and reliability.

Design prompts and optimize prompt engineering strategies.

Deploy and manage models using Databricks Model Serving and related MLOps capabilities.

Integrate AI solutions with enterprise data platforms and applications.

Collaborate with Data Engineers and Full Stack Developers to deliver end-to-end AI-powered products.

AI / GenAI

Retrieval-Augmented Generation (RAG)

Custom AI agents / Agentic workflows

Prompt engineering

Evaluation frameworks (LLM-as-a-Judge, RAG evaluation, AI testing)

Information extraction and AI document parsing

Vector search

Model serving and inference APIs

AI governance and quality assessment

Databricks AI/ML Platform

Databricks Model Serving

MLflow

Delta Lake

Vector Search

Unity Catalog

PySpark

Databricks SQL

Lakehouse architecture

Python

SQL

PySpark

Lang Chain / LangGraph / Semantic Kernel (preferred)

Azure and/or AWS

REST APIs

Cloud-native AI deployments

Experience building production-grade AI solutions.

Experience deploying GenAI applications in regulated or enterprise environments.

Databricks certifications (Machine Learning, GenAI, Lakehouse Fundamentals).

Experience with multi-agent systems and enterprise AI architectures.

Strong understanding of evaluation, observability, and governance of AI solutions.

The strongest match appears to be someone with a blend of:

Enterprise AI architecture experience.

Hands-on RAG and vector search implementation.

Databricks AI/ML platform expertise.

Custom AI agent development.

AI evaluation and governance frameworks.

Production deployment experience on Azure and/or AWS.