Data Engineer / AI Engineer / Data Platform Engineer

JOB DESCRIPTION:

We re looking for a senior hands-on Data & AI Platform Engineer to lead the design and engineering of an enterprise-scale AI-first data platform.

This is a hands-on leadership role we re looking for someone who can architect at enterprise scale while continuing to write, review, optimize, and troubleshoot production code.

Data Engineering & Databricks

• Expert-level Python, Scala, and PySpark
• Strong SQL and data modelling dimensional & normalized
• Deep experience with Databricks, Delta Lake, Unity Catalog, Jobs & Workflows
• Spark performance tuning, cluster/pool optimization and large-scale batch & streaming pipelines
• Enterprise medallion/lakehouse architecture
• Strong experience with ADLS Gen2, Azure Data Factory and Azure Event Hubs
• Metadata-driven and parameterized ingestion frameworks

AI / GenAI

• 3+ years of production LLM/GenAI experience
• Hands-on RAG, agentic/tool-calling workflows, structured outputs, embeddings and vector/hybrid retrieval
• Experience with LangChain, LlamaIndex or LangGraph
• Experience with Azure OpenAI, OpenAI or Databricks Model Serving
• Strong understanding of LLM evaluation golden datasets, regression testing, hallucination/accuracy tracking and human-in-the-loop workflows
• Metadata-driven AI solutions involving schema inference, data profiling and lineage

Architecture, Security & Governance

• 12+ years of experience in data engineering, data platforms or architecture
• Proven enterprise-scale lakehouse implementation experience
• Strong Azure security knowledge: Entra ID, Managed Identity, RBAC, ADLS POSIX ACLs, Key Vault and private endpoints
• Experience with Terraform, Azure DevOps and Databricks Asset Bundles
• Automated testing and CI/CD for data platforms
• Strong technical communication skills with the ability to present architecture and trade-offs to both technical and business stakeholders