Haystack
← Back to Jobs
Technology
IT

Lead Applied AI Engineer

ISite Technologies IncNew York, NY🇺🇸United StatesPosted 27 Aug 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
New York, NY, United States
Posted
22 hours ago
FastAPIAzureGenerative AILLMPythonReact

Job Description

Any Visa

JD:

 

Lead Applied AI Engineer

Role Summary

We are seeking an accomplished Lead Applied AI Engineer to architect and deliver advanced AI systems that seamlessly integrate Generative AI capabilities, AI agents, and modern enterprise platforms.

This role is responsible for designing, building, deploying, and scaling production-grade AI solutions that support large-scale business operations while maintaining high standards of security, reliability, governance, and responsible AI practices.

The Lead Applied AI Engineer will define technical standards, lead enterprise AI adoption, establish engineering best practices, and mentor engineering teams. This position operates at the intersection of AI innovation, enterprise architecture, platform engineering, and responsible AI governance.

Key Responsibilities

AI Solution Architecture

·                Architect comprehensive end-to-end AI systems including:

o        Advanced RAG (Retrieval-Augmented Generation) pipelines

Multi-stage retrieval and re-ranking architectures
Agent orchestration frameworks coordinating multiple specialized agents
Multi-model AI integrations leveraging model-specific strengths
Design solutions with modularity, extensibility, scalability, and operational excellence to support evolving business requirements.
AI Engineering Standards & Optimization

·                Define enterprise standards for:

o        Prompt engineering

Prompt templates and versioning
Testing methodologies
Evaluation frameworks
Establish performance optimization strategies covering:
o        Model selection criteria

Caching patterns
Resource utilization
Cost optimization
Production Deployment & Reliability

·                Lead deployment of AI solutions into production environments with:

o        Comprehensive observability

Logging and tracing
Reliability engineering practices
Graceful degradation mechanisms
Circuit breaker implementation
Real-time monitoring dashboards
Automated alerting
Incident response procedures
Ensure AI services meet stringent service-level objectives and enterprise reliability expectations.
Data & Retrieval Architecture

·                Design scalable data ingestion frameworks that process:

o        Structured data sources

Unstructured documents
Real-time event streams
Develop:
o        Vector database architectures

Hybrid search capabilities
Data preprocessing pipelines
Data quality monitoring frameworks
Ensure high-quality inputs for AI systems through cleansing, enrichment, and governance processes.
AI Evaluation & Continuous Improvement

·                Establish quantitative evaluation frameworks for AI systems.

Implement:
o        A/B testing capabilities

Performance benchmarking
User feedback analysis
Telemetry-based optimization
Drive continuous improvements across:
o        Prompts

Retrieval strategies
Agent workflows
Model configurations
Platform & Infrastructure Collaboration

·                Partner with platform and infrastructure teams to ensure readiness for AI workloads, including:

o        GPU infrastructure

Model serving platforms
Feature stores
Scalable data storage
Networking infrastructure
Define requirements for enterprise AI platform capabilities and integration patterns.
Technical Leadership & Mentoring

·                Mentor engineers through:

o        Architecture reviews

Design guidance
Code reviews
Career development support
Promote engineering excellence through:
o        Best-practice documentation

Technical training
Communities of practice
Foster a culture of responsible and ethical AI development.
Responsible AI & Compliance

·                Ensure AI solutions adhere to enterprise governance and compliance requirements.

Maintain documentation of:
o        System behavior

Decision logic
Evaluation methodologies
Apply responsible AI principles including:
o        Fairness

Transparency
Accountability
Bias mitigation
Support compliance with applicable regulatory and industry requirements.
Required Qualifications

Experience

·                7+ years of software engineering experience with a strong focus on AI/ML engineering.

Proven experience building and operating distributed systems at scale.
Demonstrated success delivering AI-driven business outcomes and leading large, complex technical initiatives.
Education

·                Bachelor's degree in Computer Science, Engineering, Data Science, or a related discipline.

Equivalent practical experience may be considered.
Generative AI Expertise

·                Deep experience designing and deploying production-grade Generative AI solutions including:

o        Advanced RAG architectures

Multi-hop retrieval and reasoning systems
Agent orchestration frameworks
Tool-using AI agents
Memory-enabled AI systems
Multi-model AI architectures
Conversational AI platforms
Enterprise Solution Delivery

·                Experience leading complex AI initiatives involving multiple cross-functional teams.

Ability to translate business objectives into:
o        Technical solutions

AI architectures
Delivery roadmaps
Experience driving initiatives from concept through production deployment and optimization.
Technical Skills

Strong hands-on expertise in:

·                Python

FastAPI
React
Distributed systems
Vector databases
Embedding models
LLM APIs
Agent orchestration frameworks
Modern cloud-native architectures
AI Engineering Best Practices

Experience establishing enterprise standards for:

·                Prompt engineering

Version control and testing
AI evaluation methodologies
Model observability
Cost and performance tracking
Benchmarking frameworks
Data-driven optimization practices
Responsible AI & Governance

Strong understanding of:

·                Responsible AI principles

Model governance
Risk management
Model validation
Change management
Production monitoring
Deployment practices in regulated environments
Preferred Qualifications

·                Experience providing technical leadership across organizational boundaries.

Strong mentoring and coaching capabilities.
Demonstrated ability to collaborate effectively with:
o        Product Management

Data Science
Engineering
Security
Compliance
Architecture
Business stakeholders
Experience in healthcare, life sciences, insurance, or other regulated industries preferred.
Primary Skills for TAG Search

Must Have

·                Generative AI

Agentic AI
RAG Architecture
AI Agents / Multi-Agent Systems
Python
FastAPI
Vector Databases
LLM Integration
AI Platform Engineering
Production AI Deployment
AI Evaluation Frameworks
Prompt Engineering
Observability & Monitoring
Enterprise Architecture
Strongly Preferred

·                React

Cloud AI Platforms (Azure/OpenAI preferred)
Healthcare Domain Experience
Responsible AI / AI Governance
Distributed Systems Engineering


 Role Descriptions: AI Engineer
Essential Skills: AI Engineer
Desirable Skills:
Keyword:
Skills: AI and Automation
Experience Required: 8-10

Similar jobs