AI Engineer – AI Solutions & Validation Frameworks

Job Title: AI Engineer – AI Solutions & Validation Frameworks
Location: NYC/NJ

Job Summary
We are seeking an AI Engineer with extensive experience designing, developing, validating, and implementing AI solutions for enterprise use cases. The role involves working closely with clients and internal stakeholders to translate business requirements into technical solutions, drive implementation efforts, and contribute to the development of a standardized AI validation framework that can be leveraged across multiple AI initiatives.
The ideal candidate combines deep AI engineering expertise with strong communication, stakeholder management, and problem-solving skills.

Key Responsibilities
Client Engagement & Requirement Gathering
Engage directly with client stakeholders to understand business challenges, requirements, and expectations for AI-driven solutions.
Conduct requirements-gathering sessions and translate business needs into technical specifications and solution designs.
Collaborate with business, technology, data engineering, and product teams to align on scope, approach, and deliverables.
Present AI solutions, technical approaches, validation methodologies, and implementation plans to clients and internal stakeholders.
Communicate implementation progress, risks, dependencies, and expectations throughout project delivery.
AI Solution Design & Implementation
Design, develop, and implement AI and Generative AI solutions based on business requirements.
Work with Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) architectures, AI agents, prompt engineering techniques, and related technologies.
Develop AI-powered applications supporting document intelligence, knowledge retrieval, conversational AI, intelligent automation, and decision support.
Build prototypes and proof-of-concepts (POCs), evaluate effectiveness, and support production implementation.
Integrate AI capabilities with enterprise applications, APIs, databases, and existing technology platforms.
Partner with data engineering and application development teams to deliver scalable and maintainable solutions.
AI Validation Framework Design & Implementation
Contribute to the design and development of a standardized AI validation framework applicable across multiple AI projects and solutions.
Define reusable validation methodologies, evaluation criteria, test scenarios, performance metrics, and quality benchmarks.
Develop approaches to assess AI solution accuracy, reliability, consistency, relevance, and performance.
Establish validation methodologies for LLM-based applications, including RAG systems, AI agents, document processing, and conversational AI.
Implement automated evaluation and testing processes to assess AI outputs and identify performance gaps.
Define processes for hallucination detection, response grounding, prompt evaluation, bias assessment, and robustness testing.
Collaborate with solution teams to integrate validation frameworks into AI development and deployment workflows.
Continuously enhance validation frameworks through project feedback, testing outcomes, and emerging AI capabilities.
Reusable Frameworks & Continuous Improvement
Develop reusable AI components, validation utilities, evaluation templates, test suites, and implementation accelerators.
Standardize AI validation processes and promote consistent evaluation practices across AI solution portfolios.
Identify opportunities to improve AI solutions through testing, performance analysis, and optimization.
Document solution architectures, validation methodologies, technical standards, and implementation best practices.
Collaborate across teams to encourage adoption of reusable AI frameworks and engineering standards.

Required Skills
6+ years of professional experience in AI/ML engineering, Generative AI development, or implementation of AI-driven analytical solutions.
Demonstrated experience designing, developing, and implementing AI solutions in enterprise environments.
Strong understanding of Generative AI concepts, LLMs, prompt engineering, and RAG architectures.
Hands-on experience with Python and relevant AI/ML frameworks and libraries.
Experience with LangChain, LangGraph, LlamaIndex, or similar frameworks.
Understanding of AI evaluation methodologies, testing approaches, model performance metrics, and validation techniques.
Experience working with APIs, databases, structured and unstructured data, and enterprise application integrations.
Ability to translate business requirements into technical solution designs and implementation plans.
Strong communication skills with experience interacting directly with clients and business stakeholders.
Ability to work independently, manage multiple priorities, and collaborate effectively across cross-functional teams.

Preferred Skills
Experience developing reusable AI evaluation or validation frameworks.
Familiarity with LLM evaluation tools and frameworks such as RAGAS, DeepEval, TruLens, LangSmith, or equivalent solutions.
Experience with automated testing, CI/CD integration, and monitoring of AI applications.
Knowledge of AI guardrails, hallucination detection, responsible AI practices, and output quality assessment.
Experience implementing enterprise Generative AI solutions across multiple business functions.

Qualifications
6+ years of professional experience in AI/ML engineering, Generative AI development, or related AI solution implementation roles.
Proven experience building and deploying AI solutions in business or enterprise settings.
Strong stakeholder engagement, communication, and problem-solving capabilities.
Ability to work independently while collaborating with technical and business teams.

We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, citizenship status, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable law.

_RighttoWorkPoster.pdf