Haystack
← Back to Jobs
Technology
NDNew Digital IT Inc. logo

AI Architect

New Digital IT Inc.United States🇺🇸United StatesPosted Oct 15, 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
United States
Posted
Yesterday
MicroservicesNeo4jAWSEncryptionSOC 2SnowflakeAzureBigQueryDatabricksGDPRGenerative AIGoogle CloudGraphQLHIPAAJavaKubernetesLLMPythonRESTTypeScript

Job Description

AI Architect

Full TIme Role - No Visa Sponsorship

Role Summary

We are looking for a senior, hands-on AI Architect to design, build and govern production-grade Generative AI solutions for enterprise clients. You will combine deep data and application architecture expertise with practical LLM engineering: RAG, vector search, agentic patterns and multi-model strategy across Anthropic, OpenAI and Gemini. You will own solutions end to end, from target-state architecture and ARB approval through secure, governed delivery on cloud.

Key Responsibilities

•      Define target-state AI, data and application architectures, reference patterns and reusable accelerators for enterprise GenAI programs.

•      Design and build RAG and agentic systems: ingestion, chunking, embeddings, hybrid retrieval, re-ranking, grounding, evaluation and observability.

•      Select and integrate LLMs (Anthropic Claude, OpenAI, Google Gemini) with sound model-routing, cost, latency, safety and fallback strategies.

•      Stay hands-on: write and review code, build proofs of concept, set engineering standards and unblock delivery teams.

•      Take architectures through Architecture Review Boards: prepare artifacts (ADRs, NFRs, logical and data architecture, integration flows), defend trade-offs and secure approvals on time.

•      Design security and access: RBAC/ABAC, identity integration, API management (APIM), secrets, network isolation, data classification, PII/PHI handling and audit trails.

•      Establish AI governance: prompt and model versioning, evaluation harnesses, guardrails, human-in-the-loop controls, lineage and responsible-AI practices.

•      Partner with data engineering on pipelines, semantic layers, metadata and data quality that make enterprise data AI-ready.

•      Advise client executives and engineering leaders, mentor architects and engineers, and contribute to pre-sales and solution shaping.

Required Qualifications

•      Experience: 15+ years in software, data and solution architecture, including at least 3 years designing and delivering production GenAI/LLM solutions.

•      Hands-on engineering: Strong Python (plus Java, TypeScript or similar), REST/GraphQL APIs and event-driven integration. Current in code, not only diagrams.

•      RAG and vector search: Demonstrated delivery of RAG systems with vector databases (e.g., Pinecone, pgvector, Azure AI Search, Vertex AI Vector Search, OpenSearch, Weaviate or Milvus). Retrieval quality tuning and evaluation.

•      LLM platforms: Hands-on with Anthropic, OpenAI and Gemini APIs, including tool/function calling, structured outputs, prompt engineering, context management and cost control.

•      Cloud: Production experience on Google Cloud Platform (strongly preferred), Azure or AWS: compute and containers (Kubernetes), managed AI services (Vertex AI, Azure OpenAI/AI Foundry, Bedrock), storage, serverless and CI/CD.

•      Data architecture: Data modelling, lakehouse/warehouse patterns, ingestion pipelines, metadata, lineage and data quality. Snowflake, BigQuery or Databricks experience.

•      Application architecture: Microservices, API design, integration patterns, resilience and non-functional requirements.

•      Security and governance: RBAC, identity (OAuth2/OIDC, Entra ID, IAM), APIM gateways, encryption and compliance (e.g., HIPAA, GDPR, SOC 2).

•      Architecture governance: Proven track record presenting to and securing approval from ARBs, with clear ADRs and trade-off analysis.

•      Communication: Clear written and verbal communication with executives and engineers. Experience leading without direct authority.

Preferred Qualifications

•      Agentic frameworks and orchestration (LangGraph, LangChain, LlamaIndex, MCP, Claude Agent SDK, Google ADK).

•      Knowledge graphs, ontology or GraphRAG (e.g., Neo4j).

•      LLMOps: evaluation frameworks, observability (e.g., LangSmith, Arize, Langfuse), guardrails and red-teaming.

•      Experience with AI-assisted engineering tools such as Claude Code for code generation and migration.

•      Cloud or architecture certifications (Google Cloud Platform Professional Cloud Architect, Azure Solutions Architect, AWS Solutions Architect, TOGAF).

•      Consulting or professional-services background, and experience building reusable accelerators.

•      Bachelor's or Master's degree in Computer Science, Engineering, Data Science or equivalent experience.

Similar jobs