Haystack
← Back to Jobs
Technology

Gen AI Architect

Shiv Software Experts LLCBridgewater, NJ🇺🇸United StatesPosted 9 Jul 2026

Quick Overview

Work Type
Hybrid
Level
Mid Senior

Job Description

Gen AI Architect

Full Time only

Experience: 12 to 15 Years

We are seeking a highly experienced Agentic AI / Generative AI Architect with 15+ years of software and solution architecture experience, combined with handson expertise in Data Science, Machine Learning, and modern agent-based AI systems. This role requires deep technical leadership in designing, implementing, and governing advanced multi-agent AI ecosystems, RAG pipelines, cloud-native AI platforms, and GenAI engineering practices. The ideal candidate will drive endtoend solution architecture for enterprise-grade AI applications while ensuring scalability, robustness, security, and operational excellence.

What You''''ll Do: 

  • Design & Implement Agentic AI Systems
  • Architect and build multi-agent, goal-driven, autonomous AI systems using frameworks such as:
    • AutoGen
    • LangGraph
    • CrewAI
  • Create intelligent agent ecosystems supporting orchestration, reasoning, and collaborative task execution.
  • Prompt Engineering & LLM Expertise
  • Apply advanced prompt engineering techniques including:
    • Few-shot prompting
    • Chain-of-thought reasoning
    • Prompt templates
  • Optimize prompt flows for deterministic, scalable LLM-driven systems
  • Cloud-Native AI Architecture
  • Design and deploy AI/LLM systems on cloud platforms such as AWS Bedrock, Azure OpenAI, Google Vertex AI, etc.
  • Ensure solutions meet enterprise NFRs including performance, security, cost-optimization, and availability.
  • RAG Pipelines, Vector Databases & MCP
  • Architect and deploy RAG pipelines using vector databases such as:
    • Pinecone
    • Weaviate
    • ChromaDB
    • FAISS
  • Implement MCP Servers and Agent-to-Agent (A2A) communication frameworks.
  • LMOPs / GenAIOPs
  • Implement end-to-end operational pipelines for GenAI applications including:
    • Continuous integration & deployment
    • Model monitoring & drift detection
    • Logging, observability, and troubleshooting mechanisms
  • Establish governance models, reusable patterns, and GenAI best practices.
  • Application & Microservices Architecture
  • Design microservices-based systems using Spring Boot, REST APIs, and secure API design patterns.
  • Implement API security, versioning, and distributed system governance.
  • Architect cloud-native applications using AWS/Azure/Google Cloud Platform, Spring Cloud, PCF, or equivalent.
  • Collaboration & Leadership
  • Work closely with Data Scientists, Product Owners, Business SMEs, and Engineering teams.
  • Lead end-to-end solution architecture for enterprise AI initiatives.
  • Conduct technical presentations, architectural reviews, and stakeholder communication.

 

Expertise You''''ll Bring:

  • 5+ years in software/solution architecture.
  • Proven experience as a Data Scientist or ML Engineer with exposure to agentic AI systems.
  • Experience designing multi-agent systems using AutoGen, LangGraph, CrewAI, etc.
  • Strong understanding of cloud AI platforms (Bedrock, Azure OpenAI, Vertex AI).
  • Hands-on experience with AI Code Assist tools such as:
    • GitHub Copilot
    • Windsurf
    • Cursor
    • AWS Q
  • Expertise in Vector Databases, RAG pipelines, MCP, and multi-agent communication.
  • Strong proficiency in Python (preferred), and optionally Java/Node.js.
  • Experience with microservices, Spring Boot, REST APIs, API security, and versioning.
  • Proficiency in Docker, Kubernetes, CI/CD pipelines.
  • Strong grasp of design patterns and architecture principles.
  • Deep understanding of cloud-native design and distributed systems.
  • Experience designing AI systems that meet NFRs: scalability, security, performance, maintainability.
  • Exceptional communication and presentation skills.
  • Ability to articulate complex AI concepts to technical and non-technical audiences.
  • Strong leadership, problem-solving mindset, and strategic thinking abilities.
  • Ability to collaborate with cross-functional teams to translate business needs into AI-powered solutions.

Skills

Docker
Microservices
Node.js
Spring
Spring Boot
AWS
Machine Learning
Azure
Generative AI
Google Cloud
Java
Kubernetes
LLM
Python
REST

Similar jobs