Haystack
← Back to Jobs
Technology
CT

Principal AI Engineer

CogniSoft TechnologiesNew York, NY🇺🇸United StatesPosted Sep 21, 2026

Quick Overview

Seniority
Leader
Work mode
Hybrid
Location
New York, NY, United States
Posted
Yesterday
MicroservicesRustC++JavaLLMPython

Job Description

  • 8–14 years of software engineering experience, with strong hands-on large-scale Python
  • Working depth in at least one systems or backend language — Go, Rust, Java, or C/C++ — and the judgment to know when to reach for it
  • Strong data structures and algorithms. 
  • Strong understanding of APIs, microservices, and system design
  • Hands-on experience building and operating data pipelines and production-grade distributed systems.

Agentic AI and LLMs

  • 2+ years of hands-on LLM engineering, with at least couple agentic system you designed and took to production
  • Production experience with agent frameworks — LangGraph, Google ADK, CrewAI, Claude Agent SDK, or equivalent — and the fluency to move between them as the ecosystem evolves
  • Experience building MCP (Model Context Protocol) servers and tool-calling interfaces
  • RAG from first principles: chunking strategy, embeddings, vector and hybrid retrieval, reranking, and response validation
  • Strong experience with vector databases (Milvus, Pinecone, Weaviate, FAISS, etc. or cloud equivalents)
  • Design of guardrails and reliability patterns — validators, policy checks, self-correction loops, deterministic fallbacks, circuit breakers, and rollback paths

Optimization

  • Deep familiarity with token optimization and context-window management — context shaping, pruning, and compaction
  • Latency and cost optimization through caching, model routing, batching, streaming, and parallel tool calls
  • Performance testing and tuning systems against defined SLOs

Similar jobs