Quick overview
Job description
AI Platform Engineer - REMOTE
Role summary
Build and operate a production agentic AI platform: orchestration, model gateway, agent registry, retrieval, guardrails, evaluation and observability. You build the layer other engineers build on, and you own its reliability, security and cost.
Core technical skills
- LLM and agent orchestration: LangGraph, LangChain, LlamaIndex, CrewAI, AutoGen, or the Google Agent Development Kit (ADK), with production multi-agent orchestration, state management and retry/fallback design.
- Model serving and gateway: LiteLLM, Portkey, Kong AI Gateway, TrueFoundry, or Vertex AI Model Garden; routing, fallback, rate limiting and token/usage capture. Serving open-weight models (Llama, Mistral, Gemma, Qwen) via vLLM, TGI, Ollama or Vertex endpoints; fine-tuning and adapter techniques (LoRA/QLoRA).
- Retrieval-augmented generation: vector stores (Vertex Vector Search, Pinecone, Weaviate, pgvector, Elasticsearch), embedding models, chunking, hybrid and semantic search, reranking, and grounding-quality evaluation.
- Agent infrastructure: Model Context Protocol (MCP) and tool/function-calling contracts, agent registries, permission enforcement, and the separation of reasoning from acting agents.
- Evaluation and LLMOps: eval harness design with RAGAS, DeepEval, Promptfoo, or OpenAI Evals; offline and online evaluation, regression suites, drift and hallucination detection; experiment tracking with MLflow, LangSmith, Langfuse, Arize Phoenix or Braintrust.
- Guardrails and AI security: NeMo Guardrails, Guardrails AI, Model Armor, Lakera; prompt-injection, jailbreak and data-leakage defense; PII detection and redaction in prompt and response.
- Observability: OpenTelemetry-based tracing for LLM and agent calls, token and latency and cost telemetry, quality-signal monitoring feeding alerting.
- Cloud and runtime: Kubernetes and containers, Google Cloud Platform preferred (GKE, Vertex AI, Cloud Run), infrastructure-as-code (Terraform), CI/CD, and SRE practice for production AI services.
- Languages: strong Python; Go or Java useful for platform services.
Experience
Typically 6+ years in software or platform engineering, with hands-on production ownership of AI or ML infrastructure. Has personally built and operated agentic or LLM systems in production, not only prototyped them.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- KI

Lead DevOps and SAP S/4HANA Public CloudRecently posted
Kanshe Infotech · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
CI/CDTechnologySalary not listed - IT
Devops EngineerRecently posted
ITBMS Inc. · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerMySQLRuby+6TechnologySalary not listed - IN
Cloud infrastructure Engineer/ Devops EngineerRecently posted
InfoPeople Corp · White Plains, NY · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerAWSAnsible+7TechnologySalary not listed - CT
Senior .NET & Angular Developer (Azure DevOps) - HYBRID (Locals to MI)Recently posted
Chandra Technologies, Inc. · Lansing, MI · Hybrid · Posted 20 Nov 2026 🇺🇸
Angular.NETPlaywright+4TechnologySalary not listed - CA

Senior Devops EngineerRecently posted
Capgemini America, Inc. · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerDynamoDBSQL+16Technology$101.1k - $182.6k/yr - CA

Senior Infrastructure Engineer - Site Reliability Engineer (SRE)Recently posted
Capgemini America, Inc. · MA · Hybrid · Posted 16 Nov 2026 🇺🇸
AWSAPI GatewaySplunk+13Technology$83.0k - $135.3k/yr - CC
Senior Databricks Platform Engineer (Terraform / AWS / GitLab CI/CD) - SPE 26-17092Recently posted
Compu-Vision Consulting, Inc. · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
GitAWSDatabricks+5TechnologySalary not listed - WO

Software Development Engineer, SRE (US Federal) with Security ClearanceRecently posted
Workday · Reston, VA · Hybrid · Posted 18 Nov 2026 🇺🇸
Technology$137k/yr - CT

DevOps with .NetRecently posted
Conch Technologies · Charlotte, NC · Hybrid · Posted 20 Nov 2026 🇺🇸
Git.NETPython+1TechnologySalary not listed - ID

Senior Data Platform EngineerRecently posted
IDR, Inc. · Atlanta, GA · Hybrid · Posted 16 Nov 2026 🇺🇸
SQLAzureSQL ServerTechnologySalary not listed - AI
DevSecOps EngineerRecently posted
Aasritha infotech Inc · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
SAFeAgileEngineeringSalary not listed - CT
Senior Google Cloud Platform DevOps / SRE Engineer on W2Recently posted
Chabez Tech LLC · Dallas, TX · On-site · Posted 20 Nov 2026 🇺🇸
EncryptionApacheBigQuery+5TechnologySalary not listed