Senior AI Integration Contractor

Responsibilities for this position include, but are not limited to:
Tooling Selection & Justification: Evaluate candidate open-source and commercial AI gateways (e.g., LiteLLM, Kong AI Gateway, TrueFoundry).
Platform Implementation: Deploy and configure the selected gateway on internal on-premise servers using Docker/Kubernetes.
Multi-Backend Routing: Configure the gateway to route, load-balance, and fail over across AWS Bedrock model endpoints, presented through one consistent, OpenAI-compatible API. Build the routing layer so self-hosted open models can be added later (Phase 2); only AWS Bedrock is in scope for this engagement.
AWS Bedrock Integration: Securely connect the on-premise gateway to AWS Bedrock; manage, route, and monitor all cloud-based LLM requests from a single location.
Security & Compliance for Cloud Traffic: Connect with no public internet egress; use FIPS 140-3 validated Bedrock endpoints and pin the AWS Region; enforce encryption in transit and at rest; and scrub or retention-limit model-invocation logs (CloudWatch/S3 capture prompts and responses, which are a PHI sink).
Guardrails & PII/PHI Redaction: Because AWS Bedrock is HIPAA-eligible under SNHD's Business Associate Agreement (BAA), clinical patient data (PHI) may be sent to region-pinned Bedrock endpoints for case-report generation. Use a self-hosted redaction tool (e.g., Microsoft Presidio), configured to fail closed (block the request if the redaction tool errors), as an added layer of protection. Redaction is required before sending data anywhere other than Bedrock, before writing logs, and on any de-identified/reporting data path. Bedrock's built-in Guardrails only see data after it reaches the cloud - use them for catching prompt-injection/jailbreak attempts, not as the main redactor. Note: automated redaction is confidence-based and is not 100% accurate on free-text clinical notes; treat it as one layer of protection, backed by the BAA and access controls, not a sole guarantee.
Enterprise Governance: Set up Role-Based Access Control (RBAC), organizational audit logs, and budget/rate limits to manage AI usage across multiple teams. Audit logs must be tamper-evident and retained at least 6 years (45 CFR 164.316(b)(2)).
Observability: Implement real-time monitoring of LLM traffic, latency, cost, and model behavior; Langfuse and/or Prometheus + OpenTelemetry (with Grafana dashboards), plus Bedrock AgentCore Observability, across all backends.
Prompt Management: Centralize prompt engineering workflows, enabling versioning and testing of prompts across Bedrock without application code changes.
Network Isolation: Ensure the platform is strictly contained within the internal network with no unauthorized external egress (network isolation rather than a literal air gap, since the pipeline must receive eCRs and write to EpiTrax).
Documentation & Handoff: Deliver a runbook covering operations, model onboarding, security protocols, and incident response, plus a knowledge-transfer session at contract end.
API Contract & Consumer Enablement: Within the first two weeks, publish a documented, versioned gateway API and a simple working 'mock' version of it, so the Full-Stack contractor can start building immediately without waiting for the real gateway to be finished.

To be considered for this position, you should have: [Skills, Education, or Experience]
Required Skills & Experience
LLM Orchestration Expertise: Proven experience implementing AI gateways (e.g., LiteLLM, Kong, Portkey, or similar) for enterprise model management.
AWS Bedrock Proficiency: Deep understanding of Bedrock APIs, model routing, and security configurations (IAM, PrivateLink).
Open-Model Serving Familiarity: Open-Model Serving (Phase 2, nice-to-have): Familiarity with self-hosted inference servers (vLLM, TensorRT-LLM, or SGLang) and open-model families (Gemma, gpt-oss, Nemotron), enough to advise on adding them later.
System Integration: Strong background in integrating on-premise software with cloud-based AI services.
Containerization: Mastery of Docker and Kubernetes for deploying the management platform on internal servers.
Data Privacy Standards: Expert knowledge of HIPAA/HITECH requirements, specifically PII/PHI redaction and secure data transit.
Compliance Frameworks: Familiarity with NIST SP 800-53 Rev. 5 and the heightened confidentiality protections that apply to clinical data (Nevada communicable-disease law and CDC NCHHSTP Data Security and Confidentiality Guidelines).

Nice to Have
Cost Optimization: Experience using LLM caching and routing strategies to reduce cloud inference costs.
Vector Databases: Experience with context management.
Infrastructure as Code: Terraform or Ansible for automating deployment of containerized environments.