Backend Engineer - AI Gateway

Job Details
Job Title: Backend Engineer - AI Gateway
Client: Korean IT Services and IT Consulting
Working Location: NJ then Plano TX
Working Style: Onsite
Employment Type: 1 year Contract / W-2
Salary: $100,000 - 130,000/year (DOE)
Benefit: Insurance and 401K March
Visa Support: No
Working Hours: 9AM to 6PM
Language: English

Position Summary
We are building an AI Gateway - a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

The Right Candidate Will Possess
A proactive, detail-oriented professional with strong communication, ownership, and problem-solving skills who can collaborate effectively across business and technical teams.

Required Qualifications & Skills
AsyncPython & Network Programming (3+yrs) - Strong hands-on experience with asyncio, aiohttp/anyio, low-level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection-pooling, keep-alive, and zero-copy buffering.
Streaming & Large-Payload Handling - Ability to manage chunked transfer, bidirectional streaming, back-pressure and high-throughput data paths for real-time LLM inference.
Protocol Knowledge - Working familiarity with HTTP/2&HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
FastAPI-Based Backend (3+yrs) - Design, develop and test production-grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
Containerization & Kubernetes - Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service-mesh (e.g., Istio/Envoy) basics.
Async Messaging & Event-Driven Design - Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back-off logic.
PostgreSQL & Vault - Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
Cloud Networking & Security - L4/L7 load balancing, TLS termination, reverse-proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let'sEncrypt).
IaC & CI/CD - Terraform modules for multi-env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue-green deployments.
Preferred Qualifications
Enterprise Security Gateways - Integration with CASB/DLP/WSS or similar edge-security platforms; centralized logging & audit trails.
IAM & SSO - Configuring Azure AD/Entra ID, Okta, SAML2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
AI/LLM Pipelines - Building prompt-engineering workflows, PII anonymization, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
API-Gateway Operations - Custom webhook/REST integrations, circuit-breaker patterns, rate-limiting, auto-retry and health-checking.
Observability & FinOps - End-to-end monitoring with Datadog, PrometheGrafana or Azure Monitor; proactive cost-optimization and 24/7 incident response (SLA/SLO).

Important Notice
Beware of scams: Cinter Technology will never contact candidates from an email address ending in @cintertechnology.work. Any such email is fraudulent.
No C2C: We do not accept Corp-to-Corp (C2C) arrangements for this position.