Haystack
← Back to Jobs
Remote
Technology
SA

Backend software engineer

SaltUnited, WV🇺🇸United StatesPosted Sep 30, 2026

Quick Overview

Seniority
Mid Senior
Work mode
Remote
Location
United, WV, United States
Posted
16 hours ago
DockerNode.jsSQLMachine LearningKubernetesPyTorchPython

Job Description

Job Description

OPPORTUNITY: Back End Engineer, AI Systems - NA (fully remote)

\n\n

About the Opportunity

\n

We are partnering with a fast-growing technology company building a new generation of AI-native applications designed to make everyday tasks, communication, organization, and workflows more intelligent and intuitive. The team is focused on creating proactive AI experiences that require minimal prompting while maintaining high levels of reliability, persistent context, and real-world task completion.

\n

\n

The Role

\n

As a Backend Engineer – AI Systems, you will help own the inference and orchestration layer powering AI interactions across the platform. Sitting at the intersection of AI models, backend infrastructure and end users, you will build production systems that transform model capabilities into fast, stable and observable services used across mobile and desktop applications.

\n

You will be responsible for designing and operating systems where latency, scalability, reliability, correctness and cost have a direct impact on the end-user experience.

\n

\n

What You'll Be Doing

\n
    \n
  • Build and operate backend systems supporting AI-powered features in production.
  • \n
  • Design scalable inference pipelines, orchestration layers and service boundaries around AI models.
  • \n
  • Develop reliable APIs that integrate effectively with frontend and machine learning systems.
  • \n
  • Optimize latency and throughput across inference, caching, batching and streaming.
  • \n
  • Own key production concerns including monitoring, logging, alerting and incident response.
  • \n
  • Troubleshoot and resolve complex issues across distributed systems operating under load.
  • \n
  • Continuously improve system performance and reliability based on real-world usage and production data.
  • \n
  • Work closely with engineering and AI/ML teams to bring new AI capabilities into production.
  • \n
\n

\n

What We're Looking For

\n
    \n
  • Strong backend engineering fundamentals with experience building and operating production systems.
  • \n
  • Experience developing high-throughput, low-latency services at scale.
  • \n
  • Familiarity with AI inference patterns including LLMs, embeddings and multimodal models.
  • \n
  • Experience working with distributed systems and debugging complex production environments.
  • \n
  • Strong understanding of API design, system reliability and performance optimization.
  • \n
  • Comfortable working in a fast-moving environment where experimentation and iteration are encouraged.
  • \n
  • A hands-on mindset with a strong bias toward shipping, measuring and improving production systems.
  • \n
\n

\n

Technology Environment

\n

Experience across some of the following would be highly relevant:

\n
    \n
  • Python
  • \n
  • Node.js
  • \n
  • PyTorch
  • \n
  • Commercial and open-source LLMs
  • \n
  • SQL and NoSQL databases
  • \n
  • Kubernetes
  • \n
  • Docker
  • \n
  • AI inference and orchestration frameworks
  • \n
  • Distributed systems and cloud infrastructure
  • \n
\n

Similar jobs