AI Software Engineer

This is a software engineering role on a production AI system. You won''t be training or fine-tuning models or running model-science experiments -  our data science team owns that. But unlike a generic application role, prompt engineering, retrieval quality, and evaluation discipline are core to this job, not someone else''s problem. The interesting work lives in the orchestration, the prompt lifecycle, retrieval, and evals -  not CRUD.

What the role looks like at this level

As a Software Engineer II on this team, you''ll break down medium-sized features, estimate them, and cut scope to ship on time. You''ll start to own tasks within the service with support from senior teammates, contribute to technical design and engineering-review proposals while thinking through failure cases, give helpful and timely code reviews, and defend your decisions in review. You''ll debug to root cause in your area, instrument your code for operations, and participate in the on-call rotation. Senior engineers are around to pair with and review your work

-  but increasingly you''ll be the one proposing the approach and carrying a feature to production.

What you must already bring

You don''t need every line below at expert depth, but the combined surface has to be covered.

Core engineering

. Expert-level async Python (3.11+). Real production asyncio / async / await experience across the request path - a synchronous-only Python background won''t be enough here.

·      FastAPI at depth: routers, dependencies, lifespan, middleware. Pydantic v2 and disciplined type hints.

·      pytest and pytest-asyncio - fixtures, async, mocking, and meaningful coverage. Standard formatting, linting, and type-checking tools are table stakes.

AI / LLM systems - the heart of the work

·      Hands-on production experience with LangGraph: state machines, conditional edges, checkpointing. Experience with LangChain alone is not the same thing - this is where most of the surface area lives.

·      LangChain core (messages, runnables, tools), and prompt engineering/ prompt lifecycle management - versioned, environment-tagged prompts with local overrides - using tracing and experiment tooling such as LangSmith .

·      Multi-agent/ multi-node workflow design - routing across specialized agents and nodes.

·      RAG with hybrid vector + lexical retrieval, and experience with a managed LLM provider such as Azure OpenAI (deployments, API versions, quotas).

·      Sound instincts for non-determinism, token budgets, timeouts, and graceful degradation,

plus familiarity with eval frameworks (e.g. LLM-as-judge and regression evals).

Data, infrastructure, and delivery

. PostgreSQL operationally - indexing, connection pools, poolers - plus pgvector and

OpenSearch/Elasticsearch hybrid (text + KNN) search.

. AWS and Kubernetes in production -  genuine fluency, beyond local container orchestration. Docker multi-stage builds; infrastructure-as-code (e.g. Terraform) and manifest overlays for multiple environments.

. Multi-environment configuration discipline -  several environments, from local through production, each with its own secrets, prompts, and resources.

And comfortable with

·      Typescript and modern Angular with RxJS when frontend work is needed. A backend-leaning candidate is welcome as long as you''re comfortable in Angular; a frontend-leaning candidate must still be solid in the Python/LLM stack.

Nice to have {genuine bonuses, none required)

·      MCP (Model Context Protocol) and SSE; database migration tooling; Redis-compatible caches.

·      Observability tooling (APM, metrics, tracing) and distributed-tracing concepts.

·      Modern Python packaging and build tooling, Make-based builds, GitHub Actions, private package registries, and encrypted-secrets workflows.

·      Load testing and end-to-end browser testing frameworks.

·      Edtech / K-12 domain awareness (standards, proficiency, learning frameworks) and FERPA-adjacent data-privacy thinking.

·      Familiarity with large-enterprise internal identity, auth, and content-metadata services - accelerates ramp, but learnable.