Haystack
← Back to Jobs
Remote
Other

Principal Platform Architect

ADDSOURCEUnited States🇺🇸United StatesPosted 13 Jul 2026

Quick Overview

Work Type
Remote
Level
Leader

Job Description

Title: Principal Platform Architect
Location: Remote (US)
Duration: Long Term
Infrastructure Self-Service Platform Engagement
Position Summary
The Principal Platform Architect owns the overall solution architecture and technical governance for a governed, multi-modal infrastructure self-service platform being delivered for a financial services client on Azure. This role is accountable for architecture decisions spanning the service catalog, Terraform module standards, the golden path GitHub Actions provisioning pipeline, multi-entry-point access (GitHub UI, API, CLI, and conversational agent), the long-term memory system (semantic and graph), and two production-grade composite Terraform patterns. The architect leads client-facing design conversations, sets technical direction for the delivery team, contributes to hands on developer and delivery and formally signs off on each milestone.
Key Responsibilities
  • Own end-to-end solution architecture across all five delivery milestones: service catalog and state backend foundation, golden path workflow and governance pipeline, entry points (GitHub Form UI, API, CLI), conversational agent, composite patterns and the long-term memory system.
  • Lead assessment of the client's existing GitHub organization, Terraform module library, Azure Blob Storage remote state, and Azure Key Vault configuration; produce standards guides, gap analyses, and prioritized remediation recommendations.
  • Define the service catalog manifest schema (YAML/JSON) and the module authoring standards (structure, variable/output contracts, tagging taxonomy) that all Terraform modules and composite patterns must follow.
  • Architect the golden path GitHub Actions workflow: stage sequence, inputs/outputs contract, OPA policy checks, security scanning gates, human-in-the-loop approval, and Terraform apply - ensuring the pipeline is reusable and entry-point agnostic.
  • Direct the identity and access architecture: GitHub OIDC federation, Azure workload identities/service principals, RBAC and least-privilege patterns, and Entra ID-based authentication for the API and CLI.
  • Guide design of the three platform entry points (GitHub Issue Forms UI, versioned REST API on Azure API Management/Functions, and CLI) ensuring all converge on the same golden path workflow without duplicated logic.
  • Architect the conversational agent integration with existing Chatbot, including tool set design (catalog lookup, parameter validation, workflow trigger, status retrieval, memory queries), multi-turn conversation handling, and guardrails against hallucination and unauthorized actions.
  • Design the long-term memory system architecture: semantic layer and knowledge graph on PostgreSQL, ontology (nodes/edges for modules, patterns, policies, deployments, approvers), and ingestion/query interfaces used by the agent and pipeline.
  • Lead the architecture and technical design of the two composite Terraform patterns (AKS-based containers; MCP integration with APIM, Azure AI Foundry, and Azure Front Door), including component selection, module wiring, and dependency mapping.
  • Serve as the primary technical escalation point for the client, participate in steering committee and governance forums, and provide milestone sign-off against each Definition of Done.
  • Review and approve OPA policy bundles, security scanning baselines, and observability/alerting design (Grafana + GitHub data source + Microsoft Teams integration) for alignment with financial services controls.
  • Mentor and provide technical direction to the Platform Engineer, Security Engineer, and Project Manager on the delivery team; enforce architectural consistency across all workstreams.
Required Qualifications
  • 10+ years in infrastructure/platform engineering with at least 4 years in a lead architect role designing and delivering cloud platform or internal developer platform (IDP) solutions.
  • 8+ years of software engineering experience, with 2+ years in senior roles shipping production systems.
  • Strong Python/GO proficiency - design expressive API contracts
  • Strong Python/GO experience for backend services, CLIs, or systems-level tooling.
  • Experience with API design - RESTful APIs, versioning, error handling, pagination, and building interfaces that developers and automation consumers depend on.
  • Experience building or working with control planes, reconciliation loops, or state machines for resource lifecycle automation.
  • Understanding of distributed systems principles - eventual consistency, failure handling, idempotency, and resilience patterns.
  • Hands-on experience with Azure and Terraform
  • Experience with Kubernetes - deploying services, understanding the resource model, and operating workloads in a cluster environment.
  • You have shipped features through the full lifecycle - design, implementation, testing, deployment, and production operation - and you have been the one answering the page when things go wrong. You understand what it takes to keep software running, not just to get it merged.
  • Deep hands-on expertise architecting solutions on Microsoft Azure, including Blob Storage, Key Vault, Entra ID, API Management, and Azure Kubernetes Service (AKS).
  • Proven track record designing Terraform-based Infrastructure as Code programs at scale, including module authoring standards, composite/reusable pattern design, and remote state management.
  • Strong experience architecting CI/CD pipelines in GitHub Actions, including reusable workflows, OIDC federated identity, environment protection rules, and approval gates.
  • Working knowledge of policy-as-code frameworks (OPA/Rego) and IaC security scanning tools (Checkov, tfsec) in a governance pipeline context.
  • Experience designing systems that integrate LLM-based conversational agents with backend automation/tooling, including guardrails, tool invocation constraints, and human-in-the-loop controls.
  • Familiarity with semantic search and knowledge graph concepts (vector embeddings, graph traversal) and experience architecting data layers that support both.
  • Demonstrated experience in a client-facing architecture role within a regulated or financial services environment, including formal milestone governance and sign-off processes.
  • Excellent written and verbal communication skills, with the ability to present architecture decisions to executive and technical stakeholders.
  • Experience with PostgreSQL extensions for vector/semantic search and graph-style queries via recursive CTEs.
  • Prior experience building or integrating with an AI agent platform (e.g., Azure AI Foundry) as an LLM backend.
  • Prior experience with AI Agent development (Azure AI Foundry)
  • Experience with Grafana observability stacks, including custom data source integration and alert routing (e.g., to Microsoft Teams).
  • Background in financial services or other highly regulated industries with strict change control and compliance requirements.
  • Experience defining OpenAPI 3.x specifications and API versioning/governance standards.
  • PostgreSQL experience - schema design, migrations, query optimization.
Core Technical Skills
  • Azure: Blob Storage, Key Vault, Entra ID, API Management, AKS, Azure AI Foundry
  • Terraform (module design, composite patterns, remote state, provider/version governance)
  • GitHub Actions (reusable workflows, OIDC, Environments, Issue Forms)
  • OPA/Rego policy-as-code; Checkov/tfsec security scanning
  • PostgreSQL (semantic/vector search, graph via CTEs)
  • REST API design (OpenAPI 3.x), CLI tooling concepts
  • Grafana observability and alerting
  • LLM/conversational agent architecture and tool-calling patterns

Skills

Azure
Compliance
GitHub Actions
Grafana
Kubernetes
LLM
PostgreSQL
Python
REST
Reconciliation
Terraform
Vault

Similar jobs