Haystack
← Back to Jobs
Technology
TT

Principal Technical Architect

THE TILTED CIRCLE LLCPrinceton, NJ🇺🇸United StatesPosted 10 Sept 2026

Quick Overview

Seniority
Leader
Work mode
Hybrid
Location
Princeton, NJ, United States
Posted
22 hours ago
SQLAWSOAuthAuditingAzureC#ComplianceDatabricks.NETGenerative AIGoogle CloudLLMOnboardingPythonRESTRegulatory ComplianceUnity

Job Description

Position: Principal Technical Architect - AI Systems, Data Platforms & Cyber Security
Location: Princeton, NJ (Hybrid)
Duration: Full Time

Please submit along with availability for next 3 days multiple slots after 3 PM EST

Experience Level: 15+ Years (Executive / Principal Level) 

Department: Enterprise Architecture / Data, AI & Security Engineering 

Role Overview 

We are seeking a visionary and hands-on Principal Technical Architect to lead the architecture, design, security, and strategic evolution of our Enterprise Data Platforms and Multi-Agent GenAI Systems. In this role, you will bridge the gap between complex enterprise data engineering, modern cloud architecture, cutting-edge Generative AI applications, and enterprise cybersecurity controls. 

You will design zero-touch automated data observability solutions, multi-agent AI pipelines, zero-trust data access patterns, and enterprise-wide GenAI adoption frameworks. The ideal candidate brings a deep technical background in Databricks, cloud platforms, Python, contract-driven LLM architectures, threat modeling for AI systems, and proven enterprise leadership in scaling and securing AI solutions across the organization. 

Key Responsibilities 

1. AI Systems & Multi-Agent Architecture 

  • Architect Multi-Agent AI Pipelines: Design end-to-end, LLM-powered multi-agent frameworks (deterministic + generative) using Pydantic contracts, asynchronous Python, and provider-agnostic model integration (e.g., OpenAI SDK, Databricks Model Serving). 
  • AI Tooling & Scaffolding: Build graph-based execution builders, dynamic YAML rules engines, capability registry patterns, and structured diagnostic retry mechanisms for LLM agents. 
  • Human-in-the-Loop Integration: Implement validation and curation layers that enable user DAG editing, schema validation, and error repair before scaffold generation. 

2. AI Security, Risk & Guardrails 

  • LLM Threat Modeling & Abuse Prevention: Lead abuse-case modeling, prompt injection defense, jailbreak mitigation, and red-teaming strategies for LLM agents and RAG architectures. 
  • Output Guardrails & Data Protection: Implement payload masking, custom SQL validation layers, row-count caps, and automated PII/PCI detection to prevent data exfiltration via AI interfaces. 
  • Identity & Access Governance: Architect hybrid identity flows (OAuth 2.0, Okta/Entra ID), Service Principal access patterns, and automated token lifecycle/rotation management (e.g., Delta Sharing tokens). 

3. Enterprise Data Platforms & Observability 

  • Databricks Estate Architecture: Lead large-scale data platform migrations, estate auto-discovery, and governance automation across Databricks workspaces (Unity Catalog, Workflows, Jobs API, Delta Lake, Delta Sharing). 
  • Data Governance & Zero-Trust Access: Enforce fine-grained authorization models including Row-Level Security (RLS), dynamic column masking, and centralized data classification in Unity Catalog. 
  • Data Quality & Observability Frameworks: Design automated, multi-tiered data quality verification platforms capable of real-time incident detection, automated table onboarding, and continuous file freshness tracking. 
  • Platform Governance & FinOps: Oversee multi-cloud cost governance (AWS, Azure, Google Cloud Platform), resource optimization, and infrastructure governance to maximize ROI while maintaining compliance. 

4. Enterprise GenAI Adoption & Governance 

  • Org-Wide Transformation: Define and execute adoption strategies for developer AI tooling (e.g., GitHub Copilot, custom LLM assistants) and establish measurement frameworks for code quality, productivity gains, and ROI. 
  • Enablement & Standards: Conduct technical workshops, build best-practices documentation, create reusable architectural patterns, and mentor engineering teams across divisions. 
  • Compliance & Security Auditing: Oversee access recertification, SIEM logging/auditing mechanisms, and regulatory compliance (e.g., regional data residency and vendor risk governance). 

Required Qualifications & Technical Expertise 

Professional Experience 

  • 10+ years of progressive experience in software engineering, enterprise data platforms, AI systems, and technical/security architecture within high-volume enterprise environments. 
  • Proven track record of architecting scalable solutions adopted across large organizations while maintaining high standards of data protection and zero-trust security. 
  • Experience leading enterprise-wide technology adoption programs, platform migrations, and security governance frameworks. 
  • Technical Stack & Competencies 

Key Leadership Capabilities 

  • Strategic Vision & Execution: Ability to map enterprise business requirements into robust, secure, contract-first architectural patterns. 
  • Cross-Functional Influence: Proven record of evangelizing new technologies, driving culture changes, and presenting technical strategy and cyber risk postures to executive stakeholders. 
  • Cost & Risk Optimization: Track record of driving cost efficiency and operational risk reduction while maintaining a rigorous security posture. 

Category 

Required Skills & Technologies 

AI & LLM Systems 

Multi-agent frameworks, OpenAI APIs, Databricks Model Serving, Async Python (aiohttp), Pydantic, Prompt Engineering, Streamlit 

Cyber Risk & AI Security 

LLM Threat Modeling (Prompt Injection, Jailbreaking), Guardrails, OAuth 2.0 / Entra ID / Okta, Service Principals, Delta Sharing Security 

Data Governance & Security 

Unity Catalog (RLS, Dynamic Column Masking, PII/PCI classification), Zero-Trust Access Patterns, SIEM logging & audit trails 

Data Engineering & Platforms 

Databricks (Unity Catalog, Workflows, Delta Lake, Jobs API), PySpark, Data Observability, SQL / Relational Databases 

Cloud & FinOps 

AWS, Azure, Google Cloud Platform, Cloud Security Architecture, Cloud Cost Governance / FinOps frameworks 

Languages & Core Tech 

Python (Advanced Async), C#, .NET Core, SQL, REST API Architecture, YAML rule engines 

Governance & Licensing 

Infrastructure & Licensing Governance, Enterprise Developer Tooling Administration, Token Lifecycle Management 

 

Similar jobs