Why this role stands out
As a Principal AI Infrastructure & Accelerator Architect, you will define the future of AI performance, leading architectural vision for cutting-edge infrastructure and hardware-software co-design, offering significant industry-wide impact. This role is perfect for a visionary technical leader who thrives on solving complex performance challenges and shaping the technical roadmap for next-generation AI technologies. You'll have the opportunity to lead a Center of Excellence and drive innovation across a global developer community.
Quick overview
Job description
Are you a visionary technical leader passionate about defining the future of AI infrastructure and squeezing every drop of performance out of advanced hardware accelerators at scale? We are seeking a Principal Architect to lead, shape, and execute our technical strategy for AI performance, optimization, and hardware-software co-design. In this elite, highly visible role, you will define the architectural vision for both AI training and serving infrastructure, delivering massive industry-wide impact.
You will spearhead our Center of Excellence (CoE), scaling our practice and guiding the technical roadmap across next-generation Tensor Processing Units (TPUs), Graphics Processing Unit (GPU) fleets, state-of-the-art ML models, and advanced compiler toolchains. Your architectural decisions will directly enable cutting-edge AI research and large-scale production deployments across Google Cloud, major enterprise customers, and the broader open-source ecosystem.
If you thrive on solving intractable performance bottlenecks and redefining what is physically and computationally possible in AI infrastructure, this is your platform.
Req# Responsibilities Define and drive the multi-year technical roadmap for high-performance AI kernels, custom operations, and hardware-software co-design targeting TPU and GPU architectures Scale and mentor a world-class technical practice, establishing architectural governance, engineering standards, and best practices across the organization Act as the principal technical liaison partnering with ML researchers, core framework architects (JAX, PyTorch), and compiler engineering teams (XLA, MLIR) to eliminate systemic bottlenecks and shape future hardware/software requirements Architect foundational infrastructure-including enterprise-grade benchmarking suites, automated autotuning frameworks, regression analysis pipelines, and comprehensive documentation-empowering the global developer community Anticipate industry shifts by tracking advancements in hardware architectures, emerging model topologies, and compiler innovations to unlock step-changes in AI training and inference efficiency Requirements Bachelor's degree in Computer Science, Electrical Engineering, or equivalent practical experience (Master's or Ph.
D. preferred) 15+ years of software engineering experience, with 8+ years focused on distributed systems, AI infrastructure, or high-performance computing (HPC) architecture 7+ years of experience designing and developing complex software systems in C++ or Python 5+ years of experience leading the architecture, design, and delivery of large-scale software products, frameworks, or developer ecosystems from inception to production Proven track record of architecting performance-critical systems at the kernel level, bridging hardware accelerators and high-level software frameworks Nice to have Deep expertise in optimizing TPU/GPU execution, leveraging low-level kernel languages/abstractions such as Pallas, Mosaic, Triton, or CUDA Comprehensive knowledge of modern ML frameworks (JAX, PyTorch), attention mechanisms, Mixture of Experts (MoEs), model quantization, and low-precision arithmetic Advanced understanding of modern accelerator architectures, including heterogeneous compute, complex memory hierarchies, data movement optimization, and multi-node scale-out fabrics Deep familiarity with compiler principles, code generation, and modern toolchains such as MLIR, OpenXLA, and LLVM Demonstrated leadership in building and scaling developer infrastructure, widely adopted Open-Source Software (OSS) libraries, and extensible high-performance APIs Exceptional strategic communication and stakeholder management skills, with a history of influencing cross-functional engineering teams, researchers, and executive leadership
Similar jobs
- VC

Java AI/ Agentic AI EngineerRecently posted
VST Consulting, Inc · Alpharetta, GA · Hybrid · Posted 16 Nov 2026 🇺🇸
DynamoDBMicroservicesAWS+6TechnologySalary not listed - CY

AWS AI Architect | Seattle , WA or Fort Lauderdale , FL(Must be local to WA or FL) – Hybrid – Local profilesRecently posted
Cyberobotix · Seattle, WA · On-site · Posted 17 Nov 2026 🇺🇸
ShellAWSSSO+4TechnologySalary not listed - QT

AI EngineerRecently posted
QUANTUM TECHNOLOGIES LLC · United States · Hybrid · Posted 16 Nov 2026 🇺🇸
MicroservicesNLPC#+11Technology$80 - $90/hr - LS

Gen AI ArchitectRecently posted
Lincoln Softtech LLC · Dallas, TX · Hybrid · Posted 16 Nov 2026 🇺🇸
Stakeholder ManagementDockerMicroservices+14TechnologySalary not listed - IA

Forward Deployment AI EngineerRecently posted
IT Associates, Inc. · New York, NY · Hybrid · Posted 16 Nov 2026 🇺🇸
AWSSOC 2C#+6TechnologySalary not listed - AS
Enterprise IT and AI ArchitectRecently posted
Algo Soft Solutions LLC · United States · Hybrid · Posted 16 Nov 2026 🇺🇸
Microsoft OfficeEncryptionOAuth+10TechnologySalary not listed - PR
W2 - AI Engineer - Locals Only - Plano, TXRecently posted
Prohires · Plano, TX · On-site · Posted 16 Nov 2026 🇺🇸
FastAPIApache SparkDatabricks+6Technology$45/hr - TS
Senior Agentic AI Engineer Santa Clara, CA (Remote)
Triunity Software · United States · Remote 🇺🇸
SnowflakeLLMAzure+4TechnologySalary not listed - TE
GenAI Engineer - Columbia, SC, Atlanta, GA, Charlotte, NC, Raleigh, NC, Durham, NCRecently posted
TechniPros, LLC · Columbia, SC · Hybrid · Posted 17 Nov 2026 🇺🇸
FastAPIFlaskSQL+16TechnologySalary not listed - BC

Forward Deployed AI Engineer, United States - BCG XRecently posted
Boston Consulting Group · Oakland, California · Hybrid · Posted 17 Nov 2026 🇺🇸
PythonData AnalysisTechnology$120k/yr - OT
Senior AI EngineerRecently posted
Oscar Technology · United States · Remote · Posted 17 Nov 2026 🇺🇸
AWSPythonTechnology$150k - $180k/yr - QT

Azure AI EngineerRecently posted
QUANTUM TECHNOLOGIES LLC · Dallas, TX · Hybrid · Posted 17 Nov 2026 🇺🇸
DockerGitMLOps+11Technology$73/hr
