Lead Distributed Systems Engineer (AI/ML Platform) - Watertown, MA

Job Description A fast-growing AI and data platform company is seeking a Lead Distributed Systems Engineer (AI/ML Platform) to architect and scale the infrastructure that powers enterprise AI and machine learning solutions. This is a highly visible technical leadership role focused on building distributed systems, production AI platforms, and cloud-native services used by large-scale customers operating in complex, regulated environments.
You will serve as a key technical authority across platform engineering, machine learning operations, and AI infrastructure. Working closely with software engineers, data engineers, and AI teams, you'll design and build the foundational systems that support model deployment, LLM applications, agent frameworks, observability, governance, and large-scale distributed workloads.
This opportunity is ideal for an engineer who enjoys solving difficult scalability challenges, driving platform strategy, and building production-grade AI systems that operate at enterprise scale. Required Skills & Experience

• 10+ years of experience building and operating distributed, cloud-native systems in production environments
• Proven experience serving as a technical lead, principal engineer, or architect for large-scale platform initiatives
• Deep expertise in distributed systems architecture, scalability, reliability, and performance optimization
• Extensive experience building and supporting production AI/ML infrastructure
• Hands-on experience deploying and operating LLM-based applications in production
• Strong understanding of Retrieval-Augmented Generation (RAG), embeddings, prompt management, and AI application architecture
• Expert-level software development experience with Python and TypeScript
• Experience with model registries, model serving platforms, and ML lifecycle management
• Strong AWS experience and knowledge of containerized environments
• Experience with Infrastructure as Code tools such as CloudFormation, CDK, or Terraform
• Experience designing secure multi-tenant platforms and handling sensitive data
• Strong background in observability, monitoring, alerting, logging, and distributed tracing
• Excellent communication skills with the ability to mentor engineers and influence technical direction

Desired Skills & Experience

• Experience with AI agent frameworks and advanced LLM orchestration
• Familiarity with Model Context Protocol (MCP) implementations
• Experience optimizing AI workloads for latency, throughput, and cost efficiency
• Knowledge of multimodal AI systems and image-based machine learning workloads
• Experience with model fine-tuning, quantization, distillation, or inference optimization
• Understanding of compliance-focused environments, including regulated industries
• Experience within life sciences, scientific software, healthcare, or laboratory data platforms

What You Will Be Doing Daily Responsibilities: 100% Hands On

• Own the architecture and technical direction of the AI/ML platform
• Design and build highly scalable distributed services supporting production AI workloads
• Develop infrastructure supporting model deployment, lifecycle management, and serving
• Create platform capabilities for LLM applications, AI agents, and retrieval-based systems
• Build reliable real-time and batch inference systems
• Define observability standards across monitoring, alerting, tracing, and performance analytics
• Implement evaluation frameworks and automated quality controls for AI applications
• Design and enforce security controls, tenant isolation, and data protection strategies
• Contribute to cloud infrastructure automation and deployment processes
• Lead architectural reviews and serve as a technical mentor for engineering teams
• Evaluate emerging technologies and drive strategic platform decisions
• Partner with cross-functional teams to improve reliability, scalability, and operational excellence

The Offer

• Competitive base salary
• Performance-based bonus opportunity
• Hybrid opportunity in Watertown, MA
• Significant architectural ownership and influence
• Opportunity to build cutting-edge AI and machine learning infrastructure
• Collaborative, engineering-focused environment with strong technical leadership

You will receive the following benefits

• Medical Insurance
• Dental Benefits
• Vision Benefits
• Paid Time Off (PTO)
• 401(k)

Applicants must be currently authorized to work in the US on a full-time basis now and in the future. Visa sponsorship is not available for this position.