Haystack
← Back to Jobs
Other
TJ

Lead Platform Architect

The Jupiter GroupHouston, TX🇺🇸United StatesPosted Oct 6, 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
Houston, TX, United States
Posted
Yesterday
AWSMLOpsSOC 2AirflowBashCDKCloudFormationComplianceGitHub ActionsGitLab CIHIPAAJenkinsKubernetesPythonTerraform

Job Description

Lead Platform Engineer / Architect: AWS & Hybrid Kubernetes

About the Role

We are looking for a Lead Platform Engineer with strong AWS architecture experience to design, build, and operate scalable cloud infrastructure for AI, Data Science, and application engineering workloads. You will work with minimal supervision alongside Platform Engineers, DevOps Engineers, and DataOps Engineers, and you will be embedded with our data engineering and data science teams to move non-essential data and workloads from on-prem Kubernetes clusters to AWS. You will also help drive automation across the organization using Infrastructure as Code (IaC), Security as Code (SaC), and CI/CD best practices.

What You'll Do

  • Design and implement AWS architectures (networking, compute, storage, IAM, observability) for data and AI/ML workloads
  • Lead the migration of non-essential data and workloads from on-prem Kubernetes clusters to AWS, including planning, cutover, and validation
  • Build and maintain hybrid connectivity and data pipelines between on-prem and AWS (e.g., Direct Connect/VPN, S3, DataSync)
  • Partner directly with data engineering and data science teams to understand workload requirements and deliver self-service platform capabilities
  • Develop and maintain IaC (Terraform, CloudFormation, or CDK) and embed security and compliance controls via SaC (policy-as-code, automated scanning)
  • Design and operate CI/CD pipelines for infrastructure, data, and application deployments
  • Manage and optimize Kubernetes environments (on-prem and EKS), including cluster operations, scaling, and cost control
  • Establish monitoring, logging, alerting, and incident response practices
  • Document architecture decisions and mentor junior engineers

Required Qualifications

  • 10+ years in DevOps, platform, or cloud engineering, with significant AWS architecture experience
  • Hands-on experience running on-prem Kubernetes clusters and integrating them with AWS
  • Strong IaC skills (Terraform preferred) and experience with security/policy-as-code tools
  • Proven CI/CD experience (GitHub Actions, GitLab CI, Jenkins, or Argo CD)
  • Experience supporting data engineering or data science workloads (e.g., Spark, Airflow, SageMaker, S3/data lake patterns)
  • Proficiency in Python and/or Bash, plus Linux and networking fundamentals
  • Ability to work independently and communicate clearly with technical and non-technical stakeholders

Preferred Qualifications

  • AWS Solutions Architect (Professional) or DevOps Engineer certification
  • Experience with data migration at scale and cost optimization (FinOps)
  • Familiarity with MLOps/DataOps tooling and GitOps practices
  • Experience in regulated environments (SOC 2, HIPAA, FedRAMP, etc.)

Similar jobs