AI Infrastructure Engineer

Position Overview:

Interested in finding system issues before they slow validation? Excited by the challenge of keeping large-scale Linux, SLURM, and GPU systems operating smoothly? This role works across Linux clusters, SLURM services, GPU infrastructure, and operational tooling that support large-scale AI and high-performance computing. We are looking for strong systems troubleshooting, sound operational judgment, and a practical approach to improving system health during deployment and validation.

Responsibilities:

• Check system health before validation by verifying node readiness, scheduler health, GPU availability, and critical cluster services.
• Support stable operation during validation by troubleshooting Linux, SLURM, GPU, network, and storage issues and by partnering with the right teams to restore healthy system behavior.
• Separate infrastructure faults from workload or test issues and turn findings into clear next steps, remediation actions, and operational guidance.
• Improve recurring issues through better diagnostics, runbooks, alerts, and targeted automation that reduces manual effort and helps teams respond more efficiently.

Qualifications:

Required Qualifications:

• 4+ years of experience in site reliability engineering, infrastructure engineering, systems engineering, or production operations in large-scale Linux environments.
• Strong Linux administration and troubleshooting skills across services, processes, networking, storage, resource contention, and system performance.
• Experience operating or troubleshooting bare-metal, HPC, or GPU clusters, including SLURM administration and an understanding of cluster health and workload execution.
• Practical Python or Bash scripting skills and familiarity with Ansible or similar tools for operational tasks.
• Experience with monitoring, telemetry, logs, dashboards, alerts, or diagnostic tooling such as Prometheus, Grafana, Alertmanager, or similar platforms.
• Demonstrated troubleshooting and remediation experience, including the ability to investigate failures, identify root causes, and restore healthy system behavior.
• A bachelor’s degree in computer science, computer engineering, electrical engineering, or a related field, or equivalent practical experience.

Preferred Qualifications:

• Experience with GPU telemetry and SLURM metrics.
• Experience with bare-metal cluster management platforms.
• Practical knowledge of bare-metal provisioning, InfiniBand, RoCE, NVLink or NVSwitch, and parallel storage failure modes.
• Experience supporting early-life cluster operations, system validation, or customer deployment environments.

Tools and Technologies:

• Linux
• SLURM
• Python
• Bash
• Ansible
• Prometheus
• Grafana
• Alertmanager