Back to Jobs
Recently postedEngineering

Observability Lead Engineer with SLO

AI ASAP LLCFoster City, CA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeOn Site
LocationFoster City, CA, United States

Job description

Job Title: Staff Observability & SLO Lead

Location: Foster City, CA

Position Overview

The Staff Observability & SLO Lead will drive observability practices and service reliability across application, deployment, and infrastructure environments. The role focuses on implementing monitoring, alerting, and Service Level Objective (SLO) / Service Level Indicator (SLI) frameworks; instrumenting systems with OpenTelemetry; analyzing telemetry and distributed traces; and supporting deployment validation, incident response, and production reliability.

Key Responsibilities

· Implement and maintain monitoring, alerting, and observability solutions using Prometheus and Grafana.

· Define, implement, and support SLO/SLI frameworks to measure and improve service reliability.

· Instrument applications and services using OpenTelemetry.

· Use Honeycomb or other distributed tracing platforms to investigate service behavior and performance.

· Analyze application, deployment, and infrastructure telemetry to identify performance issues and reliability risks.

· Support canary deployment validation and provide telemetry-based input for deployment and rollback decisions.

· Troubleshoot Linux and production environment issues.

· Participate in incident management, conduct root cause analysis, and support corrective actions.

· Collaborate with engineering, operations, and reliability teams to improve observability and service health.

Required Technical Skills

· Hands-on experience with Prometheus and Grafana.

· Monitoring and alerting implementation.

· Service Level Objectives (SLOs) and Service Level Indicators (SLIs) frameworks.

· OpenTelemetry instrumentation.

· Honeycomb or other distributed tracing platforms.

· Analysis of application, deployment, and infrastructure telemetry.

· Canary deployment validation and rollback decision support.

· Linux administration and production troubleshooting.

· Incident management and root cause analysis.

Preferred Skills

· Kubernetes observability and service health monitoring.

· Reliability engineering best practices.

Experience and Work Arrangement

· 8–10 years of relevant professional experience.

· One onsite position based in Foster City, California.

Key Areas of Expertise

· Metrics, monitoring, and alerting with Prometheus and Grafana

· SLO/SLI design and service reliability measurement

· OpenTelemetry instrumentation and distributed tracing

· Telemetry analysis across applications, deployments, and infrastructure

· Canary deployment validation and rollback support

· Linux troubleshooting, incident management, and root cause analysis

· Kubernetes observability and reliability engineering

Similar jobs