Site Reliability Engineer

Hello,

Role: Site Reliability Engineer

Location: 100% Remote

Job Description:

Site Reliability Engineer OpenSearch to help ensure the highest levels of availability, performance, scalability, and Quality of Service (QoS) for mission-critical cloud services. This role will focus on the reliability, operations, automation, and continuous improvement of distributed search and analytics platforms built on OpenSearch, while working in a diverse, globally distributed team environment.

The ideal candidate brings deep experience in site reliability engineering, DevOps, cloud operations, automation, observability, and distributed systems, with proven hands-on expertise architecting, building, deploying, operating, and optimizing high-performance OpenSearch clusters and platforms from the ground up in production environments.

General Responsibilities

• Provision, build, deploy, monitor, operate, and support cloud services in a globally distributed team environment

• Architect, build, deploy, and maintain high-performance OpenSearch clusters and platforms from the ground up

• Administer and optimize OpenSearch environments for high availability, resiliency, scalability, security, and performance

• Monitor and troubleshoot cluster health, node performance, indexing throughput, search latency, shard allocation, replication, and storage utilization

• Analyze and resolve operational issues, platform instability, and production incidents across infrastructure, platform, and application layers

• Conduct incident response, root cause analysis, and post-incident remediation to drive continuous improvement

• Maintain the integrity and security of servers, systems, and OpenSearch platform infrastructure

• Support platform lifecycle activities including installation, configuration, upgrades, patching, hotfixes, backup, restore, and disaster recovery

• Develop and maintain monitoring policies, alerting standards, operational runbooks, and support procedures

• Automate testing, deployment, scaling, recovery, and operational workflows for OpenSearch and related cloud services

• Ensure proper resource allocation and capacity planning across compute, memory, storage, and network resources

• Partner with product development and engineering teams to design and enhance service reliability and operational readiness

• Develop and implement testing strategies and document results for platform changes and operational improvements

• Support log ingestion, index management, retention policies, lifecycle management, and search performance tuning

• Work in a diverse environment and cross-train with other global team members

• Participate in an on-call rotation and support weekend or after-hours operational needs as required

Requirements

• Expert with Kubernetes, including troubleshooting, operations, management, and configuration of complex Kubernetes services.
• Proven hands-on expertise designing, building, deploying, supporting, and maintaining OpenSearch clusters and platforms from scratch in production environments

• Strong experience with OpenSearch administration, cluster architecture, performance tuning, scaling, upgrades, and troubleshooting

• Experience with index design, shard and replica strategy, cluster sizing, node management, snapshot/restore, backup, and disaster recovery

• Strong understanding of distributed systems, search platforms, indexing pipelines, query optimization, and high-availability architectures

• Expertise with Git

• Expertise with Concourse, including setup, management, and troubleshooting of new pipelines

• Expertise with Linux, specifically SUSE and Ubuntu

• Expertise with Kafka, Zookeeper, and Big Data technologies

• Expert in development of automation for testing, deployment, scalability, and management of cloud services

• Expertise with building, implementing, and/or supporting cloud monitoring tools

• Expert knowledge of cloud computing, infrastructure operations, and databases

• Expert understanding of web services, networking, virtualization, and internet protocols

• Ability to multitask and handle various projects, deadlines, and changing priorities

• Excellent communication and prioritization skills

• Expertise with security fundamentals as they pertain to SaaS multi-tenant application systems

• Strong interpersonal, presentation, and customer service skills

Desired Qualifications

• Experience with AWS services including Route 53, EC2, S3, CloudWatch, DynamoDB, RDS, IAM, ACM, KMS, and VPC

• Experience deploying and operating OpenSearch in AWS-based environments

• Experience with Cloud Foundry-based environments

• Experience with Jenkins, Chef, and/or Terraform

• Exposure to and understanding of troubleshooting IP networks and application stacks

• Experience with observability tools such as Prometheus and Grafana

• Experience with log ingestion pipelines, index lifecycle management, retention strategies, and search platform security controls

• Familiarity with capacity forecasting, performance benchmarking, and resilience testing for distributed search platforms

Education

• BS/BA degree in Computer Science, Management Information Systems, or related IT discipline preferred

• Allowable substitution: An additional four (4) years of experience may be substituted for a BS/BA degree

• 8+ years of experience

Additional Requirements

• Participation in an on-call rotation for handling P1 incidents is required

• Flexible schedule which may include weekend or after-hours work

• Ability to work effectively in a diverse, collaborative, and globally distributed team environment