Haystack
← Back to Jobs
Technology
IS

Site Reliability Engineer

IMR Soft LLCEast Hartford, CT🇺🇸United StatesPosted Oct 13, 2026

Quick Overview

Seniority
Mid Senior
Work mode
On Site
Location
East Hartford, CT, United States
Posted
Yesterday
API GatewayLoad BalancingDNSWAF

Job Description

Position: Site Reliability Engineer
Location: (Hartford, CT or Wellesley, MA) 3 days onsite is must
Duration: 6 months
Role Summary
The Site Reliability Engineer is responsible for preventing production incidents across the Clients Salesforce CRM and servicing platforms. This hands-on role identifies reliability risks, builds proactive detection and automation, validates high-risk changes, tests resilience, and converts incident lessons into permanent engineering improvements.
Core Responsibilities
1. Reliability Indicators and Service Health
Implement and maintain SLIs, SLOs, and dashboards for incident-relevant platform behavior.
For GPS, monitor availability, login/authentication, member-search success, CTI or screen-pop success, critical API success, and p95/p99 response time.
For YAVA, monitor call/session setup, critical transaction success, escalation-to-agent success, API dependency health, and end-to-end latency.
Measure shared dependency health across DNS, load balancing, WAF, API gateway, MQ, mainframe/DB2, cloud, telephony, and vendors.
2. Proactive Monitoring and Early Detection
Build synthetic probes for critical GPS agent and YAVA customer journeys using safe production validation patterns.
Create dashboards and alerts that identify the failing hop and correlate latency, errors, traffic, and concurrency.
Implement leading indicators for configuration drift, rising queue depth, DNS failures, certificate risk, routing mismatches, capacity saturation, and tail latency.
Validate what is actually served in production, including live certificates, DNS answers, routing, and load balancer pool membership.
Tune alerts so each alert is actionable, correctly routed, and linked to a runbook.
3. Change and Upgrade Validation
Assess technical risk for in-scope application, infrastructure, platform, security, and vendor changes.
Define and execute pre-change checks, post-change verification, and next-business-peak validation when impact may be delayed.
Verify that high-risk changes include regression evidence, rollback criteria, and tested recovery procedures.
Correlate alerts and incidents with the change timeline to distinguish planned work from unexpected regression.
Automate repeatable validation and provide self-service checks to teams making changes.
4. Dependency and Configuration Reliability
Maintain current technical dependency maps, owners, escalation paths, and critical configuration assumptions.
Continuously validate DNS, certificates, firewall/WAF behavior, load balancer health checks, API routing, MQ connectivity, database connectivity, and vendor endpoints.
Identify missing monitoring, latent configuration defects, unsupported components, and single points of failure.
Track platform lifecycle and certificate risks and drive remediation before they create incidents.
5. Resilience, Capacity, and Failure Containment
Run load, failover, recovery, and targeted chaos tests for critical paths.
Assess capacity against call volume, agent demand, transaction volume, concurrency, MQ limits, connection pools, and backend constraints.
Review retries, timeouts, circuit breakers, and graceful degradation to prevent cascading failures and backlogs.
Validate fallback options, including transfer or alternate routing when a dependent platform or vendor is unavailable.
6. Incident Analysis and Prevention
Support major-incident diagnosis with end-to-end telemetry and dependency knowledge.
Lead or contribute to blameless technical postmortems and evidence-based root cause analysis.
Ensure preventive actions are specific, testable, owned, and time-bound.
Analyze trends by change source, dependency, detection method, impact duration, and repeat failure mode.
Implement or coordinate permanent fixes and verify that corrective actions reduce the identified risk.
7. Automation and Toil Reduction
Develop automation for health checks, configuration validation, certificate checks, DNS and routing tests, post-change verification, and recurring diagnostics.
Create reusable tooling and runbooks that reduce manual error and speed triage.
Eliminate recurring operational work when engineering automation provides a safer and repeatable solution.
8. Cross-Team Execution
Partner with GPS, YAVA, application, network, security, telephony, MQ/mainframe, database, cloud, and vendor teams.
Communicate risks, findings, and recommendations clearly to both technical and leadership audiences.
Drive actions through influence and evidence while respecting ownership of the underlying platforms.
Participate in production-readiness, change, incident, and reliability review forums.

Similar jobs

Browse related jobs

Apply