Lead Splunk Engineer with Security Clearance

Position: Lead Splunk Engineer
Location: Alexandria, VA (Hybrid)
Clearance: Active TS/SCI Required Position Overview
We are seeking a Lead Splunk & Cyber Data SME Engineer (8+ Years Experience) to lead the architectural design, sizing, deployment, and operationalization of a multi-site enterprise log aggregation and telemetry pipeline for the Test Resource Management Center (TRMC). In this leadership role, you will lead high-level technical discussions with government stakeholders and prime partners, drive baseline architectural decisions, and oversee the end-to-end implementation of central/edge Cribl Stream clusters, Splunk Indexer Clusters, Splunk Enterprise Security (ES), and archive integration.

Key Responsibilities
Architectural Design & Client Leadership: Lead program kickoff meetings, validate reference architectures, confirm platform sizing, decide central substrates (on-prem VMware vs. Azure), and lead design/implementation discussions with client leadership.

Cribl Stream Pipeline Architecture: Architect and oversee remote Cribl worker group deployments across 3 sites (6 VMs total) with edge reduction/shaping, persistent queuing for 24-hour link loss, compressed TLS transport, and central routing/replay pipelines.

Splunk Enterprise & Security Architecture: Architect Splunk Indexer Clusters (RF3/SF2 resilience) with tiered retention (hot/warm ~180 days, cold ~730 days). Deploy dedicated ad-hoc search heads and Splunk Enterprise Security (ES) search heads, baseline correlation searches, and risk frameworks.

Data Onboarding & CIM Normalization: Direct the five-gate data onboarding process for priority sources, ensuring TA configurations, field extractions, and full Splunk Common Information Model (CIM) compliance.

Archive & System Control Plane: Integrate S3-compatible or Azure Blob object stores as Cribl destinations for full-fidelity off-license archiving and on-demand replay. Establish control plane components (Cribl Leader, Deployment Server, License Manager, Monitoring Console).

As-Built Documentation & Reference Pattern: Author comprehensive as-built documentation, operational runbooks, component inventories, and reference architectures to enable repeatable deployments across future classification domains.

Required Qualifications
Clearance: Active Top Secret / SCI Experience: 8+ years of hands-on experience in enterprise Splunk architecture, data engineering, and complex cyber telemetry pipelines.

Splunk Suite: Master-level experience with Splunk Enterprise, Indexer Clustering, Search Head Deployer/Clusters, Enterprise Security (ES), IT Service Intelligence (ITSI), and SOAR.

Cribl Stream: Extensive experience with Cribl Stream architectural design, edge reduction, routing pipelines, worker groups, and archive replay.

Data Engineering: Proven track record leading data onboarding, custom field extractions, and CIM compliance.

Infrastructure: Solid expertise in virtualization (VMware) and cloud infrastructure (Azure).

Preferred Qualifications
Splunk Certified Architect or Enterprise Security Certified Admin.
Certified Cribl Stream Architect or Admin.