
Cybersecurity RL Environment Reviewer — Security Engineering (Defensive, Cloud & Infrastructure)
Quick overview
Job description
Cybersecurity RL Environment Reviewer — Security Engineering (Defensive, Cloud & Infrastructure)
Part-Time, Remote
About idlexperts
idleXperts is idler's quality assurance network — the part-time, remote contributors ("experts") who review idler's reinforcement-learning environments before they're delivered to customers. Experts are the last check between what engineering builds and what a customer receives: they confirm a task is solvable as given, confirm the grading actually tests the right thing, and flag anything that would make the task unfair or ambiguous to an AI model attempting it.
About the Role
We're building a dedicated bench of cybersecurity reviewers to QA our defensive-security and infrastructure-security RL environments: detection-engineering tasks, incident-response and root-cause-analysis tasks, cloud/IAM misconfiguration tasks, and digital-forensics tasks. You'll work through environments we've already built and tell us whether they're fair, correct, and ready to ship.
Part-time, flexible-hours contract work; we'll scope weekly workload directly with you.
What You'll Do
Review a described incident, misconfiguration, or detection gap and confirm it's realistic, reproducible from the given logs/environment, and solvable by a skilled practitioner using only the provided context.
For detection/remediation tasks, confirm the grading actually distinguishes a correct fix from an incorrect or incomplete one (e.g., a rule that only coincidentally matches the sample data).
For forensics tasks, confirm the planted artifact is genuinely recoverable through sound method, not by guessing.
Flag inconsistent files, missing context, unclear requirements, or anything that would make the task unsolvable or ambiguous.
Write a short, clear justification for every decision.
What We're Looking For (Minimum Qualifications)
Hands-on experience in at least one of: SOC/detection engineering, incident response, cloud security (AWS/GCP/Azure IAM and misconfiguration), or digital forensics.
Comfortable working with logs, cloud consoles/CLIs, and a common detection-rule language (e.g., Sigma, KQL, or a SIEM's native query language), or standard forensic tooling.
Able to clearly explain, in writing, why a finding is or isn't realistic and correctly graded.
Reliable and self-directed, comfortable working async with limited oversight.
Nice to Have
A certification such as GCIH, GCFA, GREM, CISSP, or a cloud-security credential (e.g., AWS Security Specialty).
Experience building or tuning detection content professionally, not just consuming it.
CTF experience in forensics/misc categories, or past blue-team competition experience (e.g., CPTC, collegiate cyber defense).
Compensation
$65–$100/hour — solid, demonstrable experience in one of the above areas, able to independently verify straightforward tasks.
$125–$250/hour — deep specialist experience (5+ years, advanced certifications, or a track record of building production detection/IR capability), able to handle ambiguous or disputed calls.
How to Apply
Share your resume/LinkedIn plus any relevant certifications, write-ups, or portfolio material. We'll follow up with a short, paid sample review (one task, roughly 30–45 minutes) before starting regular work.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- RM
Information Systems Security OfficerRecently posted Auto Apply
Resource Management Concepts, Inc. · Charleston, South Carolina · On-site · Posted 20 Nov 2026 🇺🇸
AWSEncryptionPKI+3TechnologySalary not listed - OK

Deputy Chief Information Security Officer (CISO), USRecently posted Auto Apply
OKX · New York · Hybrid · Posted 20 Nov 2026 🇺🇸
Administrative$250.4k - $375.7k/yr - AI

Senior System Safety EngineerRecently posted Auto Apply
Anduril Industries · Costa Mesa · Hybrid · Posted 20 Nov 2026 🇺🇸
ANSYSJiraEngineering$166k - $220k/yr - AI

Senior Cyber Software EngineerRecently posted Auto Apply
Anduril Industries · Irvine · Hybrid · Posted 19 Nov 2026 🇺🇸
RustEncryptionAnsible+10Technology$191k - $253k/yr - DE

Senior Manager, Identity and Access ManagementRecently posted Auto Apply
Delinea · U.S. Remote · Remote · Posted 19 Nov 2026 🇺🇸
AWSMFAOAuth+13Salary not listed - MI

Senior Security EngineerRecently posted Auto Apply
Mixpanel · San Francisco · Hybrid · Posted 19 Nov 2026 🇺🇸
GCPGitAWS+16Technology$230k - $300k/yr - MS

Security Licensed Officers RialtoRecently posted Auto Apply
Marksman Security LLC · Rialto · Hybrid · Posted 19 Nov 2026 🇺🇸
LMS$20/hr - KH

Senior Security Engineer, Product SecurityRecently posted Auto Apply
Komodo Health · United States · Hybrid · Posted 19 Nov 2026 🇺🇸
GCPRustGit+11Technology$200k/yr - AS
Program Security RepresentativeRecently posted Auto Apply
Analytic Services Inc · Falls Church · Hybrid · Posted 19 Nov 2026 🇺🇸
ComplianceSAP$99k - $144k/yr - RE
Director, Security and ComplianceRecently posted Auto Apply
Rebuy · United States · Remote · Posted 19 Nov 2026 🇺🇸
RecruitingGCPOWASP+10$175k - $200k/yr - DE

Security Analyst III (Afternoons)Recently posted Auto Apply
Deepwatch · Tampa · Remote · Posted 19 Nov 2026 🇺🇸
SplunkJiraLinuxTechnology$95k - $105k/yr - PH

Staff Platform & Security EngineerRecently posted Auto Apply
Project Healthy Minds · New York · On-site · Posted 19 Nov 2026 🇺🇸
SQLSpringGit+10Technology$220k - $260k/yr