Quick overview
Job description
About the Role
Join a team working on AI model evaluation for the defense sector. You will apply operational expertise to assess how models handle military activity scenarios, helping improve the accuracy, clarity, and appropriate boundaries of their responses.
What You'll Do
Map military activity scenarios to an approved taxonomy, distinguishing conceptual and lawful policy or readiness support from operational decision support and other controlled areas.
Identify nuanced boundary cases that may be misclassified and explain the judgment behind them.
Evaluate AI-generated responses for source handling, uncertainty, reasoning quality, stated limitations, and operational utility.
Flag inappropriate refusals of valid conceptual or policy questions.
Document edge cases and disagreements, maintaining transparent guidance for model training and review.
Contribute operational, targeting, and fires expertise to ongoing project guideline refinement.
What We're Looking For
Extensive hands-on experience in joint or service-specific fires, targeting, or weapons employment decision-making, such as experience as a combat pilot, battle manager, JTAC, artillery officer, naval strike officer, targeteer, or air and missile defense specialist.
Advanced tactical training or equivalent allied training.
Teaching, leadership, or evaluation experience, such as instructor, mission commander, or debrief facilitator.
Strong written and verbal communication, with the ability to explain operational judgment clearly and succinctly.
Ability to assess targeting and fires utility, distinguish boundaries, and provide clear justifications.
Operational familiarity with the law of armed conflict or rules of engagement is a plus. AI evaluation, wargaming, or red-teaming experience is also valued.
Compensation & Benefits
$70 to $130 per hour. Visa sponsorship is not available.
Location
Remote.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- HA

Counsel, Intellectual PropertyRecently posted Auto Apply
Hadrian Automation · Los Angeles · On-site · Posted 22 Nov 2026 🇺🇸
RoboticsCopyrightIntellectual Property+2$200k - $290k/yr - GR

Chief Operating Officer (COO)Recently posted Auto Apply
Granted · New York City · On-site · Posted 22 Nov 2026 🇺🇸
ForecastingHugging Face$250k - $300k/yr - EL

Deployment Strategist, Public Sector (Federal & SLED) - North AmericaRecently posted Auto Apply
Elevenlabs · United States · Remote · Posted 22 Nov 2026 🇺🇸
Case ManagementEnterprise SalesHIPAA+3Salary not listed - PC

Del Amo - Shift SupervisorRecently posted Auto Apply
Philz Coffee · 21434 Hawthorne Blvd · Hybrid · Posted 22 Nov 2026 🇺🇸
ExpressMental Health$23 - $25/hr - CL

Founding GTM LeadRecently posted Auto Apply
Clera · San Francisco · On-site · Posted 22 Nov 2026 🇺🇸
OutreachOnboardingSalary not listed - CL

Senior Measurement StrategistRecently posted Auto Apply
Clera · United States · On-site · Posted 22 Nov 2026 🇺🇸
SQLAccount ManagementCampaign Management+1$65 - $67/hr - CL

Founding Head of ProductRecently posted Auto Apply
Clera · San Francisco · On-site · Posted 22 Nov 2026 🇺🇸
Machine LearningSalary not listed - CL

Applied AI ResearcherRecently posted Auto Apply
Clera · New York · Remote · Posted 22 Nov 2026 🇺🇸
Computer VisionDeep LearningPyTorch+1Salary not listed - BB

Electrical Lead InstallerRecently posted Auto Apply
Bell Brothers · Hayward · Hybrid · Posted 22 Nov 2026 🇺🇸
HVAC$42 - $53/hr - XD

Member of Technical Staff, Embodied CapabilitiesRecently posted Auto Apply
Xdof · San Mateo Hybrid · Hybrid · Posted 22 Nov 2026 🇺🇸
RoboticsCADLLM+2$150k - $280k/yr - AC
Electronic Document Room Supervisor – FDA Regulatory SubmissionsRecently posted Auto Apply
Akicita Cyber · Silver Spring · On-site · Posted 22 Nov 2026 🇺🇸
$78k - $90k/yr - AC
Senior Federal Records Manager (CRM Preferred) – FDARecently posted Auto Apply
Akicita Cyber · Silver Spring · On-site · Posted 22 Nov 2026 🇺🇸
$110k - $130k/yr