Quick overview
Job description
We are seeking experienced US registered nurses to review and assess AI-generated responses from a health coaching agent, with a focus on safety, regulatory compliance, and clinical content accuracy.
You will work asynchronously on a dedicated review platform, performing short, structured evaluations to help train and refine AI models for patient-facing health guidance.
This is a remote, flexible and short-term project.
Key Responsibilities:
Structured Grading: Score AI agent responses against a pre-defined rubric, applying the highest-priority criteria consistently across a large volume of items.
Safety Review: Identify responses that could cause patient harm missed red flags, unsafe advice, failure to escalate or signpost to care.
Clinical Accuracy: Assess factual correctness, completeness and guideline alignment of clinical content.
Written Feedback: Provide brief, specific rationale for each failed response and general guidance on how it could be improved.
Qualifications
Licensure: Active RN license for US
Experience: Minimum 2 years of clinical nursing practice.
Knowledge: Strong grounding in clinical guidelines, patient safety and scope-of-practice boundaries.
Attention to detail: Proven accuracy and consistency in clinical documentation or review work.
Technical proficiency: Comfortable using structured review tools and digital platforms.
Language: Native or near-native written English.
Legal Status
You will have the right to work in your country of residence.
You will work as an independent contractor.
Why Join Us?
Flexible Work Arrangements: Part-time, remote and fully asynchronous.
Competitive Compensation: Hourly rate
Professional Development: Hands-on experience in AI evaluation and model fine-tuning, shaping how AI is deployed in patient care.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- NR

Registered Nurse (RN) - Full-Time Swing ShiftRecently posted Auto Apply
Northpoint Recovery Holdings, LLC · Loveland · Hybrid · Posted 12 Nov 2026 🇺🇸
Microsoft OfficeComplianceHIPAA+4Healthcare$33 - $52/hr - MG
Patient Monitoring Coordinator (RN) Auto Apply
MedElite Group, LLC. · Hollywood · Hybrid · Posted 5 Nov 2026 🇺🇸
NursingHealthcare$29 - $36/hr - AB
Registered Nurse (RN), Home Care - PRNRecently posted Auto Apply
AbriCare, Inc. · North Kansas City · Hybrid · Posted 12 Nov 2026 🇺🇸
OnboardingBLSCare Coordination+3HealthcareSalary not listed - PP

Charge Nurse | Registered Nurse (RN)Recently posted Auto Apply
Ppgreatplains · Wichita · On-site · Posted 11 Nov 2026 🇺🇸
Microsoft ExcelACLSBLS+6Healthcare$42 - $46/hr - CH
Dispensary RNRecently posted Auto Apply
Camas Health Recovery Center · Airway Heights · Hybrid · Posted 11 Nov 2026 🇺🇸
Microsoft OfficeComplianceNursingHealthcare$45 - $55/hr - AH
Admissions Registered Nurse (RN)Recently posted Auto Apply
ANX Home Healthcare & Hospice Care · Burlingame · Hybrid · Posted 11 Nov 2026 🇺🇸
NursingSchedulingHealthcare$118k - $142k/yr - GR

Registered Nurse (RN), Pre-Operative/PACU/OR - Full TimeRecently posted Auto Apply
Grand Rapids Ophthalmology · Grand Rapids · Hybrid · Posted 11 Nov 2026 🇺🇸
Microsoft ExcelMicrosoft PowerPointACLS+4HealthcareSalary not listed - GR

Registered Nurse (RN) - Full TimeRecently posted Auto Apply
Grand Rapids Ophthalmology · Grand Rapids · Hybrid · Posted 11 Nov 2026 🇺🇸
Microsoft ExcelMicrosoft PowerPointACLS+4HealthcareSalary not listed - GI
Low Acuity Home Health Nurse (RN or LPN) Auto Apply
Givinghhc · Fairview · On-site · Posted 22 Oct 2026 🇺🇸
ComplianceNursingPatient Care+1Healthcare$35 - $55/hr - VH

Vascular Access Nurse Consultant - PICC RNRecently posted Auto Apply
VTP Holdings · Fairfield · Hybrid · Posted 11 Nov 2026 🇺🇸
BLSNursingSchedulingHealthcare$50 - $70/hr - VH

Registered Nurse (RN) - PICC NurseRecently posted Auto Apply
VTP Holdings · Vallejo · Hybrid · Posted 11 Nov 2026 🇺🇸
BLSNursingSchedulingHealthcare$50 - $70/hr - VH

Registered Nurse (RN) – PICC NurseRecently posted Auto Apply
VTP Holdings · Napa · Hybrid · Posted 11 Nov 2026 🇺🇸
BLSNursingSchedulingHealthcare$50 - $75/hr
