Quick overview
Job description
We are seeking experienced US registered nurses to review and assess AI-generated responses from a health coaching agent, with a focus on safety, regulatory compliance, and clinical content accuracy.
You will work asynchronously on a dedicated review platform, performing short, structured evaluations to help train and refine AI models for patient-facing health guidance.
This is a remote, flexible and short-term project.
Key Responsibilities:
Structured Grading: Score AI agent responses against a pre-defined rubric, applying the highest-priority criteria consistently across a large volume of items.
Safety Review: Identify responses that could cause patient harm missed red flags, unsafe advice, failure to escalate or signpost to care.
Clinical Accuracy: Assess factual correctness, completeness and guideline alignment of clinical content.
Written Feedback: Provide brief, specific rationale for each failed response and general guidance on how it could be improved.
Qualifications
Licensure: Active RN license for US
Experience: Minimum 2 years of clinical nursing practice.
Knowledge: Strong grounding in clinical guidelines, patient safety and scope-of-practice boundaries.
Attention to detail: Proven accuracy and consistency in clinical documentation or review work.
Technical proficiency: Comfortable using structured review tools and digital platforms.
Language: Native or near-native written English.
Legal Status
You will have the right to work in your country of residence.
You will work as an independent contractor.
Why Join Us?
Flexible Work Arrangements: Part-time, remote and fully asynchronous.
Competitive Compensation: Hourly rate
Professional Development: Hands-on experience in AI evaluation and model fine-tuning, shaping how AI is deployed in patient care.
Similar jobs
- NH
WIC Supervisor
NewAuto ApplyNATIVE HEALTH
Phoenix, AZ🇺🇸On-site9 hours agoBLSHealthcare - MC
Med Tech (Certified) Per diem
NewAuto ApplyMonarch Communities
Bernards, NJ🇺🇸$24 - $25/hrOn-site11 hours agoMedication AdministrationHealthcare - MC
Wellness Nurse (LPN)
NewAuto ApplyMonarch Communities
Rehoboth Beach, DE🇺🇸$32 - $36/hrOn-site11 hours agoMicrosoft ExcelMicrosoft OfficeMedication Administration+1Healthcare - MC
Care Manager (CNA) PRN
NewAuto ApplyMonarch Communities
Rehoboth Beach, DE🇺🇸$20 - $22/hrOn-site11 hours agoNursingHealthcare - EQ
Qualified Health Professional - Level 1, LMSW/LMHC
NewAuto ApplyEquus
New York, NY🇺🇸$70k - $72k/yrHybrid11 hours agoMental HealthHealthcare - AA
Veterinary Receptionist/CSR
NewAuto ApplyAlliance Animal Health
Fort Smith, AR🇺🇸$11 - $15/hrOn-site11 hours agoHealthcare - EU
Client Portfolio Manager – Medical Device Services
NewAuto ApplyEurofins
Maryland Heights, MO🇺🇸On-site11 hours agoOnboardingCRMCompliance+6Healthcare - SF
Dental Assistant
NewAuto ApplySalud Family Health
Longmont, CO🇺🇸$21 - $27/hrOn-site11 hours agoWorkdayBLSPatient Care+1Healthcare - EU
Client Portfolio Manager – Medical Device Services
NewAuto ApplyEurofins
Lancaster, PA🇺🇸On-site11 hours agoOnboardingCRMCompliance+6Healthcare - HC

SLP - Full Time (Inpatient Rehab Department)
NewAuto ApplyHoly Cross Hospital
Fort Lauderdale, FL🇺🇸On-site12 hours agoHealthcare - EV
Supervisor, Program Management
NewAuto ApplyEVERSANA
Chesterfield, MO🇺🇸$44.3k - $61.8k/yrRemote12 hours agoMicrosoft ExcelMicrosoft WordMicrosoft PowerPoint+2Healthcare - IS
Phlebotomist
NewAuto ApplyInternational SOS Government Medical Services
Washington, DC🇺🇸$23 - $35/hrOn-site13 hours agoPatient CareBLSHealthcare
