Quick overview
Job description
Project Overview
We are sourcing independent Information Specialists to provide their expertise for an AI benchmark evaluation project. As AI models increasingly generate professional-grade data management strategies, information retrieval systems, and content architecture deliverables, their accuracy relies entirely on robust, expert-crafted training data. The objective of this project is to autonomously produce high-quality evaluation tasks, strong prompts, and clear, well-structured rubrics that generate clean, reliable data for model training.
Project Deliverables & Scope
Operate autonomously to design complex evaluation frameworks and provide structured training data. Expected deliverables include:
- Task & Prompt Creation: Generating realistic, high-quality prompts that compel the AI model to produce complex, professional-grade deliverables specific to information science, data management, and knowledge organization.
- Rubric Development: Writing clear, well-structured evaluation rubrics with criteria that are highly specific, non-ambiguous, and easy to score.
- Benchmark Evaluation Data Generation: Producing clean, reliable training data that directly aids in the evaluation and refinement of AI models handling complex information architecture and retrieval tasks.
- Quality Assurance & Fact-Checking: Ensuring all generated tasks and scoring criteria reflect strict, real-world data governance standards, research methodologies, and information management best practices.
Required Expertise
To successfully fulfill the deliverables of this project, Contractors must possess deep industry knowledge to craft realistic professional scenarios.
Core skillset includes:
- Demonstrable professional expertise within the information science, library science, data governance, or knowledge management sectors, with a deep understanding of industry standards, metadata schemas, and search methodologies.
- Strong writing and prompt generation skills, with the ability to design highly realistic, complex information retrieval task scenarios for AI evaluation.
- Proficiency in rubric generation, specifically the ability to create objective, non-ambiguous scoring criteria that leave no room for subjective interpretation.
- A meticulous, detail-oriented approach to fact-checking data architectures, search mechanics, and knowledge bases to generate reliable data for system benchmarking.
We offer a pay range of $10-to-$30 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.
Engagement Type: Freelance / Independent Contractor
Workplace Type: Remote
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- CR

Technical TrainerRecently posted Auto Apply
Crusoe · Tulsa · On-site · Posted 16 Nov 2026 🇺🇸
OnboardingWorkdayAssembly+1$85k - $100k/yr - EF

Group Fitness InstructorRecently posted Auto Apply
EōS Fitness · St Petersburg · Hybrid · Posted 17 Nov 2026 🇺🇸
GreenhouseEducation$15 - $35/hr - EF

Certified Personal TrainerRecently posted Auto Apply
EōS Fitness · St Petersburg · Hybrid · Posted 17 Nov 2026 🇺🇸
OnboardingGreenhouse$18/hr - AM

Heating, Ventilation, Air Condition and Refrigeration-HVACR InstructorRecently posted Auto Apply
Americaatwork · Reading · On-site · Posted 17 Nov 2026 🇺🇸
Curriculum DevelopmentHVAC$60k - $75k/yr - NS
Software Trainer IIRecently posted Auto Apply
Nemetschek SE2 · Pasadena, California · On-site · Posted 16 Nov 2026 🇺🇸
ZoomTechnology$64.9k - $81k/yr - UL
EMT Skills Instructor - Per DiemRecently posted Auto Apply
Unitek Learning · Tempe, AZ · On-site · Posted 16 Nov 2026 🇺🇸
Education$18/hr - UL
Primary Instructor - EMT Per DiemRecently posted Auto Apply
Unitek Learning · Tempe, AZ · On-site · Posted 16 Nov 2026 🇺🇸
Education$28/hr - UP

Certified Athletic Trainer - Industrial Injury Prevention (ATC, OT, PT) part-time 10 hours / weekRecently posted Auto Apply
US Physical Therapy · Kinston, North Carolina · On-site · Posted 16 Nov 2026 🇺🇸
BLSOccupational TherapyPhysical TherapyHealthcare$33 - $39/hr - GI
Certified Personal Coach - Lake NonaRecently posted Auto Apply
GOLFTEC Intellectual Property LLC · Orlando, Florida · On-site · Posted 16 Nov 2026 🇺🇸
$55k - $70k/yr - WW

Studio Coach - Burlington, VTRecently posted Auto Apply
Weight Watchers · United States - Studios · Hybrid · Posted 16 Nov 2026 🇺🇸
RecruitingOnboarding$18 - $21/hr - WW

Studio Coach - Sheboygan, WIRecently posted Auto Apply
Weight Watchers · United States - Studios · Hybrid · Posted 16 Nov 2026 🇺🇸
RecruitingOnboarding$18 - $21/hr - VE

Veterinary Nursing Trainer - Schaumburg, ILRecently posted Auto Apply
Veterinary Emergency Group (VEG) · Schaumburg · Hybrid · Posted 16 Nov 2026 🇺🇸
OnboardingDiversity & InclusionNursingHealthcare$33 - $65/hr
