Quick overview
Job description
Overview
We are hiring Quality Control (QC) evaluators for a project that spans both Search and Linguistic tasks. You will perform quality checks on initial ratings produced by evaluators—validating user intent assessment, query annotation, text classification, and multimedia (video, image, audio) reviews for relevanc, accuracy, and guideline adherence. The ideal candidate combines strong language skills with sharp analytical judgment, excellent documentation and feedback skills, and familiarity with short-form video culture.
What you will do
- Audit samples of initial ratings for relevance, intent alignment, taxonomy compliance, and linguistic quality.
- Re-rate items (double-scoring) to verify labels; resolve discrepancies and document decisions.
- Perform linguistic quality checks (grammar, tone, clarity, local relevance) and media comprehension reviews.
- Classify and track errors using an agreed error taxonomy; surface trends and edge cases.
- Provide clear, actionable feedback to evaluators; contribute to guideline clarifications and updates.
- Participate in and lead calibration sessions to drive inter-rater agreement.
- Track and report QC metrics (e.g., agreement rates, accuracy, error trends) and meet QC SLAs.
- Triage escalations, handle ambiguous cases, and ensure data integrity and compliance.
- Use internal tools to complete, track, and submit QC tasks accurately.
Project Details
Engagement Type: Freelance
Rate: USD 28.61/hour
Estimated Hours: 15-30 hours/week
Requirement
- Native or near-native speaker level in English language with strong reading, writing, and listening comprehension.
- Excellent grammar, writing, and proofreading skills.
- Strong video and image comprehension; good hearing ability for audio tasks.
- Solid understanding of search engines, user intent, and web content.
- High attention to detail, consistency, and guideline adherence.
- Ability to deliver clear written feedback and rationale.
Preferred qualifications
- Experience in QC/QA for text classification, query annotation, data labeling, or search quality evaluation.
- Familiarity with short dramas and local short-drama culture/platforms.
- Experience with calibrations, sampling, scorecards, and root-cause analysis.
- Comfortable working with productivity and quality targets; proficient with spreadsheets/QA tools.
- Background in linguistics, communications, media studies, information science, or related field.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- AU
Sr. Manager Quality AssuranceRecently posted Auto Apply
Aumovio · Morganton, NC · On-site · Posted 17 Nov 2026 🇺🇸
Project ManagementEngineeringSalary not listed - T&
QA/QC MEP Construction Project Manager - Data CentersRecently posted Auto Apply
Turner & Townsend · Hamlet, North Carolina · On-site · Posted 17 Nov 2026 🇺🇸
PMPProcurementProject ManagementOperations & Project ManagementSalary not listed - CO
Senior Test Automation EngineerRecently posted Auto Apply
City of New York · Brooklyn, NY · On-site · Posted 17 Nov 2026 🇺🇸
AzureCI/CDAzure DevOpsTechnologySalary not listed - EA

Test & Validation Integration Engineer - FederalRecently posted Auto Apply
ELEVI Associates · Chantilly · Hybrid · Posted 17 Nov 2026 🇺🇸
AnsibleBashBootstrap+5Technology$155k - $200k/yr - TP

Associate Director, R&D Quality AssuranceRecently posted Auto Apply
Tarsus Pharmaceuticals · United States - Remote · Remote · Posted 16 Nov 2026 🇺🇸
Microsoft OfficeComplianceRoot Cause Analysis+1$150k - $210k/yr - SI

Principal Engineer, Product Validation | 1685Recently posted Auto Apply
SiTime Corporation · United States · Hybrid · Posted 17 Nov 2026 🇺🇸
MATLABIoTPython+1Technology$148.3k - $203.9k/yr - SI

Validation Engineer | 1804Recently posted Auto Apply
SiTime Corporation · Arizona · Hybrid · Posted 17 Nov 2026 🇺🇸
ANSYSIoTPython+2EngineeringSalary not listed - SI

Test Technician | 1649Recently posted Auto Apply
SiTime Corporation · California · Hybrid · Posted 17 Nov 2026 🇺🇸
ComplianceIoTData Analysis+1$21 - $40/hr - SI

Sr. Test Engineer | 1813Recently posted Auto Apply
SiTime Corporation · Arizona · Hybrid · Posted 17 Nov 2026 🇺🇸
MATLABC++IoT+3Technology$99.9k - $137.4k/yr - SI

Sr. Test Engineer | 1794Recently posted Auto Apply
SiTime Corporation · Arizona · Hybrid · Posted 17 Nov 2026 🇺🇸
MATLABC++IoT+3TechnologySalary not listed - KI
Sr Qualification Engineer - 2271Recently posted Auto Apply
Kioxia · Folsom, CA · Hybrid · Posted 16 Nov 2026 🇺🇸
Engineering$99.6k - $165.9k/yr - SI

Test Development Engineer | 10040Recently posted Auto Apply
SiTime Corporation · California · Hybrid · Posted 13 Nov 2026 🇺🇸
Continuous ImprovementIoTData Analysis+1Engineering$116.9k - $160.7k/yr
