Data Annotation Engineer

JD:

We are looking for an Annotation Program Lead to manage our AI Quality Annotation Program. This individual will oversee the operational execution of human review activities that establish the gold-standard datasets used to measure conversational AI performance, safety, and compliance.

The ideal candidate combines strong program management skills with experience in quality assurance, data annotation operations, and stakeholder engagement.

• 5+ years of experience in program management, operations, quality assurance, data labeling, or related fields.
• Lead the day-to-day operation of the AI annotation program.
• Manage a team of annotators responsible for reviewing customer conversations and AI interactions.
• Develop annotation guidelines, procedures, and quality standards.
• Establish calibration programs and quality assurance processes to ensure consistency across reviewers.
• Partner with Data Scientists to create and maintain gold-standard datasets.
• Monitor annotation accuracy, throughput, and quality metrics.
• Experience leading teams and managing operational workflows.
• Strong organizational and stakeholder management skills.
• Ability to analyze quality metrics and drive continuous improvement initiatives.
• Excellent written and verbal communication skills.
• Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.