Data Annotation reviewer (QA) (GenAI/LLM)

Data Annotation reviewer (QA) (GenAI/LLM) role - Onsite -

Location: Louisville, KY

Duration: 2+ months with possibility if extensions

Role Description

As QA at Centific specializing in Generative AI Data Services, you will play a pivotal role in driving the success of our cutting-edge projects focused on building and enhancing Large Language Models (LLMs). Your primary responsibility will be to adhere to quality review instructions, pass, review or reject human-labeled data crucial for training AI models. In this dynamic and evolving field, you will be at the forefront of bridging the gap between human expertise and artificial intelligence, contributing significantly to the development of next-generation AI technologies. As a problem solver and highly organized lead, you will collaborate closely with internal teams and a global workforce, ensuring seamless communication, efficient execution, and successful delivery of projects.

Task Overview:
As a Data Annotation Reviewer, you will evaluate data annotation and bounding box responses from annotators. Your job is to ensure the correct annotation was applied within the input parameters match and the output is accurate and relevant according to the guidelines.

Key Responsibilities:

• Review annotations to confirm they match the intended purpose of the guidelines.
• Validate that the annotator answer is appropriate for the chosen function.
• Check if the Annotation output includes all necessary information to filfull the guidelines.
• Identify and assess individual data points in the AI response for accuracy and relevance.
• Provide comments and feedback to clarify decisions or flag inconsistencies.

Language Proficiency:

• Native level fluency in English.
• Strong written and verbal communication skills in English.

Skills and Responsibilities

• Assisting in the development and implementation of quality assurance policies.
• Collaboration on project guidelines and supplemental learning materials.
• Ensure compliance with client-specific requirements and performance standards in quality aspects of project execution.
• Provide feedback and recommendations to PM team regarding UI design.
• Review annotators and arbitrators weekly via the production monitoring process.
• Review metrics and spot check individual tasks.
• Record and flag quality issues.
• Conduct end-of-day reporting to ensure smooth project handoffs and continuity across shifts.

Key Qualifications:

• Located in Lousiville, KY and ability to work at client- on-site.
• Required: Diploma and at least 2-3 years of experience in quality management, particularly in data services for AI.
• Desired: Bachelor's degree, Linguistics or other related field of study
• Experience writing data-labeling project guidelines and materials (or similar work in another field).
• Ability to create useful and impactful insights for annotators and clients.
• Basic familiarity with Microsoft Office 365 including Outlook, Excel, and PowerPoint.
• General knowledge of online communication
• Excellent communication both written and spoken.
• Collaborative and solution focused.
• Confidence and the ability to give/take feedback.
• Ability to follow project directions and perform time bound tasks accurately and efficiently.
• Ability to maintain quality while performing repetitive tasks.
• Detail-oriented and problem-solving mindset.
• Organized and focused enough to work independently as a role player within a team environment.

Availability:

• Minimum of 30 hours per week.
• Willingness to participate in training sessions and demos.

Certifications:

• Successful completion of required certifications is mandatory.