Why this role stands out
At Glean, you'll leverage cutting-edge AI to drive impactful innovations in enterprise search and AI assistants, fostering significant career growth in a rapidly expanding tech leader. This hybrid role is ideal for analytical minds passionate about shaping the future of work and who thrive in a collaborative, forward-thinking environment. Apply now to join a company recognized for its groundbreaking work and to develop your skills within a dynamic team.
Quick overview
Job description
- Run ongoing human labeling for priority AI workflows, including response quality, task success, MCP and tool use, end-to-end Cowork-style workflows.
- Triage bad-query reports, downvotes, escalations, and routed quality issues; categorize failure modes and feed clean analysis back to partner teams.
- Perform qualitative analysis to identify recurring patterns such as hallucinations, retrieval failures, tool-use issues, weak grounding, and poor workflow completion.
- Follow and improve labeling guidelines and rubrics so judgments are consistent, realistic, and useful for model and product improvement.
- Maintain high-quality labeled datasets, golden sets, and regression slices; monitor coverage, drift, leakage, and difficulty distribution.
- Participate in calibration exercises with human graders and validate LLM-as-a-judge outputs against human labels to improve consistency and judge quality.
- Partner with engineers, PMs, QA, and eval owners on pre/post-change quality reads, benchmark updates, and release-readiness decisions.
- Contribute to recurring quality reporting by surfacing top failure modes, coverage gaps, quality shifts, and actionable recommendations.
- 3–5 years of experience in data labeling, data analysis, QA, or a related field.
- Strong analytical judgment and attention to detail, with the ability to apply nuanced rubrics consistently.
- Experience evaluating AI-generated outputs or working with NLP, search, recommendation, or other ML systems.
- Familiarity with structured qualitative analysis, basic SQL/data retrieval, and spreadsheet-based workflows.
- Clear written and verbal communication skills, with the ability to explain findings to both technical and non-technical audiences.
- Ability to work independently in ambiguous, fast-moving environments and collaborate effectively across functions.
By clicking “Submit Application,” I confirm that I have read the Global Data Privacy Notice and the Applicant Arbitration Agreement, and I agree to the terms.
Similar jobs
- IN
MDM AnalystRecently posted Auto Apply
IntegriChain · Pune, MH · Remote 🇮🇳
Microsoft ExcelData AnalysisEngineeringSalary not listed - PO

Director - Analytics Auto Apply
Pocketfm · Bengaluru · On-site 🇮🇳
Stakeholder ManagementSQLLooker+6Salary not listed - WA
Senior Data Analyst Auto Apply
WSA APAC · Hyderabad · Hybrid 🇮🇳
SQLNumPyArticulate+8TechnologySalary not listed - NI
Senior Data Operations Analyst, NIQ One PlatformRecently posted Auto Apply
NielsenIQ · Gurgaon · On-site 🇮🇳
Operations & Project ManagementSalary not listed - NI
Principal Data Operations Analyst, NIQ One PlatformRecently posted Auto Apply
NielsenIQ · Gurgoan · On-site 🇮🇳
Operations & Project ManagementSalary not listed - BG
Data Consultant - LogisticsRecently posted Auto Apply
Bosch Group · coimbatore · On-site 🇮🇳
OraclePL/SQLSQL+4LogisticsSalary not listed - ZI
Safety Data ConsultantRecently posted Auto Apply
Zifo · Chennai, Tamil Nadu · Hybrid 🇮🇳
Stakeholder ManagementOraclePharmacovigilance+1Salary not listed - CA
Workforce Management AnalystRecently posted Auto Apply
Capgemini · Pune, Maharashtra · Hybrid 🇮🇳
Microsoft OfficeSchedulingNetwork AdministrationHuman ResourcesSalary not listed - SM
SMSTR62-Senior AnalystRecently posted Auto Apply
Smartstream Limited · Mumbai, Maharashtra · On-site 🇮🇳
Stakeholder ManagementSQLMicrosoft Excel+13Salary not listed - ON

FP&A Data AnalystRecently posted Auto Apply
Onecom · Gurugram · On-site 🇮🇳
SQLMicrosoft ExcelLooker+17TechnologySalary not listed - EU
Data AnalystRecently posted Auto Apply
Eurofins · Bengaluru, KA · Hybrid 🇮🇳
SQLT-SQLGit+16TechnologySalary not listed - GU

Data Analyst (Finance)Recently posted Auto Apply
Guidepoint · Mumbai · Hybrid 🇮🇳
Stakeholder ManagementMicrosoft ExcelTableau+2TechnologySalary not listed
