Quick overview
Job description
This role is for one of our clients
We are seeking detail-oriented and analytical Business Analysts – Non-US Raters to support a project focused on evaluating and comparing responses generated by multiple AI chatbots. The role involves reviewing chatbot outputs, applying predefined evaluation criteria, conducting structured assessments, and delivering accurate, well-documented judgments.
Candidates should have strong analytical skills, business analysis experience, attention to detail, and the ability to work independently in a remote, fast-paced environment. This opportunity is suitable for professionals interested in contributing to AI evaluation and improving the quality, relevance, and reliability of AI-generated responses.
Job Details
- Role: Business Analyst – Non-US Raters
- Experience: 3+ years
- Employment Type: Short-term contract
- Contract Duration: 16 weeks
- Location: Argentina, Peru, Mexico, Brazil, Chile, Colombia, India, and Pakistan
- Compensation: USD 15 per hour
- Start Date: Immediate
- Work Mode: Fully remote
- Availability: Full-time commitment of 8 hours per day
- Time Zone Requirement: Minimum 4 hours of overlap with the US Pacific Time (PST) time zone
- Work Schedule: Five working days per week, including weekends when required, with two weekdays off
Key Responsibilities
- Review, analyze, and compare responses generated by multiple AI chatbots against predefined evaluation criteria.
- Conduct structured assessments to determine the accuracy, relevance, completeness, clarity, and overall quality of chatbot responses.
- Analyze business requirements and translate them into clear evaluation guidelines and functional specifications.
- Gather, interpret, and document relevant information to support consistent and objective assessments.
- Examine workflows, processes, and data to identify gaps, inconsistencies, and opportunities for improvement.
- Prepare evaluation reports, summaries, and supporting documentation with a strong focus on accuracy.
- Collaborate with project managers and cross-functional teams to clarify requirements and resolve evaluation-related questions.
- Track task progress, quality metrics, and project deliverables while meeting established deadlines.
- Identify recurring response-quality issues and provide actionable observations based on the defined assessment framework.
- Maintain consistency, confidentiality, and high-quality standards throughout the evaluation process.
Required Qualifications and Skills
- Bachelor's degree in Business Administration, Finance, Computer Science, or a related field.
- At least 3 years of experience as a Business Analyst or in a similar analytical role.
- Strong analytical thinking, problem-solving, and decision-making abilities.
- Excellent written communication skills and the ability to explain findings clearly.
- Strong attention to detail and the ability to assess information objectively.
- Proficiency in Microsoft Excel, SQL, Power BI, Tableau, or comparable analytical tools.
- Experience with requirement gathering, process mapping, and business documentation.
- Ability to work independently, manage multiple priorities, and meet strict deadlines.
- Strong documentation skills and the ability to follow structured evaluation guidelines.
- Familiarity with Agile or Scrum methodologies is an advantage.
- Willingness to commit to a full-time, 16-week remote engagement with the required time-zone overlap.
Engagement Benefits
- Opportunity to contribute to projects focused on AI evaluation and response quality.
- Fully remote work arrangement with the flexibility of working from your location.
- Exposure to structured AI assessment workflows and quality evaluation processes.
- Opportunity to contribute to projects with international reach.
- Clearly defined contract duration, hourly compensation, and evaluation responsibilities.
Important Engagement Details
- This is a project-based contract lasting 16 weeks.
- Work will consist of evaluating multiple chatbot responses and submitting a final assessment for each assigned task.
- Candidates must be available for 8 hours per day, five days per week, with four hours of overlap with US Pacific Time.
- The schedule may include weekend work, with two weekdays off each week.
- Candidates are expected to maintain consistent evaluation quality, follow project guidelines, and complete assigned tasks within the required timelines.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- EN
Business Analyst - TechnicalRecently posted Auto Apply
Endava · Pune, MH · Hybrid · Posted 20 Nov 2026 🇮🇳
Stakeholder ManagementDockerMicroservices+15Operations & Project ManagementSalary not listed - OA

Analyst/Sr. Analyst, OperationsRecently posted Auto Apply
Oaknorth · Gurugram · Hybrid · Posted 20 Nov 2026 🇮🇳
OnboardingCase ManagementCompliance+4Salary not listed - SA

Business Technology Engineer IIRecently posted Auto Apply
Samsara · Bengaluru · Hybrid · Posted 19 Nov 2026 🇮🇳
GCPGoogle WorkspaceAWS+14EngineeringSalary not listed - CH
PLM AnalystRecently posted Auto Apply
ChargePoint · India · Remote · Posted 19 Nov 2026 🇮🇳
ComplianceSalary not listed - TO

Business Analyst III Auto Apply
Tomtom · Pune · Hybrid · Posted 26 Oct 2026 🇮🇳
Stakeholder ManagementBusiness AnalysisData AnalysisOperations & Project ManagementSalary not listed - ZE

Senior Product Specialist - Implementation/OnboardingRecently posted Auto Apply
Zenoti · Hyderabad · Hybrid · Posted 18 Nov 2026 🇮🇳
Microsoft OfficeHTMLCRM+4Salary not listed - WI
Senior Product AnalystRecently posted Auto Apply
Wise · Hyderabad · On-site · Posted 18 Nov 2026 🇮🇳
OnboardingComplianceKYC+1Salary not listed - SS
SAP PP/QM Senior Functional AnalystRecently posted Auto Apply
Sopra Steria · Noida, Uttar Pradesh, UTTAR PRADESH · Hybrid · Posted 17 Nov 2026 🇮🇳
JiraTechnologySalary not listed - ST

Strategy and Analytics Business PartnerRecently posted Auto Apply
Stripe · Bengaluru · Hybrid · Posted 19 Nov 2026 🇮🇳
SQLCFACPA+4Salary not listed - CO
EMEA SAP Functional Analyst – Finance O2C | BangaloreRecently posted Auto Apply
Colliers · Bengaluru, KA · On-site · Posted 17 Nov 2026 🇮🇳
Stakeholder ManagementJiraAgile+1TechnologySalary not listed - SG
SAP SD Functional AnalystRecently posted Auto Apply
Syngenta Group · Pune, MH · Hybrid · Posted 17 Nov 2026 🇮🇳
Continuous ImprovementJiraAgileTechnologySalary not listed - CA

HSSE&C Systems & Performance AnalystRecently posted Auto Apply
Castrol · Pune · Hybrid · Posted 17 Nov 2026 🇮🇳
RecruitingOnboardingMicrosoft Excel+6Salary not listed