Data Scientist, Conversational AI / Contact Center

Title: Data Scientist, Conversational AI / Contact Center
Location: Boston, MA  (Hybrid)
Duration: 6 months (possibility of an extension)
JD:

Role Summary

Key Responsibilities

• Design and execute evaluation frameworks for LLM, RAG, and multi-turn conversational AI systems used in Contact Center and voice-agent environments.
• Develop metrics to assess customer intent recognition, conversation quality, guardrail effectiveness, and business outcomes.
• Analyze Contact Center, IVR, and voice-agent interactions to identify areas of failure, drift, and performance degradation.
• Perform prompt tuning and experimentation to improve model accuracy and reliability.
• Conduct root-cause analysis of conversational failures and recommend remediation strategies.
• Measure performance across different model configurations, prompts, and guardrail implementations.
• Partner with Contact Center, AI Engineering, Product, and Operations teams to validate solutions before production deployment.
• Build dashboards and reports that communicate model effectiveness and operational impact.
• Support fraud-related Contact Center customer service use cases, including intent detection, call routing, self-service automation, and multi-turn conversation flows.

Required Qualifications

• Strong background in Data Science, Machine Learning, Generative AI, or a related quantitative field.
• Experience analyzing customer interactions, call transcripts, conversation flows, intent classification, and Contact Center KPIs.
• Deep understanding of model evaluation techniques and metrics, including:
  • Precision@K
  • Recall@K
  • Mean Reciprocal Rank (MRR)
  • F1 Score
  • Retrieval and generation quality assessment
• Experience performing experimentation, statistical analysis, and performance benchmarking.
• Strong Python programming skills.
• Experience with machine learning libraries and frameworks such as Scikit-learn, XGBoost, Pandas, NumPy, and related tools.
• Ability to communicate technical findings succinctly to highly technical stakeholders.

Preferred Qualifications

• Experience with:
  • Generative AI and LLM ecosystems
  • Multi-agent systems
  • RAG/Agentic RAG architectures
  • Amazon Bedrock
  • AWS SageMaker
  • Databricks
  • MLflow
  • LangSmith
  • Weights & Biases
• Knowledge of conversational AI, IVR systems, digital assistants, and voice agents.
• Experience in financial services, fraud detection, or customer service automation.

Success Criteria

• Develop reliable evaluation methodologies for Contact Center conversational AI and voice-agent systems.
• Quantify the effectiveness of fraud self-service voice agents.
• Optimize prompts, retrieval strategies, and guardrails using empirical evidence.
• Deliver actionable insights that improve customer experience, Contact Center efficiency, and model performance.
• Establish measurable KPIs for intent detection and multi-turn conversation success.

Important Note

This role is primarily a Data Science and AI Evaluation position with a strong Contact Center/Conversational AI focus, not an AI Engineering or deployment-focused role. The emphasis is on measuring, analyzing, validating, and improving AI system performance in real-world Contact Center interactions, rather than building production deployment pipelines.