Start searching
Back to Jobs
Recently postedTechnology

Senior Data Scientist + Gen AI

AIT Global, Inc.Mountain View, CA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationMountain View, CA, United States
AWSAirflowConfluenceLLMPythonGenerative AIAzureMachine LearningJiraPrompt EngineeringLangChainRAG

Job description

Job Title: Senior Data Scientist + Gen AI
Location: Mountain View, CA
RAG pipelines leveraging LLMs
Prompt engineering (system prompts, tool prompts, function calling, versioning with evaluations)
Evaluation & observability, including:
  • Ground truth setup
  • Confusion metrics
  • Hallucination rate measurement
  • LLM-as-judge with human review
  • Cost & latency monitoring
  • Retrieval strategies & prompt patterns
  • Context management
  • Hallucination mitigation
  • Faithfulness & grounding validation
  • Agentic workflows
  • Multi-step reasoning
  • Tool orchestration
  • Agent success rate measurement
  • LangChain / LlamaIndex (or equivalent) proficiency
  • Cloud LLM providers
  • Azure OpenAI
  • AWS Bedrock
  • Vertex AI
  • Workflow orchestration
  • Airflow
  • Dagster
  • Security & privacy
  • PII handling
  • RBAC
  • Strong Python expertise with real production experience
  • Success Criteria
  • Production-ready prompt + RAG system
  • Measurable quality lift (accuracy, faithfulness, confusion metrics)
  • Reduced hallucinations
  • Repeatable evaluation harness
  • Observable system (quality, latency, cost, drift monitoring)
Skills:
  • Large Language Models, AWS, Prompt Engineering, Python, Machine Learning, Gen AI, Confluence, Jira

Similar jobs

Apply