Back to Jobs
Recently postedTechnology

Data Scientist

New Digital IT Inc.United States🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationUnited States
AWSAirflowConfluenceLLMPythonGenerative AIAzureMachine LearningJiraPrompt EngineeringLangChainRAG

Job description

Role: Senior Data Scientist + Gen AI


Job description: Senior Engineer with Data Science + Gen AI

Project focus

Build production-grade Generative AI systems with emphasis on:

  • RAG pipelines leveraging LLMs.
  • Prompt engineering:
    • System prompts.
    • Tool prompts.
    • Function calling.
    • Versioning with evaluations.
  • Evaluation and observability:
    • Ground-truth setup.
    • Confusion metrics.
    • Hallucination-rate measurement.
    • LLM-as-judge with human review.
    • Cost and latency monitoring.
  • Retrieval strategies and prompt patterns.
  • Context management.
  • Hallucination mitigation.
  • Faithfulness and grounding validation.
  • Agentic workflows:
    • Multi-step reasoning.
    • Tool orchestration.
    • Agent success-rate measurement.
  • LangChain/LlamaIndex—or equivalent—proficiency.
  • Cloud LLM providers:
    • Azure OpenAI.
    • AWS Bedrock.
    • Vertex AI.
  • Workflow orchestration:
    • Airflow.
    • Dagster.
  • Security and privacy:
    • PII handling.
    • RBAC.
  • Strong Python expertise with real production experience.

Success criteria

  • Production-ready prompt and RAG system.
  • Measurable quality lift: accuracy, faithfulness, and confusion metrics.
  • Reduced hallucinations.
  • Repeatable evaluation harness.
  • Observable system: quality, latency, cost, and drift monitoring.

Listed skills: Large Language Models, AWS, Prompt Engineering, Python, Machine Learning, Gen AI, Confluence, Jira.

Similar jobs