AI Safety & Evaluation Engineer

AI Safety & Evaluation Engineer

Summary Own responsible AI deployment — red-teaming, guardrails, hallucination detection, and evaluation. You''re the last line of defense ensuring our agents and models are safe, accurate, and trustworthy in production.

Role Expectations

• Design and run red-teaming and adversarial testing against models and agents (jailbreaks, prompt injection, misuse).
• Build guardrails: input/output filtering, policy enforcement, content moderation, and safe fallback behavior.
• Develop hallucination detection and grounding checks; measure and reduce factual error rates.
• Design evaluation frameworks and benchmarks for accuracy, safety, robustness, and bias — both offline and in production.
• Define and enforce responsible AI standards, documentation, and deployment gates.
• Partner with agent and ML teams to close safety gaps before and after release.

Required Skills

• 6+ years in ML/AI, security, or evaluation-focused engineering.
• Strong Python and deep familiarity with LLM behavior, failure modes, and prompt engineering.
• Hands-on experience building evaluations, benchmarks, or guardrail/moderation systems.
• Understanding of red-teaming, adversarial attacks, and hallucination mitigation techniques.
• Rigorous, data-driven approach to measuring model quality and risk.

Value adds:

• Experience with eval tooling and safety frameworks.
• Background in AI security, alignment research, or trust & safety.
• Familiarity with AI governance, regulatory, or compliance requirements.

Core Stack:

    Python · Evaluation Frameworks · Guardrails & Moderation · Red-       Teaming · Prompt Engineering · Hallucination Detection