Job Description

Hands-on reinforcement learning experience.
Experience using LLMs for agents, evaluation, reasoning, automation, or benchmark workflows.
Strong Python experience for ML, data workflows, experimentation, and analysis.
Experience designing and running experiments with statistical and analytical rigor.
Strong understanding of evaluation metrics, scoring frameworks, performance comparison, and benchmark design.
Experience analyzing structured logs, run outputs, model/agent performance, and experiment results.
Ability to work across APIs, logs, CLI/tools, data structures, and platform workflows.
Strong communication skills to translate experiment findings into platform improvement requirements.
Ability to work inside client-owned repositories, infrastructure, workflows, and security controls.
The ability to develop text-based and multimodal agents using Python and LLMs
The ability to use frameworks (e.g., LangChain) to build agents that can manage reasoning flows

AI Evaluation & Benchmarking Engineer

Quick Overview