Copy of Senior Python Developer (AI Evaluation & Benchmarking)
Quick overview
Job description
An enterprise client is seeking experienced Senior Python Developers to help build the next generation of Artificial Intelligence systems. This opportunity support the client who is a leading AI platform that enables organizations to build intelligent applications through high-quality human feedback, AI evaluation, and model alignment.
The selected consultants will contribute to AI research by designing programming benchmarks, evaluating AI-generated code, and helping improve the performance, reasoning, and reliability of frontier AI models. This is an excellent opportunity for experienced software engineers who enjoy solving complex technical problems while contributing to the future of Generative AI.
This opportunity is ideal for senior software engineers with strong Python expertise who enjoy writing high-quality code, reviewing technical solutions, and working on AI-related projects.
What You'll Do:
- Design and develop coding benchmarks used to evaluate frontier AI models.
- Analyze AI-generated code for correctness, reliability, efficiency, and edge cases.Build and maintain scalable data pipelines that support AI evaluation workflows.
- Create structured programming scenarios to test reasoning, debugging, and code quality.
- Work with large codebases and multi-language software environments.
- Collaborate with teams focused on improving how AI models understand, generate, and evaluate software.
- Write clean, maintainable, and well-tested Python code following software engineering best practices.
Requirements:
- 4+ years of professional software engineering experience (required).
- Expert-level proficiency in Python.
- Experience working at a high-growth technology company or top-tier software organization.
- Proficiency in at least one additional programming language such as JavaScript, Go, C++, or similar.
- Experience with CI/CD pipelines and automated testing frameworks such as pytest, Mocha, or JUnit.
- Strong understanding of software engineering best practices, debugging, and code quality.
- Excellent analytical and problem-solving skills.
Nice to Haves:
- Experience with AI/ML evaluation, model benchmarking, or Generative AI.
- Background in security engineering.
- Significant contributions to open-source software projects.
- Experience working with large-scale distributed systems or enterprise software platforms.
- Fully remote contract opportunity.
- Compensation ranges from $80–$100 USD per hour.
- Expected workload is 10–39 hours per week, depending on project needs.
- Weekly payments for approved work completed during the previous week.
- Work volume may fluctuate throughout the duration of the engagement.
Hiring Process:
- Submit your proposal.
- Complete a short qualification form provided by the client.
- Qualified candidates will receive an Upwork contract offer and onboarding instructions.
- After accepting the contract, candidates will gain access to the client's evaluation.
- Candidates who successfully pass the evaluation will be invited to a technical interview with the client.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- MA

Software Engineer - Magnet One (front-end, back-end, or full-stack opportunities) Auto Apply
Magnetforensics · Waterloo · Remote 🇨🇦
DockerShellGit+25TechnologyC$75k - C$175k/yr - ST

Full Stack EngineerRecently posted Auto Apply
StackAdapt · Canada · Hybrid · Posted 19 Nov 2026 🇨🇦
Material UIMySQLRuby+9TechnologyC$140k/yr - WO

Full Stack Engineer InternRecently posted Auto Apply
Workstream · Vancouver · Hybrid · Posted 19 Nov 2026 🇨🇦
RecruitingRubySQL+8TechnologyC$19 - C$22/hr - SI
SAP iXp Intern - Full-stack Software DeveloperRecently posted Auto Apply
SAP IT Business Systeme · Vancouver, BC · Hybrid · Posted 16 Nov 2026 🇨🇦
SpringAngularJava+4TechnologyC$26 - C$37/hr - TA

Developer Relations Engineer Auto Apply
Tailscale · Remote (Canada) · Remote · Posted 13 Nov 2026 🇨🇦
GitCADDiversity & Inclusion+2TechnologyC$170.2k/yr - AL

Senior Full-Stack Developer (Python)Recently posted Auto Apply
AlayaCare · Montréal · Hybrid · Posted 16 Nov 2026 🇨🇦
DockerSQLGit+8TechnologySalary not listed - AL

Junior Fullstack Developer (Python)Recently posted Auto Apply
AlayaCare · Montréal · Hybrid · Posted 16 Nov 2026 🇨🇦
DockerGitPython+5TechnologySalary not listed - AL

Fullstack Developer (Python)Recently posted Auto Apply
AlayaCare · Montréal · Hybrid · Posted 16 Nov 2026 🇨🇦
DockerPHPSQL+8TechnologySalary not listed - SO

Senior Full Stack Engineer,Media TeamRecently posted Auto Apply
Solink · Ottawa Office · Hybrid · Posted 16 Nov 2026 🇨🇦
GCPRustAWS+8TechnologyC$115.8k - C$165k/yr - WE

Senior Salesforce Developer Auto Apply
Webflow · Canada (BC & ON only) · Remote · Posted 30 Oct 2026 🇨🇦
GitScrumCAD+5TechnologyC$116.6k - C$163.3k/yr - OK

Senior Full-Stack Engineer, Builder PlatformRecently posted Auto Apply
Okta · Toronto · Hybrid · Posted 18 Nov 2026 🇨🇦
DockerNode.jsReact+7TechnologyC$165k/yr - CA

Senior Full Stack Software Engineer Auto Apply
Canadavisa · Toronto Office · Hybrid · Posted 10 Nov 2026 🇨🇦
FastAPINext.jsPHP+9TechnologyC$150k - C$165k/yr