Remote :: AI Testing Engineer (Multimedia Domain) || Contract
Quick overview
Job description
Position: AI Testing Engineer (Multimedia domain)
Location: Remote
Experience: 3–4 Years Experience
Role Overview
Looking for an AI Testing Engineer to validate and assure the quality of AI/ML systems powering real-time contextual advertising on live video streams.
This position is critical for native moments validation and system-level quality assurance in a real-time streaming environment.
Key Responsibilities:
Validate AI-generated outputs against labelled datasets and expected outcomes.
Perform content validation for native moments detection across video streams.
Design and implement automated testing frameworks using Python.
Build validation frameworks and test harnesses for:
- JSON parsing and comparison.
- Structured output validation using predefined schemas.
- Data validation and consistency checks.
Conduct multimodal validation of video, audio, and transcript outputs.
Execute system integration and end-to-end testing.
Track accuracy metrics, log defects, and perform root cause analysis. Automate regression testing and validation pipelines across datasets.
Collaborate with Applied Scientists to validate LLM and Generative AI outputs.
Work within AWS environments using services such as Amazon S3, AWS Lambda, and Amazon CloudWatch.
Required Skills & Experience:
Strong Python programming and scripting skills for automation and testing.
Experience in AI/ML model validation and QA workflows.
Hands-on experience with JSON parsing, schema comparison, and data validation pipelines.
Experience testing multimodal AI systems, including:
o Computer Vision.
o NLP.
o Audio and speech processing.
o LLM outputs and Generative AI pipelines.
- Good understanding of structured output validation and AI accuracy evaluation techniques.
Familiarity with AWS cloud services, particularly S3, Lambda, and CloudWatch.
Experience with media and video content analysis workflows is preferred. - Ability to work across both manual and automated testing approaches.
Thanks,
IT First Source
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- TM

BarbackRecently posted Auto Apply
The Mather in Tysons · The Mather (Tysons · Hybrid · Posted 19 Nov 2026 🇺🇸
Inventory ManagementNursingPoint of Sale+2$17 - $24/hr - MP

Virtual Rare Disease Specialist (Contract)Recently posted Auto Apply
Mirum Pharmaceuticals · US Remote · Remote · Posted 19 Nov 2026 🇺🇸
OnboardingCRMCompliance+4$15k/mo - MS

Quail Springs Mall SupervisorRecently posted Auto Apply
Marksman Security LLC · Oklahoma City · Hybrid · Posted 19 Nov 2026 🇺🇸
Salary not listed - MS

Director of SecurityRecently posted Auto Apply
Marksman Security LLC · Durham · Hybrid · Posted 19 Nov 2026 🇺🇸
PayrollPerformance ManagementScheduling$70k/yr - MS

Security Licensed Officers RialtoRecently posted Auto Apply
Marksman Security LLC · Rialto · Hybrid · Posted 19 Nov 2026 🇺🇸
LMS$20/hr - LE
Manager, Safety and HealthRecently posted Auto Apply
LG Energy Solution Arizona · Queen Creek · Hybrid · Posted 19 Nov 2026 🇺🇸
Stakeholder ManagementOnboardingCompliance+11Salary not listed - LE
Sr. Specialist, Safety and HeathRecently posted Auto Apply
LG Energy Solution Arizona · Queen Creek · Hybrid · Posted 19 Nov 2026 🇺🇸
OnboardingMicrosoft OfficeCompliance+2Salary not listed - ML

Principal Governance, Risk and Compliance AnalystRecently posted Auto Apply
Mission Lane · Remote · Remote · Posted 19 Nov 2026 🇺🇸
OnboardingGCPGoogle Workspace+13$138.0k - $180k/yr - MH

Operations & Administrative CoordinatorRecently posted Auto Apply
Mood Health · Seattle · Hybrid · Posted 19 Nov 2026 🇺🇸
OnboardingGoogle WorkspaceMental Health+1$22 - $25/hr - LH

Temporary Part Time Floor Leader - Roosevelt FieldRecently posted Auto Apply
Lush Handmade Cosmetics · Garden City · Hybrid · Posted 19 Nov 2026 🇺🇸
ArticulateDiversity & InclusionReconciliation+1$20/hr - KS

Implementation SpecialistRecently posted Auto Apply
Keeper Security · Remote · Remote · Posted 19 Nov 2026 🇺🇸
RecruitingOnboardingGoogle Workspace+10Salary not listed - LO

Industrial InsulatorRecently posted Auto Apply
Loenbro · Afton · Hybrid · Posted 19 Nov 2026 🇺🇸
ComplianceFLSASalary not listed