Quick Overview
Seniority
Mid Senior
Work mode
On Site
Location
Atlanta, GA, United States
Posted
19 hours ago
DockerSQLitePython
Job Description
Role: AI Systems Performance & Benchmarking Engineer
Location: Onsite - Multiple locations to client office
Key responsibilities:
Location: Onsite - Multiple locations to client office
Key responsibilities:
- Review AgentSysPerf's architecture end to end: benchmark adapters, measurements, analyzers, storage, replay, reporting and dashboards.
- Review state persistence, container lifecycle, concurrency and the AgentInvoker protocol implementation.
- Integrate up to 4 new agentic benchmarks (e.g., τ-bench, BrowseComp, GAIA, BFCL). This includes configuration, CLI execution, model/runtime integration, metrics capture and results persistence.
- Measure latency, throughput and resource use on Intel hardware.
- Improve run variance, repeatability, fault handling and SQLite write concurrency during large sweeps.
- Add new schema fields (policy hashes, topology IDs, route decisions, resource placement) to the storage and telemetry pipelines.
- Python, Linux internals, Docker, cgroups and Linux perf tools.
- Hardware profiling.
- SQLite and OTLP/OpenTelemetry exporters.
- Benchmarking LLMs or agents, including inference engines and runtime configurations.
- Xeon platform tuning.
- Terminal-Bench.
- Experience with AgentSysPerf itself.
Similar jobs
- SC
Oracle Cloud Senior Engineer- Active Secret clearance is must
NewServIT Consulting LLC
United States🇺🇸HybridYesterdayDockerOracleCompliance+1 - VC
Learning & Development Specialist /Trainer
NewVega Consulting Solutions
Baltimore, MD🇺🇸On-site19 hours agoLMSLearning & DevelopmentMicrosoft Office+2 - NI
Data Modeler at a Major Japanese Bank
NewNSD International, Inc.
Charlotte, NC🇺🇸Hybrid19 hours agoAzureDatabricks - RL
Mechanical Design Lead
NewResource Logistics
Allentown, PA🇺🇸Hybrid19 hours agoAssemblyAutoCADCAD+2 - SC
Service360 Data Governance Lead- 100% Remote
NewSmart Caliber Technology
United States🇺🇸Remote19 hours agoOracleSalesforceCRM+5 - ST
Cable Installer Technician in Ashburn, VA, Onsite , w2 only
NewSR Talent Solution Inc
Ashburn, VA🇺🇸On-site19 hours agoFiber - SO
Google Analytics Consultant
NewSystem One
United States🇺🇸Hybrid19 hours agoSQLA/B TestingCSS+6 - AG
CAD Designer Electrical
NewAIT Global, Inc.
Calgary, AB🇺🇸Hybrid19 hours agoAssemblyAutoCADCAD+1 - SY
Neo4J Platform Architect
NewSyrenCloud LLC
United States🇺🇸Hybrid19 hours agoNeo4jAWSCloudFormation+1 - 4C
Database Architects
New4 Consulting Inc
United States🇺🇸Hybrid19 hours agoRequirements Gathering - EG
Senior EDI with Healthcare Payer (Facets Ecosystem) Support Specialist
NewEverest Global Solutions
Seattle, WA🇺🇸Hybrid19 hours agoOracleSQLSQL Server+10 - CT
Azure Administration
NewCynosure Technologies LLC
Burlington, MA🇺🇸On-site19 hours agoMFASSOActive Directory+4
