Quick overview
Job description
Machine Learning Engineer / AI & LLM Specialist (SME) Location: McLean, VA
Key Requirement: Hands-on experience testing and evaluating Retrieval-Augmented
Generation (RAG) systems Overview: Square Peg Technologies is seeking a Machine Learning Engineer SME to support an
Intelligence Community customer in McLean, VA. This role focuses on developing, fine tuning, testing, and evaluating AI/ML models, particularly large language models (LLMs) and
Retrieval-Augmented Generation (RAG) systems. The ideal candidate has strong Python and deep learning experience with expertise in GPU accelerated computing, model benchmarking, and AI system performance evaluation.
Key Responsibilities
- Develop, train, and fine tune LLMs and neural networks using custom datasets.
- Test and evaluate RAG systems to assess model performance and effectiveness.
- Design and execute experiments to benchmark LLM performance against established evaluation criteria.
- Develop and evaluate machine learning models using Python and deep learning frameworks.
- Leverage GPUs to accelerate model training, inference, and testing.
- Develop data preprocessing workflows, select appropriate ML libraries, and optimize model performance.
- Test, validate, and compare AI/ML models for specific use cases.
- Develop SQL queries to extract, transform, and analyze data.
- Use GitHub and Jenkins to support software development and testing workflows.
- Document experimental results, model performance, and technical recommendations.
Required Qualifications
- Hands-on experience testing and evaluating RAG systems.
- Experience fine tuning LLMs or neural networks using custom datasets.
- Experience developing machine learning models and ensembles, including preprocessing, model selection, and performance evaluation. ,
- Strong programming experience with Python or R.
- Experience with deep learning frameworks such as PyTorch, TensorFlow, or Keras.
- Proficiency with SQL, including common table expressions, set operations, aggregate functions, and nested subqueries.
- Experience with GitHub and Jenkins.
- Hands-on experience leveraging GPUs for accelerated computing.
- Experience testing, evaluating, and validating AI systems, particularly LLM based systems.
- Experience designing and executing experiments to benchmark LLM performance.
Desired Qualifications
- Experience with Hugging Face Transformers and Hugging Face Hub.
- Experience developing benchmarks to measure LLM performance.
- Experience with cloud computing development and architecture.
- Experience with Flask or similar web development frameworks.
- Experience developing text classification and topic modeling solutions using Scikit Learn or deep learning models.
- Experience developing semantic search applications.
- Experience creating dashboards and visualizations using Tableau.
- Ability to communicate model methodologies, technical decisions, and experimental results.
- Experience with verification and validation test benches.
- Knowledge of Explainable AI (XAI) techniques.
- Experience with ONNX (Open Neural Network Exchange).
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- WO

Mid / Senior Software Development Engineer - ML Ops (US Federal) with Security ClearanceRecently posted
Workday, Inc. · Reston, VA · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerGCPMicroservices+13Technology$163.8k/yr - IS
Python AI/ML EngineerRecently posted
IT Solutions · Malvern, PA · Hybrid · Posted 20 Nov 2026 🇺🇸
FastAPIAWSPython+1TechnologySalary not listed - TA

Software Engineer / AI & Machine Learning Engineer with Security ClearanceRecently posted
TalentOps · Chantilly, VA · Hybrid · Posted 20 Nov 2026 🇺🇸
HTMLApacheMachine Learning+1TechnologySalary not listed - TE
AI/ML Engineer - Denver, CO, Phoenix, AZ, St. Louis, MO, Nashville, TN, Kansas City, KS.Recently posted
TechniPros, LLC · Denver, CO · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerFastAPISQL+13TechnologySalary not listed - QU
AI/ML Engineer – Vertex AI - Denver, CO, Phoenix, AZ, St. Louis, MO, Nashville, TN, Kansas City, KS.Recently posted
QTech US Inc · Denver, CO · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerFastAPISQL+13TechnologySalary not listed - EX
Senior Machine Learning Engineer- (Generative AI)Recently posted
Excelraise LLC · Raleigh, NC · Hybrid · Posted 20 Nov 2026 🇺🇸
SAFeDynamoDBFastAPI+20TechnologySalary not listed - PS
Senior Machine Learning EngineerRecently posted
Precision System Design Inc. · Cleveland, OH · Remote · Posted 20 Nov 2026 🇺🇸
DockerSQLAWS+8TechnologySalary not listed - TE
AI/ML Engineer - Chicago, IL, Minneapolis, MN, Detroit, MI, Madison, WI, Columbus, OH.Recently posted
TechniPros, LLC · Chicago, IL · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerSQLGit+21TechnologySalary not listed - RI

Engineering Intern - ML Compilers (Spring 2027 Co-op)
Rivian · Palo Alto, CA · On-site 🇺🇸
RecruitingSpringBackground Checks+7$45 - $51/hr - NG

Fulltime Job -Machine Learning Engineer-Irving, TXRecently posted
Newt Global · Irving, TX · Hybrid · Posted 20 Nov 2026 🇺🇸
Stakeholder ManagementMachine LearningTechnologySalary not listed - BA

AI/ML EngineerRecently posted
Booz Allen Hamilton · San Diego, CA · On-site · Posted 19 Nov 2026 🇺🇸
DockerAWSScikit-learn+11Technology$77.6k - $176k/yr - IT

AI/ML EngineerRecently posted
IT Trailblazers, LLC. · Sunnyvale, CA · Hybrid · Posted 19 Nov 2026 🇺🇸
C++PyTorchPython+2TechnologySalary not listed
