Machine Learning Engineer (AI & LLM Models) with Security Clearance

Machine Learning Engineer / AI & LLM Specialist (SME) Location: McLean, VA​
Key Requirement: Hands-on experience testing and evaluating Retrieval-Augmented
Generation (RAG) systems Overview: Square Peg Technologies is seeking a Machine Learning Engineer SME to support an
Intelligence Community customer in McLean, VA. This role focuses on developing, fine tuning, testing, and evaluating AI/ML models, particularly large language models (LLMs) and
Retrieval-Augmented Generation (RAG) systems. The ideal candidate has strong Python and deep learning experience with expertise in GPU accelerated computing, model benchmarking, and AI system performance evaluation.

Key Responsibilities: • Develop, train, and fine tune LLMs and neural networks using custom datasets. • Test and evaluate RAG systems to assess model performance and effectiveness. • Design and execute experiments to benchmark LLM performance against established evaluation criteria. • Develop and evaluate machine learning models using Python and deep learning frameworks. • Leverage GPUs to accelerate model training, inference, and testing. • Develop data preprocessing workflows, select appropriate ML libraries, and optimize model performance. • Test, validate, and compare AI/ML models for specific use cases. • Develop SQL queries to extract, transform, and analyze data. • Use GitHub and Jenkins to support software development and testing workflows. • Document experimental results, model performance, and technical recommendations.

Required Qualifications: • Hands-on experience testing and evaluating RAG systems. • Experience fine tuning LLMs or neural networks using custom datasets. • Experience developing machine learning models and ensembles, including preprocessing, model selection, and performance evaluation. ,• Strong programming experience with Python or R. • Experience with deep learning frameworks such as PyTorch, TensorFlow, or Keras. • Proficiency with SQL, including common table expressions, set operations, aggregate functions, and nested subqueries. • Experience with GitHub and Jenkins. • Hands-on experience leveraging GPUs for accelerated computing. • Experience testing, evaluating, and validating AI systems, particularly LLM based systems. • Experience designing and executing experiments to benchmark LLM performance.

Desired Qualifications: • Experience with Hugging Face Transformers and Hugging Face Hub. • Experience developing benchmarks to measure LLM performance. • Experience with cloud computing development and architecture. • Experience with Flask or similar web development frameworks. • Experience developing text classification and topic modeling solutions using Scikit Learn or deep learning models. • Experience developing semantic search applications. • Experience creating dashboards and visualizations using Tableau. • Ability to communicate model methodologies, technical decisions, and experimental results. • Experience with verification and validation test benches. • Knowledge of Explainable AI (XAI) techniques. • Experience with ONNX (Open Neural Network Exchange).