PySpark Data Engineer with Security Clearance

Data Engineer (TS/SCI Preferred)
Pay rate: $60-85hr W2 Overview We are seeking a Data Engineer to support a high-impact defense program focused on delivering advanced data solutions for mission-critical operations. This position offers the opportunity to work with modern cloud and big data technologies, building scalable data architectures and pipelines that support enterprise analytics, data integration, and decision-making across a secure environment. The ideal candidate will bring strong expertise in Databricks, PySpark, SQL, and DevOps practices, along with experience designing and optimizing large-scale data processing solutions. Location: Pentagon (Onsite) Duration: 1+ Year Contract with Extension Potential Clearance: Active TS/SCI Preferred (Secret Clearance Considered) Background Investigation: Required Key Responsibilities
Architect, develop, and maintain scalable data pipelines supporting enterprise data ingestion, processing, and transformation.
Design and optimize ETL workflows for large, complex datasets using PySpark and SQL.
Build and manage data engineering solutions within the Databricks platform.
Implement performance tuning strategies to improve reliability, scalability, and efficiency of data processing workloads.
Develop and maintain CI/CD pipelines to support automated testing, deployment, and monitoring.
Containerize and deploy applications using modern technologies such as Docker and Kubernetes.
Collaborate with software engineers, analysts, architects, and stakeholders to deliver scalable data solutions.
Apply DevOps principles and best practices to streamline development and deployment processes.
Support data governance, data quality, and system performance initiatives.
Troubleshoot and resolve data pipeline, platform, and performance-related issues.
Required Qualifications
Active TS/SCI clearance preferred; candidates with an active Secret clearance will also be considered.
Strong experience working within Databricks environments. Advanced proficiency with:
PySpark
SQL
ETL development and optimization
Data pipeline design and engineering
Experience implementing CI/CD pipelines and DevOps best practices.
Hands-on experience with Git version control.
Experience deploying and managing containerized applications using Docker and Kubernetes.
Strong understanding of distributed data processing and enterprise-scale data architectures.
Ability to optimize and tune high-volume data processing workloads.
Excellent problem-solving and analytical skills. Preferred Qualifications
Experience supporting federal or defense programs.
Databricks Certified Data Engineer Associate or Professional certification.
Experience working with financial management data.
Qlik dashboard and reporting experience.
Experience with secure federal data platforms such as Advana or Warfighter Data Platform (WDP).
Familiarity with cloud-native data engineering architectures and modern data integration frameworks. Technical Environment
Databricks
PySpark
SQL
Docker
Kubernetes
Git
CI/CD Pipelines
DevOps
Enterprise Data Platforms
Qlik Analytics