Spark Data Engineer (Secret to TS/SCI Poly- Cleared) with Security Clearance

Zachary Piper Solutions is seeking a Spark Data Engineer to support a federal customer in the DC metro area. This role focuses on big data engineering, optimization, and performance tuning using Apache Spark. The ideal candidate will have strong experience in data engineering, SQL query optimization, and distributed computing. Customer-facing skills and excellent communication are essential for success in this position.

Work Environment: Washington, DC metro area; hybrid and remote considered for exceptional candidates Clearance: Minimum for a DoD Secret clearance, can hold up to a TS/SCI w/ FSP Responsibilities: • Design and implement scalable data pipelines and architectures using Apache Spark. • Optimize Spark jobs for performance and efficiency across large datasets. • Collaborate with clients to understand requirements and deliver technical solutions. • Support data engineering tasks including ETL, data warehousing, and query optimization. • Assist with integration of Spark-based solutions into cloud environments (AWS, Azure, GCP). • Provide technical guidance on best practices for big data processing and analytics. • Participate in short- to medium-term customer engagements, ensuring successful delivery of projects.

Qualifications: • Bachelor’s degree in Computer Science or related field required. • Minimum 5 years of experience in data engineering and distributed computing. • Strong expertise in Apache Spark and performance optimization techniques. • Proficiency in SQL and experience with data warehousing and query tuning. • Familiarity with Python or Scala programming languages. • Working knowledge of cloud ecosystems (AWS, Azure, or GCP). • Excellent communication and customer-facing skills. • Ability to manage scope, timelines, and deliverables in technical projects.

Preferred: • Experience with machine learning and data science concepts. • Familiarity with CI/CD pipelines and MLOps practices. • Knowledge of Databricks platform and big data architecture design.

Compensation: • Salary Range: $150,000 - $260,000 (based on experience). • Benefits: Cigna medical, dental, vision, 401k, 20 days paid time off, 11 federal holidays, and sick leave as required by law.