Senior Data Engineer

Role: Senior Data Engineer

Location: Richmond/Mc Lean, VA (Locals Only)

Exp: 7 years

Mandatory: Ex-employee Capital One

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data platforms. The ideal candidate should have strong experience in Python, SQL, ETL/ELT, cloud data platforms, and distributed data processing.

Key Responsibilities

• Design and develop scalable ETL/ELT data pipelines for large-volume datasets.
• Develop data solutions using Python, SQL, Spark/PySpark, and related technologies.
• Build and optimize data warehouses, data lakes, and data processing workflows.
• Work with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Develop data integrations from databases, APIs, files, and other sources.
• Implement data quality, validation, monitoring, and performance optimization.
• Work with Kafka or other streaming technologies for real-time data processing.
• Collaborate with Data Scientists, Architects, BI teams, and business stakeholders.
• Troubleshoot production data issues and ensure pipeline reliability.
• Follow best practices for CI/CD, version control, security, and data governance.

Required Skills

• 7+ years of experience in Data Engineering.
• Strong Python and SQL skills.
• Hands-on experience with Apache Spark/PySpark.
• Strong experience in ETL/ELT and data pipeline development.
• Experience with at least one cloud platform: AWS/Azure/Google Cloud Platform.
• Experience with data warehouses such as Snowflake, Redshift, BigQuery, or Azure Synapse.
• Experience with Airflow, Databricks, or similar orchestration/data platforms.
• Good understanding of relational and NoSQL databases.
• Experience with Git, CI/CD, and Agile development.
• Strong problem-solving and communication skills.