Senior Data Quality QA Engineer

Job Title: Senior Data Quality QA Engineer

Location: Minneapolis, MN (Hybrid, 3 days /week onsite)

Duration: Long Term Contract

Description:

The candidate should have hands-on experience with Databricks, AWS Glue, Amazon Redshift, SQL, Python, and cloud-based data platforms. Experience testing data migration, data transformation, reporting, and AI-enabled development workflows is highly preferred.

Responsibilities

• Design and execute end-to-end test strategies for data engineering and analytics projects.
• Validate ETL/ELT pipelines built using AWS Glue and Databricks.
• Perform data quality testing across multiple data sources and data warehouses.
• Develop SQL queries to validate source-to-target data transformations.
• Verify data completeness, consistency, accuracy, duplicates, and reconciliation.
• Perform testing of Amazon Redshift data warehouse solutions.
• Validate batch and incremental data loads.
• Create automated data validation frameworks using Python and SQL.
• Work closely with Data Engineers, AI Developers, Product Owners, and Business Analysts.
• Review business requirements and create detailed test scenarios and test cases.
• Identify, document, and track defects through resolution.
• Perform regression, integration, system, and user acceptance testing.
• Support CI/CD pipelines and automated testing processes.
• Use AI-assisted development tools (such as GitHub Copilot, Amazon Q, or similar AI coding assistants) to improve test automation, SQL generation, and productivity.
• Ensure data governance, data lineage, and quality standards are maintained.
• Participate in sprint planning, backlog grooming, and Agile ceremonies.
• Prepare test execution reports and quality metrics for stakeholders.

Required Skills

• 7–8 years of experience in Software QA with strong focus on Data Quality Testing.
• Strong SQL skills with experience writing complex queries.
• Hands-on experience with Databricks.
• Experience testing AWS Glue ETL pipelines.
• Experience validating data in Amazon Redshift.
• Strong knowledge of ETL/ELT testing methodologies.
• Experience with Python for test automation and data validation.
• Good understanding of cloud platforms, preferably AWS.
• Experience validating structured and semi-structured data (CSV, JSON, Parquet, Delta).
• Knowledge of data reconciliation and source-to-target validation.
• Experience using defect management tools such as Jira.
• Experience with Git and CI/CD pipelines.
• Strong analytical and troubleshooting skills.
• Excellent communication and documentation skills.

Preferred Skills

• Experience with AI-enabled software development and testing.
• Experience using GitHub Copilot, Amazon Q, ChatGPT, or similar AI tools.
• Knowledge of Apache Spark and PySpark.
• Experience with Delta Lake.
• Familiarity with Airflow or other workflow orchestration tools.
• Experience with Tableau, Power BI, or QuickSight report validation.
• Exposure to data governance and data quality tools.
• Experience working in Agile/Scrum environments.

Nice to Have

• Experience with machine learning data validation.
• Knowledge of Great Expectations or similar data quality frameworks.
• AWS Certification or Databricks Certification.
• Experience with large-scale enterprise data migration projects.