Job Title: Senior Consultant – QA Automation Engineer (Databricks Data Platform)
Location: Manhattan, NY (Hybrid – 2 Days/Week)
Remote: Remote candidates will be considered if able to support EST business hours
Employment Type: Contract
Rate: $47–49/hr
We are seeking an experienced QA Automation Engineer with strong expertise in Databricks, AWS, and Delta Sharing to support enterprise-scale data platform initiatives. This role is ideal for professionals with a strong background in data validation, automation testing, and cloud-based data platforms who can ensure the quality, security, and reliability of modern data engineering solutions.
If you have hands-on experience testing Databricks pipelines, Medallion Architecture, Delta Sharing, and AWS-based data platforms, we''d like to hear from you.
Top Required Skills:
- Databricks
- AWS
- Delta Sharing
Key Responsibilities:
- Develop and execute comprehensive test plans, test cases, and validation strategies for Databricks-based data platforms.
- Validate data ingestion pipelines, ETL/ELT processes, and data transformations across Bronze, Silver, and Gold (Medallion) layers.
- Build and maintain automated data testing frameworks using SQL and automation tools to support continuous regression testing.
- Test Delta Sharing, Unity Catalog, role-based access controls (RBAC), and data security mechanisms to ensure secure and governed data access.
- Validate data quality, completeness, consistency, accuracy, and performance across enterprise data pipelines.
- Collaborate closely with Data Engineers, Product Owners, Developers, and Business Stakeholders to identify defects and improve overall data quality.
- Support release planning, deployment validation, production verification, and post-release testing.
- Track defects, report testing metrics, and communicate testing status, risks, and mitigation plans.
- Contribute to enterprise data governance initiatives, data catalog migration, and data quality framework adoption.
- Promote QA best practices and continuous improvement across the Databricks ecosystem.
Required Qualifications:
- 6–9 years of experience in QA Automation and Data Platform Testing.
- Strong hands-on experience with Databricks and AWS.
- Experience testing Delta Sharing, Unity Catalog, and enterprise data security controls.
- Strong SQL skills for data validation, reconciliation, and automated testing.
- Experience validating ETL/ELT pipelines and modern data architectures.
- Knowledge of Medallion Architecture (Bronze, Silver, Gold).
- Experience with automated testing frameworks for data platforms.
- Strong analytical, troubleshooting, and communication skills.
- Experience working in Agile/Scrum environments.
Preferred Qualifications:
- Experience with Data Governance, Data Quality, and Metadata Management.
- Exposure to cloud-native data engineering environments.
- Experience supporting enterprise-scale data modernization initiatives.
- Knowledge of CI/CD and DevOps practices for data platforms.
If you''re passionate about Data Quality, QA Automation, Databricks, and modern cloud data platforms, we''d love to hear from you.
