Back to Jobs
Recently postedOther

Senior Quality Engineer – Data Platform

Millennium Global TechnologiesDallas, TX🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationDallas, TX, United States
SQLGitApache SparkAWSETLAirflowApacheContinuous ImprovementPythonReconciliationpytestRESTAzureKubernetesAgileCI/CD

Job description

Senior Quality Engineer – Data Platform
Mission
Own comprehensive, risk-based validation of complex IRIS data-platform capabilities and provide actionable
evidence of functional, data, integration, resiliency, and release quality, using production-quality Python
automation, reusable validation frameworks, and automated CI/CD quality gates.
Key responsibilities
 Design, develop, test, and maintain production-quality Python test automation, applications, reusable
components, and validation frameworks that support complex platform quality-engineering initiatives.
 Design risk-based test strategies for complex features, integrations, data flows, and releases.
 Build and maintain scalable automated suites for REST APIs, JSON contracts, ETL/ELT pipelines,
transformations, orchestration workflows, Spark-based processing, Kubernetes-based services, and cloud
dependencies.
 Apply strong software-design practices, including object-oriented programming, modular design,
parameterization, configuration management, structured logging, exception handling, packaging, and debugging,
to Python automation frameworks and validation utilities.
 Design complex data-reconciliation approaches that validate source-to-target completeness, accuracy,
transformation logic, duplicates, null handling, referential integrity, and exception behavior.
 Lead unit, regression, integration, end-to-end, negative, boundary, failure, recovery, resiliency, performance, and
scalability validation for assigned capabilities.
 Create reusable Python frameworks, validation libraries, templates, fixtures, controlled test data, and self-service
capabilities that are adopted across workflows and teams.
 Partner with data consumers, application teams, platform engineers, and software engineers to translate
requirements and risks into validation approaches and improve the quality of delivered datasets.
 Analyze complex application defects, pipeline failures, performance bottlenecks, and data-quality issues across
application, data, orchestration, and environment layers; isolate failure conditions and provide actionable root-
cause evidence.
 Validate application and pipeline behavior across supported cloud and Kubernetes configurations, dependencies,
scaling conditions, and recovery scenarios.
 Implement and use observability, monitoring, logging, and alerting capabilities to validate data quality, application
reliability, and timely pipeline execution across cross-component workflows.
 Maintain traceability among requirements, risks, tests, defects, execution results, and release-quality evidence;
identify systemic coverage gaps and drive remediation through test design, engineering improvements, or explicit
risk acceptance.
 Embed automated quality gates into CI/CD pipelines and drive cloud-native development, DevOps, and quality-
engineering practices across applicable platform components.
 Drive improvements in test automation, pipeline reliability, performance, and operational efficiency through
reusable engineering patterns and continuous automation.
 Review test designs and automation code, mentor Quality Engineers, and contribute technical documentation,
knowledge sharing, and reusable implementation examples.
 Use AI-assisted software and test development responsibly to accelerate Python coding, test generation, test-
data creation, debugging, troubleshooting, and documentation while maintaining documented human review of
all generated outputs for correctness, security, maintainability, relevance, and coverage.
Expected deliverables
 Risk-based feature and release test strategies.
 Production-quality Python test automation, validation frameworks, and reusable components committed to
source control.
 Cross-component automated unit, functional, API, data, regression, integration, and end-to-end test suites.

 Reusable Python frameworks, libraries, fixtures, templates, controlled test data, and self-service validation
capabilities adopted across workflows.
 Complex data-reconciliation, resiliency, and failure/recovery validation assets.
 Defect evidence, root-cause analyses, and retest confirmation for complex cross-component issues.
 Requirements-to-test traceability, quality-risk assessments, and release-readiness evidence.
 Automated CI/CD quality gates, observability capabilities, and actionable diagnostic information.
 Test reviews, mentoring outputs, coverage-improvement plans, technical documentation, and knowledge-transfer
materials.
Required skills and experience
 Strong Python software-development and automated quality-engineering experience, including object-oriented
programming, classes, modules, functions, parameters, exception handling, reusable component design, and
pytest or comparable testing frameworks.
 Proficiency with Python logging, configuration management, command-line parameters, debugging, structured
error-handling patterns, dependency management, and Python packaging.
 Experience writing unit, integration, regression, and end-to-end tests for Python applications, automation
frameworks, reusable components, and transformation logic.
 Knowledge of software-design principles, maintainable coding practices, source control, code reviews, reusable
framework design, and automated quality gates.
 Strong SQL development, query optimization, data validation, and data-reconciliation capabilities.
 Experience with data engineering and the development and testing of ETL/ELT pipelines.
 Strong hands-on experience with Apache Spark or PySpark and distributed data processing.
 Experience using Python for data transformation, test automation, validation, and operational tooling.
 Strong hands-on experience with Apache Airflow or comparable workflow-orchestration platforms.
 Knowledge of data modeling, data warehousing, source-to-target mapping, and transformation concepts.
 Strong REST API testing, JSON validation, data-pipeline testing, negative testing, boundary testing, and failure-
recovery validation capabilities.
 Strong hands-on experience validating containerized applications, data workloads, and services in Kubernetes-
based environments.
 Strong experience with cloud-native platforms and AWS services; experience with Azure is also beneficial.
 Strong hands-on experience with Git, CI/CD, DevOps practices, automated quality gates, and deployment-
validation processes.
 Experience with observability, structured logging, monitoring, alerting, diagnostics, and operational analytics.
 Knowledge of data quality, lineage, governance, traceability, and release-evidence practices.
 Ability to create reproducible defects, isolate failure conditions, perform root-cause analysis, assess risk, and
communicate actionable quality evidence.
 Demonstrated commitment to automation, reusable engineering patterns, continuous improvement, and
operational excellence.
 Strong hands-on proficiency with data reconciliation, enterprise data-platform testing, resiliency testing, test-
framework development, and root-cause analysis.
 Demonstrated ability to lead validation of complex cross-component changes and guide other quality engineers.
 Knowledge of AI-assisted software development, including the effective use of coding copilots and large
language models to accelerate development, testing, test-data creation, documentation, debugging, and
troubleshooting.
 Demonstrated ability to review and validate AI-generated code, tests, documentation, and recommendations for
correctness, security, maintainability, relevance, and coverage.
Preferred qualifications
 Experience with non-functional testing, distributed systems, production incident analysis, or data observability.
 Experience creating testing frameworks or common enterprise QE assets.
 Experience communicating release risk to senior technical and product stakeholders.
 Experience working in Agile product teams and vendor-delivered engagements.

Similar jobs