Quick overview
Job description
Senior Data Engineer - Data Quality & Observability
About the Opportunity
A highly regarded investment-focused organization is seeking a Senior Data Engineer to lead the design and evolution of enterprise data quality and observability capabilities across a large-scale research and analytics platform. This role is responsible for ensuring that critical business data is accurate, timely, traceable, and production-ready before it reaches end users.
The position sits at the intersection of data engineering, platform reliability, governance, and automation. The successful candidate will develop reusable quality controls, improve operational visibility across data pipelines, and establish engineering standards that support trusted decision-making across the business.
What You'll Be Doing
Build and Scale Data Quality Solutions
- Architect and develop a shared framework for validating and monitoring data across multiple platforms and domains.
- Create configurable quality checks and reusable components that can be adopted by engineering teams across the organization.
- Implement automated testing and validation processes that identify issues before they impact downstream users or applications.
Improve Visibility Into Data Health
- Develop monitoring, alerting, and operational dashboards that provide real-time insight into data reliability and pipeline performance.
- Establish measurable quality standards and operational metrics to track platform health.
- Enhance transparency through lineage, metadata, ownership, and auditability initiatives.
Strengthen Data Governance and Integrity
- Ensure consistency and accuracy of complex financial and investment-related datasets throughout their lifecycle.
- Validate historical data accuracy, revisions, and source changes to maintain confidence in analytical outputs.
- Implement controls that support traceability, reproducibility, and regulatory-grade data management practices.
Enhance Platform Reliability
- Design preventative controls that stop incomplete, invalid, or high-risk data from progressing through production systems.
- Lead investigation and resolution efforts for data incidents, collaborating with stakeholders to restore service and prevent recurrence.
- Drive improvements in operational readiness, resiliency, and recovery procedures.
Cloud & Infrastructure Engineering
- Manage quality and observability solutions within a modern cloud environment.
- Automate deployments, testing, and infrastructure management using infrastructure-as-code and CI/CD practices.
- Optimize solutions for scalability, maintainability, cost efficiency, and long-term operational support.
Partner Across Teams
- Collaborate closely with data engineers, software engineers, product teams, and business stakeholders.
- Define data quality standards, operational procedures, and best practices.
- Produce documentation, implementation guidance, and onboarding materials that promote consistent adoption across the organization.
Required Experience
- 6+ years of experience in Data Engineering, Software Engineering, or a related technical discipline.
- Strong expertise building production-scale data platforms and reliability-focused solutions.
- Advanced proficiency with Python and SQL.
- Hands-on experience with Snowflake, Apache Airflow, AWS, and modern CI/CD practices.
- Experience implementing data validation frameworks, monitoring solutions, reconciliation processes, and operational controls.
- Strong understanding of data lineage, metadata management, historical data accuracy, and reference data concepts.
- Experience working with financial, market, investment, or enterprise business datasets.
- Ability to troubleshoot complex data issues and deliver scalable long-term solutions.
Preferred Experience
- Experience with graph databases such as Neo4j.
- Familiarity with data observability and quality platforms including Great Expectations, Soda, Grafana, OpenTelemetry, or related technologies.
- Knowledge of security master data, entity resolution, and cross-source data integration.
- Exposure to AI-enabled analytics or research platforms.
- Experience with anomaly detection, statistical profiling, testing frameworks, or reliability engineering practices.
- Advanced degree in Computer Science, Engineering, Mathematics, Physics, or another quantitative discipline.
Technology Stack
- Python
- SQL
- Snowflake
- Apache Airflow
- AWS
- Terraform
- Git
- CI/CD Tooling
- Neo4j (preferred)
- Data Quality & Observability Tools (Great Expectations, Soda, Grafana, OpenTelemetry, OpenLineage, etc.)
Why This Role Matters
This position will be instrumental in establishing trusted, high-quality data across a sophisticated investment research ecosystem. Success in the role means creating scalable controls, reducing operational risk, improving observability, and enabling stakeholders to confidently rely on business-critical data products and analytics.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- KS

SAS EngineerRecently posted
Keypixel Software Solutions · United States · Hybrid · Posted 18 Nov 2026 🇺🇸
Data AnalysisEngineeringSalary not listed - CG
Sr. Data Engineer with Security ClearanceRecently posted
Contact Government Services, LLC · Albany, NY · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLShellGit+10Technology$144.8k - $209.1k/yr - PE

Sr. Data Engineer (OBI Advanced Analytic Method Augmentation) with Security ClearanceRecently posted
Prescient Edge · Washington, DC · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerSQLETL+5Technology$160k - $180k/yr - AP

Sr Snowflake Data EngineerRecently posted
Aptino · United States · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLApache SparkETL+3TechnologySalary not listed - AT
Azure Data EngineerRecently posted
Arbor Tek Systems · Cincinnati, OH · On-site · Posted 18 Nov 2026 🇺🇸
GitApache SparkETL+7TechnologySalary not listed - DT
Data Engineer -Scala Expert /Remote/10+yrs onlyRecently posted
Digipulse Technologies, Inc · United States · Remote · Posted 18 Nov 2026 🇺🇸
SQLScalaGit+4TechnologySalary not listed - KT

Data Engineer - Mid LevelRecently posted
Kforce Technology Staffing · Irving, TX · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLScalaETL+2TechnologySalary not listed - MT
Senior Kafka Engineer / AdministratorRecently posted
Microgreen Technologies LLC · St. Louis, MO · On-site · Posted 18 Nov 2026 🇺🇸
EngineeringSalary not listed - CT
Senior Kafka Engineer / AdministratorRecently posted
Chabez Tech LLC · St. Louis, MO · On-site · Posted 18 Nov 2026 🇺🇸
EngineeringSalary not listed - AT
Salesforce Data Migration Lead
Atika Tech · Holmdel, NJ · On-site · Posted 7 Nov 2026 🇺🇸
Stakeholder ManagementMySQLOracle+7TechnologySalary not listed - PB
Software Data Engineer (SWD6) with Security ClearanceRecently posted
Point Blank Technologies · Laurel, MD · Hybrid · Posted 18 Nov 2026 🇺🇸
AWSPowerShellPython+3TechnologySalary not listed - AT

Data Engineer || Remote || W2 RoleRecently posted
AKAASA Technologies · United States · Remote · Posted 18 Nov 2026 🇺🇸
SQLGitApache Spark+16TechnologySalary not listed
