Back to Jobs
Recently postedFull timeTechnology

Senior Data Engineer

Selby JenningsWestport, CT🇺🇸United States

Quick overview

SeniorityMid Senior
Employment typeFull Time
Work modeHybrid
LocationWestport, CT, United States
Neo4jSQLGitAWSSnowflakeAirflowApacheGrafanaPythonTerraformCI/CD

Job description


Senior Data Engineer - Data Quality & Observability
About the Opportunity

A highly regarded investment-focused organization is seeking a Senior Data Engineer to lead the design and evolution of enterprise data quality and observability capabilities across a large-scale research and analytics platform. This role is responsible for ensuring that critical business data is accurate, timely, traceable, and production-ready before it reaches end users.

The position sits at the intersection of data engineering, platform reliability, governance, and automation. The successful candidate will develop reusable quality controls, improve operational visibility across data pipelines, and establish engineering standards that support trusted decision-making across the business.


What You'll Be Doing

Build and Scale Data Quality Solutions

  • Architect and develop a shared framework for validating and monitoring data across multiple platforms and domains.
  • Create configurable quality checks and reusable components that can be adopted by engineering teams across the organization.
  • Implement automated testing and validation processes that identify issues before they impact downstream users or applications.

Improve Visibility Into Data Health

  • Develop monitoring, alerting, and operational dashboards that provide real-time insight into data reliability and pipeline performance.
  • Establish measurable quality standards and operational metrics to track platform health.
  • Enhance transparency through lineage, metadata, ownership, and auditability initiatives.

Strengthen Data Governance and Integrity

  • Ensure consistency and accuracy of complex financial and investment-related datasets throughout their lifecycle.
  • Validate historical data accuracy, revisions, and source changes to maintain confidence in analytical outputs.
  • Implement controls that support traceability, reproducibility, and regulatory-grade data management practices.

Enhance Platform Reliability

  • Design preventative controls that stop incomplete, invalid, or high-risk data from progressing through production systems.
  • Lead investigation and resolution efforts for data incidents, collaborating with stakeholders to restore service and prevent recurrence.
  • Drive improvements in operational readiness, resiliency, and recovery procedures.

Cloud & Infrastructure Engineering

  • Manage quality and observability solutions within a modern cloud environment.
  • Automate deployments, testing, and infrastructure management using infrastructure-as-code and CI/CD practices.
  • Optimize solutions for scalability, maintainability, cost efficiency, and long-term operational support.

Partner Across Teams

  • Collaborate closely with data engineers, software engineers, product teams, and business stakeholders.
  • Define data quality standards, operational procedures, and best practices.
  • Produce documentation, implementation guidance, and onboarding materials that promote consistent adoption across the organization.

Required Experience
  • 6+ years of experience in Data Engineering, Software Engineering, or a related technical discipline.
  • Strong expertise building production-scale data platforms and reliability-focused solutions.
  • Advanced proficiency with Python and SQL.
  • Hands-on experience with Snowflake, Apache Airflow, AWS, and modern CI/CD practices.
  • Experience implementing data validation frameworks, monitoring solutions, reconciliation processes, and operational controls.
  • Strong understanding of data lineage, metadata management, historical data accuracy, and reference data concepts.
  • Experience working with financial, market, investment, or enterprise business datasets.
  • Ability to troubleshoot complex data issues and deliver scalable long-term solutions.

Preferred Experience
  • Experience with graph databases such as Neo4j.
  • Familiarity with data observability and quality platforms including Great Expectations, Soda, Grafana, OpenTelemetry, or related technologies.
  • Knowledge of security master data, entity resolution, and cross-source data integration.
  • Exposure to AI-enabled analytics or research platforms.
  • Experience with anomaly detection, statistical profiling, testing frameworks, or reliability engineering practices.
  • Advanced degree in Computer Science, Engineering, Mathematics, Physics, or another quantitative discipline.

Technology Stack
  • Python
  • SQL
  • Snowflake
  • Apache Airflow
  • AWS
  • Terraform
  • Git
  • CI/CD Tooling
  • Neo4j (preferred)
  • Data Quality & Observability Tools (Great Expectations, Soda, Grafana, OpenTelemetry, OpenLineage, etc.)

Why This Role Matters

This position will be instrumental in establishing trusted, high-quality data across a sophisticated investment research ecosystem. Success in the role means creating scalable controls, reducing operational risk, improving observability, and enabling stakeholders to confidently rely on business-critical data products and analytics.


Will your resume get past this employer's screening? Check your resume against this job, free

Similar jobs