Quick Overview
Job Description
We are seeking a Senior Data Software Engineer to implement monitoring, incident management, and operational capabilities for an automated Data Platform Monitoring solution. In this role, you will design and maintain robust monitoring rules, incident automation, and operator health views to ensure the reliability and observability of data products and pipelines.
Responsibilities Implement monitoring rules for data pipeline health, freshness, volume, run outcomes, and other agreed monitoring objectives Develop data processing and transformation logic for monitoring events and operational data Build automated Jira incident creation based on detected monitoring events Ensure incidents contain the required information, including component, region, DAG/job, run, timestamp, detecting check, error reference, and severity Support incident recurrence handling, stakeholder notification, and monitoring of incident status Develop and maintain the operator workspace/health view for monitoring data products and pipeline status Enable integration of monitoring data with Snowflake and other components of the solution Participate in testing monitoring rules and incident automation against real and historical failures Troubleshoot issues, improve monitoring logic, and support solution handover Requirements 3+ years of experience in data engineering roles Expertise in Apache Airflow, Snowflake, and dbt Proficiency in building and maintaining data pipelines, transformation logic, and operational data flows Background in implementing monitoring, observability, and alerting for data platforms Skills in incident automation and integration with tools such as Jira Familiarity with applying AI capabilities within data platform or monitoring solutions Understanding of pipeline health metrics, including freshness, volume, and run outcomes Capability to troubleshoot complex data platform issues and drive continuous improvement English proficiency at an Upper-Intermediate level (B2) or higher
Similar jobs
- QU
System Engineer (SDA) - Enterprise with Security Clearance
NewQinetiQ US
Chantilly, VA🇺🇸$175k - $220k/yrHybridYesterdayMachine LearningTechnology - US
Senior Embedded Imaging Software Engineer
NewUrban Sky
Denver, Colorado🇺🇸$140k - $180k/yrOn-site1 hour agoFPGACUDAC+++1Technology - OI
Senior Software Engineer
NewOPUS IVS INC
Dexter, Michigan🇺🇸Hybrid1 hour agoDjangoFlaskAWS+12Technology - IW
Search Engineer
NewInfo Way Solutions
Austin, TX🇺🇸On-site22 hours agoGenerative AIEngineering - NE
Application Software Engineer
NewNeotecra, Inc
New York, NY🇺🇸Hybrid22 hours agoDockerFirebaseSQL+9Technology - SA
Principal - Senior Embedded Software Engineer
NewStaffRight Associates, LLC
San Diego, CA🇺🇸$150k - $220k/yrOn-site22 hours agoAssemblyC++Python+1Technology - LT
Specialist, Systems Engineering with Security Clearance
NewL3Harris Technologies
Canoga Park, CA🇺🇸$92k - $171k/yrOn-site22 hours agoEngineering - LT
Manager, Software Engineer with Security Clearance
NewL3Harris Technologies
Rochester, NY🇺🇸$117k - $218k/yrOn-site22 hours agoFPGAScrumCSS+3Technology - SS
AI Immersive Coach / Senior Software Engineer – AI Coach
NewSpear Staffing
Dallas, TX🇺🇸Hybrid22 hours agoGitJavaPython+4Technology - RD
Software Engineer III
NewRandstad Digital
Charlotte, NC🇺🇸$65 - $75/hrHybrid22 hours agoSQLGitAWS+15Technology - QU
Space Systems Engineer - Integration with Security Clearance
NewQinetiQ US
Chantilly, VA🇺🇸$165k - $225k/yrHybrid22 hours agoConfluenceMachine LearningJira+1Technology - LT
Senior Specialist, Software Engineer with Security Clearance
NewL3Harris Technologies
Colorado Springs, CO🇺🇸$111.5k - $207.5k/yrHybrid22 hours agoSAFeDockerMicroservices+16Technology