Data Engineer

Job at a Glance
Title: Data Engineer (onsite)
Location: Raleigh, NC
Contract: W2 only, 12-month contract with potential for extension or conversion to full time with the client.
Pay: $ 80-85/hour + optional medical, dental, vision, 401(k) match

Overview
Provide day-to-day support for Databricks notebooks, workflows, jobs, and data pipelines across Commercial and Digital platforms. Monitor, troubleshoot, and resolve production issues impacting data processing, reporting, and analytics. Perform root cause analysis for job failures and implement preventive measures to improve platform reliability. Support notebook optimization, code refactoring, performance tuning, and operational automation initiatives. Manage job scheduling, dependency management, and execution monitoring to ensure adherence to business SLAs.

Key Responsibilities

• Provide support for Databricks notebooks, workflows, jobs, and data pipelines.
• Monitor, troubleshoot, and resolve production issues impacting data processing, reporting, and analytics.
• Perform root cause analysis for job failures and implement preventive measures.
• Support notebook optimization, code refactoring, performance tuning, and operational automation initiatives.
• Manage job scheduling, dependency management, and execution monitoring to ensure adherence to business SLAs.
• Conduct ongoing data quality assessments and validations across commercial data assets.
• Investigate, troubleshoot, and resolve data discrepancies, data integrity issues, and pipeline failures.
• Implement automated data quality checks, reconciliation processes, and monitoring frameworks.
• Partner with business and IT stakeholders to identify and remediate data quality gaps affecting downstream analytics and reporting.
• Support data governance standards, metadata management, and auditability requirements.
• Evaluate and enhance the operational architecture supporting Commercial, Digital, and Ex-US data ecosystems.
• Identify opportunities to improve scalability, reliability, maintainability, and operational efficiency of data platforms.
• Develop and maintain operational runbooks, support documentation, monitoring dashboards, and standard operating procedures.
• Recommend architectural and process improvements that reduce operational overhead and improve system performance.
• Drive automation initiatives to minimize manual interventions and improve support responsiveness.
• Provide Level 2/3 support for data platform incidents and service requests.
• Coordinate issue resolution activities across business, IT, and vendor teams.
• Participate in incident triage, root cause analysis, and post-incident reviews.
• Ensure timely resolution of production issues while maintaining business continuity.
• Support ingestion, transformation, and distribution processes within Databricks and associated data platforms.
• Monitor pipeline health and proactively address processing bottlenecks and performance issues.
• Assist with data refreshes, environment migrations, deployment support, and release validation activities.
• Support ongoing enhancements and operational maintenance of commercial data assets.
• Collaborate with business users, analytics teams, data engineers, and platform administrators to address operational requirements.
• Provide regular operational status updates, issue tracking, and resolution reporting.
• Participate in governance reviews, support planning sessions, and operational improvement initiatives.
• Ensure adherence to established operational standards, security guidelines, and platform best practices.
• Maintain comprehensive documentation of notebooks, data flows, operational procedures, and support processes.
• Develop knowledge repositories and support materials to improve team efficiency and operational readiness.
• Support knowledge transfer activities and operational handoffs across teams.
• Identify opportunities to improve system stability, data accuracy, and operational performance.
• Support implementation of monitoring, alerting, and observability frameworks.
• Contribute to platform modernization and continuous improvement initiatives.
• Drive operational excellence through proactive issue prevention, process optimization, and automation.

Required Skills

• 10–12 years of overall experience, preferably with a Life Sciences background.
• Experience supporting Databricks notebooks, workflows, jobs, and data pipelines.
• Strong troubleshooting and root cause analysis skills.
• Experience with data quality assessments, validations, and automated checks.
• Knowledge of data governance standards, metadata management, and auditability.
• Ability to evaluate and improve operational architecture for scalability and reliability.
• Experience with incident management, issue resolution, and post-incident reviews.
• Familiarity with data ingestion, transformation, and distribution processes.
• Ability to monitor pipeline health and address processing bottlenecks.
• Experience with environment migrations, deployment support, and release validation.
• Strong collaboration skills with business and IT stakeholders.
• Excellent documentation and knowledge management skills.
• Ability to develop and maintain operational runbooks and dashboards.
• Proactive approach to operational support and automation initiatives.

Required Education

• No Education Requirements

Preferred Skills

• Experience with cloud platforms such as AWS.
• Knowledge of data quality frameworks and automation tools.
• Familiarity with data governance and compliance standards.
• Experience with data pipeline orchestration tools.
• Knowledge of SQL, Python, or Scala for data processing.
• Experience with monitoring and observability tools.
• Strong understanding of data security and privacy standards.

Why Should I Apply?
Join a dynamic team supporting critical data platforms that drive business insights. This role offers the opportunity to work on high-impact projects with a focus on operational excellence and automation.

About CEI:
As a trusted technology partner, CEI delivers solutions that help our customers transform their business and achieve meaningful results. From strategy and custom application development through application management - our technology and digital experience services are tailored to meet each unique need of our customers. Our staffing solutions bring specialized skills to complement our customers'' workforce and project requirements.

#ZR
#INDGEN