Back to Jobs
Recently postedTechnology

Gen Re-Azure Data Engineer Lead-

Purple Drive Technologies LLCStamford, CT🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeOn Site
LocationStamford, CT, United States
SQLApache SparkETLDatabricksVaultAzureSQL ServerCI/CD

Job description

Job Description

Skill: Azure Data Engineer - Lead

Roles & Responsibilities:

  • Lead the Azure Data Engineering team and provide day-to-day technical direction, delivery oversight, and issue resolution.
  • Design, develop, orchestrate, and maintain scalable ADF pipelines for batch ingestion, data integration, transformation, and downstream processing.
  • Develop complex and reusable data-transformation frameworks using PySpark and Azure Databricks notebooks.
  • Design metadata-driven ingestion, transformation, validation, reconciliation, logging, and restart/recovery components.
  • Build and maintain solutions across ADLS Gen2, Azure Databricks, SQL databases, storage accounts, Azure Key Vault, and upstream/downstream enterprise systems.
  • Migrate RDBMS and traditional ETL workloads to modern Data Lake, ELT, and cloud-native processing patterns.
  • Optimize PySpark workloads for partitioning, caching, shuffle reduction, file sizing, scalability, and efficient processing of large datasets.
  • Implement data-quality rules, control totals, exception handling, auditability, monitoring, alerting, and operational support mechanisms.
  • Support data lake and SQL Server control/governance frameworks, including standards, templates, naming conventions, and directory structures.
  • Analyze defects, incidents, change requests, and enhancement needs; perform root-cause analysis and implement sustainable fixes.
  • Convert functional and business requirements into technical designs, estimates, implementation plans, and deployment activities.
  • Perform code reviews and enforce engineering standards, security practices, source-control discipline, CI/CD, and documentation quality.
  • Coordinate with onsite architects, business analysts, QA teams, application teams, production support teams, and customer stakeholders.
  • Own offshore deliverables, dependencies, risks, status reporting, release readiness, and post-production stabilization.
  • Mentor team members, build technical capability, and ensure high-quality, timely completion of deliverables.

Insurance / Reinsurance Domain Expectations:

The candidate should have practical experience working with Insurance and/or Reinsurance data, preferably across several of the following areas:

  • Policy, contract, coverage, and exposure data.
  • Premium, billing, claims, and loss transactions.
  • Broker, client, insured, and ceding company / cedant information.
  • Treaty and facultative reinsurance concepts.
  • Reinsurance premiums, recoveries, claims, and settlements.
  • Catastrophe, event, and loss-related datasets.
  • Financial, accounting, bordereaux, and reconciliation feeds.
  • Regulatory, management, and downstream analytical reporting

Similar jobs