Quick overview
Job description
Job Description
Skill: Azure Data Engineer - Lead
Roles & Responsibilities:
- Lead the Azure Data Engineering team and provide day-to-day technical direction, delivery oversight, and issue resolution.
- Design, develop, orchestrate, and maintain scalable ADF pipelines for batch ingestion, data integration, transformation, and downstream processing.
- Develop complex and reusable data-transformation frameworks using PySpark and Azure Databricks notebooks.
- Design metadata-driven ingestion, transformation, validation, reconciliation, logging, and restart/recovery components.
- Build and maintain solutions across ADLS Gen2, Azure Databricks, SQL databases, storage accounts, Azure Key Vault, and upstream/downstream enterprise systems.
- Migrate RDBMS and traditional ETL workloads to modern Data Lake, ELT, and cloud-native processing patterns.
- Optimize PySpark workloads for partitioning, caching, shuffle reduction, file sizing, scalability, and efficient processing of large datasets.
- Implement data-quality rules, control totals, exception handling, auditability, monitoring, alerting, and operational support mechanisms.
- Support data lake and SQL Server control/governance frameworks, including standards, templates, naming conventions, and directory structures.
- Analyze defects, incidents, change requests, and enhancement needs; perform root-cause analysis and implement sustainable fixes.
- Convert functional and business requirements into technical designs, estimates, implementation plans, and deployment activities.
- Perform code reviews and enforce engineering standards, security practices, source-control discipline, CI/CD, and documentation quality.
- Coordinate with onsite architects, business analysts, QA teams, application teams, production support teams, and customer stakeholders.
- Own offshore deliverables, dependencies, risks, status reporting, release readiness, and post-production stabilization.
- Mentor team members, build technical capability, and ensure high-quality, timely completion of deliverables.
Insurance / Reinsurance Domain Expectations:
The candidate should have practical experience working with Insurance and/or Reinsurance data, preferably across several of the following areas:
- Policy, contract, coverage, and exposure data.
- Premium, billing, claims, and loss transactions.
- Broker, client, insured, and ceding company / cedant information.
- Treaty and facultative reinsurance concepts.
- Reinsurance premiums, recoveries, claims, and settlements.
- Catastrophe, event, and loss-related datasets.
- Financial, accounting, bordereaux, and reconciliation feeds.
- Regulatory, management, and downstream analytical reporting
Similar jobs
- DC

Data EngineerRecently posted
Dahl Consulting · United States · Remote 🇺🇸
SQLScalaGit+13Technology$67 - $83/hr - SI

Data Engineer With API gateways Exp -- 100% remote -- Only W2Recently posted
Siri Infosolutions Inc · United States · Remote 🇺🇸
SQLAPI GatewaySelenium+5TechnologySalary not listed - ET

Data Engineer / Data Scientist – Microsoft Fabric & Purview
Edify Technologies, Inc. · Irving, TX · Hybrid 🇺🇸
SQLETLDatabricks+6Technology$50 - $60/hr - D&
Data Engineer III - Rosslyn, VA - Active Top Secret Clearance with Security ClearanceRecently posted
Day & Zimmermann Federal Services Career Portal · Rosslyn, VA · On-site 🇺🇸
SQLApache SparkAirflow+3Technology$122.5k - $175k/yr - AG

Data EngineerRecently posted
ASCII Group LLC · New York, NY · On-site 🇺🇸
Stakeholder ManagementSQLApache Spark+12Technology$70/hr - SY

PostgreSQL DBARecently posted
Symphony Corporation · United States · Remote 🇺🇸
OracleShellGit+11TechnologySalary not listed - VC

Data Engineer LeadRecently posted
Vish Consulting Services, Inc. · United States · Hybrid 🇺🇸
SQLGitETL+4TechnologySalary not listed - PD

SAP Data EngineerRecently posted
PY DATA, INC. · United States · Hybrid 🇺🇸
SQLETLTechnologySalary not listed - VA
Data EngineerRecently posted
Valueprosite · Texas City, TX · Hybrid 🇺🇸
DockerMicroservicesOracle+20TechnologySalary not listed - CS

Operational Data EngineerRecently posted
Cynet Systems · New York, NY · Hybrid 🇺🇸
SQLGitSOC 2+3Technology$50 - $55/hr - HS
Data EngineerRecently posted
Hierarch Soft Technologies, Inc. · Charlotte, NC · Hybrid 🇺🇸
SQLApache SparkETL+6TechnologySalary not listed - KI

Lead Data EngineerRecently posted
Kanshe Infotech · Houston, TX · Hybrid 🇺🇸
GitApache SparkAWS+7TechnologySalary not listed