Senior Data Engineer

Title: Senior Data Engineer

Location : 100% Remote

Job Type: C2C

- Engagement: Long-term, client-facing (healthcare domain)
- Experience: 8+ years in data engineering

JD:
1. Databricks depth (must be hands-on and current). Real, recent Databricks/PySpark/Spark SQL/Delta Lake experience. Look for depth in Spark performance tuning (partitioning, skew handling, caching, AQE, Z-ordering), Delta Lake (ACID, schema evolution, time travel), and ideally Unity Catalog and Delta Live Tables.
2. Azure-native stack. Azure, so we strongly prefer Azure over AWS/Google Cloud Platform: Azure Data Factory (ADF), ADLS Gen2, Azure Databricks, Synapse, and Entra ID.
3. Healthcare + HIPAA experience. Hands-on experience with healthcare data (claims, eligibility, provider/member data, EHR, or EDI 837/835/834) and, importantly, HIPAA/PHI compliance in practice masking/de-identification, RBAC, encryption, and audit controls.

JOB DESCRIPTION
Senior Data Engineer (Onshore)
- Design, build, and maintain scalable data pipelines and ETL/ELT workflows.
- Develop and optimize data models, data warehouses, and integration processes.
- Ingest, transform, and consolidate data from multiple structured and unstructured sources.
- Ensure data quality, integrity, and consistency across systems.
- Optimize data storage, processing performance, and query efficiency.
- Implement data governance, security, and compliance best practices.
- Collaborate with analytics, BI, and application teams to meet data requirements.
- Develop and maintain documentation for data architecture and pipelines.
- Monitor, troubleshoot, and resolve data pipeline and performance issues.
- Provide technical guidance to junior engineers and contribute to best practices.
- Support Agile delivery and provide regular status updates.

Gaurav Tripathi

Sr Associate Talent Acquisition

KAnand Corporation

Email: