Azure Data Engineer

Location: Hamilton, New Jersey (Local candidates preferred for onsite/F2F interview)

Job Summary

We are looking for an experienced Azure Data Engineer with strong expertise in Azure data platforms, ETL/ELT development, big data processing, and data warehousing. The ideal candidate should have experience in the Banking, Financial Services, and Insurance (BFSI) domain and be comfortable working with distributed teams across design, development, and testing phases.

Required Technical Skills

• Azure Data Factory (ADF)
• Azure Databricks
• Python
• Apache Spark
• SQL Server
• Azure Cosmos DB
• Azure Data Lake
• Data Warehouse Design
• ETL/ELT Development
• Data Integration & Data Engineering

Desired Skills

• Power BI
• Azure DevOps
• Event Hub
• Data Governance
• GDPR/HIPAA Compliance
• Performance Tuning & Cost Optimization

Key Responsibilities

Data Ingestion & Integration

• Design and develop scalable data ingestion pipelines using Azure Data Factory and Event Hub.
• Integrate data from multiple sources including on-premises systems, cloud platforms, APIs, and databases.
• Ensure reliable, secure, and efficient movement of data across environments.

Data Transformation

• Build ETL/ELT workflows using Databricks, Spark, and Python.
• Cleanse, transform, normalize, and enrich data for analytics and reporting.
• Implement business rules and data validation processes.

Data Storage & Management

• Design, implement, and maintain Azure Data Lake and Data Warehouse solutions.
• Optimize storage architecture for scalability, performance, and cost efficiency.
• Manage structured and unstructured data using SQL Server and Cosmos DB.

Collaboration

• Partner with business stakeholders, architects, analysts, and development teams.
• Ensure data quality, availability, and consistency for reporting and analytics initiatives.
• Work effectively with multi-location teams.

Security & Compliance

• Implement data governance and security best practices.
• Ensure compliance with BFSI regulations and standards, including GDPR and HIPAA requirements.
• Maintain data privacy and access controls.

Performance & Cost Optimization

• Monitor, troubleshoot, and optimize data pipelines and workloads.
• Improve performance of Databricks jobs and Spark workloads.
• Leverage Azure services for dynamic scaling and cost management.

Required Qualifications

• Strong BFSI domain experience.
• Hands-on experience with Azure Data Factory, Databricks, Python, Spark, SQL Server, and Cosmos DB.
• Experience designing enterprise-scale data pipelines and data warehouse solutions.
• Excellent communication and stakeholder management skills.
• Experience working with geographically distributed teams.