Azure Data Engineer

Azure Data Engineer
Location: Pleasanton, CA (on site 3 days)

Data engineer Jobd 
Azure Data Factory (ADF) Expert

Overview:

Azure Data Factory (ADF) Expert to design, develop, and manage data integration solutions on Microsoft Azure. The ideal candidate will have deep expertise in ADF pipelines, data flows, and integration with various Azure services, ensuring secure and efficient data movement across systems.

Key Responsibilities

Design and implement data pipelines using Azure Data Factory for ETL processes.

Develop and optimize data flows for batch and real-time data integration.

Integrate ADF with Azure services such as Azure Synapse, Data Lake, and Blob Storage.

Monitor, troubleshoot, and enhance ADF pipelines for performance and reliability.

Ensure data security and compliance with organizational standards.

Collaborate with data engineers, architects, and business stakeholders to deliver solutions.

Document processes and provide knowledge transfer to team members.

Required Skills

Expertise in Azure Data Factory (ADF) pipeline and data flow development.

Strong understanding of ETL concepts and data integration patterns.

Experience with Azure services: Data Lake, Blob Storage, Synapse Analytics.

Proficiency in SQL and data transformation techniques.

Knowledge of Azure security and role-based access control.

Hands-on experience with monitoring and troubleshooting ADF pipelines.

Good to Have

Experience with Azure DevOps for CI/CD of ADF pipelines.

Knowledge of Power BI or other visualization tools.

Familiarity with Python or .NET for custom activities in ADF.

Understanding of big data technologies and data warehousing concepts