Consultant - Databricks Azure

Job Requirements

Mandatory Skills

• Bachelor’s or higher degree in Computer Science or a related discipline; or equivalent (minimum 5+ years work experience).

• At least 3+ years of consulting or client service delivery experience on Azure Microsoft data engineering.

• At least 1+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Synapse/Azure Databricks, Microsoft Fabric

• Hands-on experience implementing data ingestion, ETL and data processing using Azure services: Fabric, onelake, ADLS, Azure Data Factory, Azure Functions, services in Microsoft Fabric etc.

• Minimum of 1+ years of hands-on experience in Azure and Big Data technologies such as Fabric, databricks, Python, SQL, ADLS/Blob, pyspark/SparkSQL.

• Minimum of 1+ years of RDBMS experience

• Experience in using Big Data File Formats and compression techniques.

• Experience working with Developer tools such as Azure DevOps, Visual Studio Team Server, Git, etc.

Primary Roles and Responsibilities

An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency.

Preferred Skills

• Experience developing and deploying ETL solutions on Azure cloud using ADF, Notebooks, Synapse analytics, Azure functions and other services.

• Experience developing and deploying ETL solutions on Azure cloud using services in Microsoft Fabric . Building medallion architecture using Fabric. Metadata driven ingestion frameworks, Building Data quality frameworks, Data transformation(SCD type1, SCD type 2) using Fabric notebooks or databricks

• Added advantage to have exposure to Microsoft Co-pilot, Gen AI fundamentals and industry supporting use cases.

• Microsoft certification’s role based. (DP600, DP203,DP900, AI102, AI900..)

• Knowledge of Microsoft Powerbi, reports/dashboards and generate insights for business users.

• Knowledge of Azure RBAC and IAM. Understanding access controls and security on Azure Cloud.

• Inclined with Microsoft vision and road map around latest tools and technologies in market.

An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency.

• Experience developing and deploying ETL solutions on Azure cloud using ADF, Notebooks, Synapse analytics, Azure functions and other services.

• Experience developing and deploying ETL solutions on Azure cloud using services in Microsoft Fabric . Building medallion architecture using Fabric. Metadata driven ingestion frameworks, Building Data quality frameworks, Data transformation(SCD type1, SCD type 2) using Fabric notebooks or databricks

• Added advantage to have exposure to Microsoft Co-pilot, Gen AI fundamentals and industry supporting use cases.

• Microsoft certification’s role based. (DP600, DP203,DP900, AI102, AI900..)

• Knowledge of Microsoft Powerbi, reports/dashboards and generate insights for business users.

• Knowledge of Azure RBAC and IAM. Understanding access controls and security on Azure Cloud.

• Inclined with Microsoft vision and road map around latest tools and technologies in market.