Data Engineer

Job Overview

We are seeking a hands-on Data Engineer with strong Microsoft Azure experience to design, develop, and maintain reliable, scalable data pipelines and datasets that support analytics and operational insights. The ideal candidate will have extensive experience with Azure Data Factory (ADF), Azure Blob Storage, Azure SQL, data modeling, and ETL/ELT processes.

Key Responsibilities

• Design, develop, schedule, and monitor batch and near-real-time data pipelines using Azure Data Factory (ADF).

• Manage Azure Blob Storage, including containers, access tiers, lifecycle policies, and data partitioning.

• Design and optimize Azure SQL databases, including schema design, indexing, query optimization, and performance tuning.

• Develop dimensional data models using star and snowflake schemas, 3NF, and Data Vault 2.0 concepts.

• Implement data quality, validation, governance, lineage, and schema evolution standards.

• Optimize pipeline performance, implement monitoring and alerting, and troubleshoot production issues.

• Collaborate with analytics, product, and operations teams to deliver secure, reliable, and well-documented data products.

• Apply security best practices, including RBAC, encryption, data masking, and key management.

• Develop reusable components and follow best practices for data engineering and CI/CD.

Required Qualifications

• Bachelor's degree with 5 years of experience, or 4 years of experience in lieu of a degree.

• Ability to obtain and maintain a Public Trust clearance, requiring U.S. citizenship.

• Strong hands-on experience with Azure Data Factory, Azure Blob Storage, and Azure SQL.

• Knowledge of star/snowflake schemas, 3NF, Data Vault, and Kimball vs. Inmon data warehousing methodologies.

• Proficiency in SQL and data formats such as JSON, Parquet, CSV, and Avro.

• Experience with Python (preferred) or Scala for data transformations and automation.

• Experience with Git-based workflows and Azure DevOps or equivalent CI/CD tools.

• Experience monitoring, troubleshooting, and maintaining production data pipelines.

Preferred Qualifications

• Experience with Azure Synapse Analytics and large-scale ETL/ELT processing.

• Familiarity with Microsoft Fabric, Lakehouse, OneLake, and data pipelines.

• Power BI semantic modeling, DAX, and incremental refresh experience.

• Knowledge of Event Hubs, Azure Functions, Databricks, or Spark Structured Streaming.

• Experience with Terraform, Bicep, or ARM templates.

• Familiarity with Microsoft Purview or similar data cataloging and lineage tools.

I look forward to your response. Please feel free to reach out if you have any questions.