Haystack
← Back to Jobs
Remote
Other
EP

Data Engineers

Empower ProfessionalsUnited States🇺🇸United StatesPosted Sep 18, 2026

Quick Overview

Seniority
Mid Senior
Work mode
Remote
Location
United States
Posted
17 hours ago
SQLT-SQLData PipelineReconciliationScheduling

Job Description

Role: Data Engineer – SQL to Microsoft Fabric Modernization/Migration
Location: 100% remote
This is a 10-week project (that is about 2.5 months) – candidates MUST be ok with that duration
Targeting October 12th or 19th as a start date

This is the formal job description:
We are seeking a Data Engineer to support a Microsoft Fabric modernization initiative focused on migrating existing on-premises SQL Stored Procedures workloads into a Fabric Lakehouse environment. This role will be responsible for converting SQL/T-SQL stored procedures into Fabric notebooks, building and orchestrating data pipelines, validating migrated datasets, and creating technical documentation to support ongoing operations.
This position will primarily support the Silver Layer within a Medallion Architecture and will play a critical role in ensuring data accuracy, consistency, and scalability throughout the migration effort.
Team structure is small, must be highly motivated and self-motivated. Will have access to some SMEs but this role is primarily responsible for the vast majority of the work here.
Must be comfortable working in a small team with one other to deliver fast and quick with great quality.
Key Responsibilities

Convert existing SQL/T-SQL stored procedures into Microsoft Fabric Notebooks using PySpark and Spark SQL.

Design and build the Silver Layer of a medallion architecture in Fabric using Lakehouse, ensuring parity between Fabric data structures and existing on-premises SQL environments.

Develop and maintain orchestration frameworks using Microsoft Fabric Data Pipelines.

Build automated validation and reconciliation processes to ensure migrated data matches source systems on a 1:1 basis.

Create reusable testing frameworks and validation artifacts to continuously verify data integrity and accuracy.

Support data pipeline monitoring, scheduling, and orchestration activities within Microsoft Fabric.

Partner with architects, engineers, and business stakeholders to ensure successful migration outcomes.

Develop and maintain technical documentation detailing solution design, implementation approaches, validation procedures, and operational support processes.

Analyze and troubleshoot data issues across Fabric notebooks, pipelines, and Lakehouse objects.


Required Qualifications

5-7+ years of experience as a Data Engineer or similar role.

1-2+ years of hands-on experience with Microsoft Fabric.

Strong expertise in SQL and T-SQL, including:

Stored procedures

Query optimization

Data transformations

SQL-based business logic

Experience migrating or translating SQL-based processing into modern cloud-based data platforms.

Strong hands-on experience with:

PySpark

Spark SQL

Microsoft Fabric Notebooks

Experience developing and managing data pipelines and orchestration workflows.

Deep understanding of:

Lakehouse architecture

Microsoft Fabric data flows

Data Pipelines

Medallion Architecture concepts

Experience building automated data quality, reconciliation, and validation frameworks.

Strong technical documentation and communication skills.


Preferred Qualifications

Experience working with metadata-driven frameworks.

Experience implementing orchestration and scheduling strategies within Microsoft Fabric.

Familiarity with enterprise data migration initiatives.

Experience supporting cloud-based analytics and modern data platform solutions.
Technical Environment

Microsoft Fabric

Fabric Notebooks

Fabric Data Pipelines

Lakehouse Architecture

Medallion Architecture (Silver Layer Focus)

SQL / T-SQL

PySpark

Spark SQL

Similar jobs

Browse related jobs

Apply now