Back to Jobs
Technology

Data Integration Engineer

Virtualan SoftwareLibertyville, IL🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationLibertyville, IL, United States
FastAPIMicroservicesSQLETLSnowflakeApacheKafkaPythonRESTCI/CD

Job description

Job Title: Senior Apache Kafka DeveloperExperience: 7–10+ YearsLocation: Mettwa, IL, USA (4 days at work)Employment Type: Contract / Full-Time
Industry: Life science is preferred We are seeking a highly skilled Data Integration Engineer to design, build, andoptimize our next-generation data streaming pipelines and integration services. In thisrole, you will be responsible for building real-time and batch data pipelines that bridgeour operational systems, cloud data warehouse, and Master Data Management (MDM)platforms. You will leverage a modern data stack centered around Python, FastAPI,Apache Kafka, Snowflake, and Reltio to ensure our data is fluid, synchronized, andhighly reliable. Pipeline Development: Design and deploy scalable real-time streamingpipelines using Confluent Apache Kafka and robust batch ETL/ELT workflowsusing Python. API Ingestion & Microservices: Build and maintain high-performance,asynchronous REST APIs and data connectors using FastAPI to facilitateseamless app-to-app data integrations. Data Warehousing: Optimize data extraction from Snowflake using Snowpipe,streaming ingestion frameworks, and advanced SQL transformations into Kafkatopics Master Data Management (MDM): Integrate data sources with the Reltioplatform via the Reltio Integration Hub (RIH) and APIs to manage entityresolution, data cleansing, and consolidate golden customer/product profiles. System Monitoring: Implement automated monitoring, error handling, validationchecks, and data quality routing across all integration layers. Collaboration & DevOps: Work closely with Data Science, Analytics, andSoftware Engineering teams to understand data requirements and implementCI/CD deployment pipelines for data services.Required Skills:CoreProgramming
Strong Python proficiency with hands-on experience utilizingasynchronous frameworks like FastAPI.
Streaming &Messaging
Deep understanding of Apache Kafka architecture (topics,partitions, producers/consumers, schema registries).
DataWarehousing
Expert knowledge of Snowflake architecture, queryoptimization, and native ingestion tooling.
MDM Platforms Direct experience working with Reltio multidomain MDM,
workflow modelers, or Reltio data pipelines.
Database Skills Advanced SQL capability alongside knowledge of transactional
databases, NoSQL systems, and JSON parsing.
Experience required:5 years of industry experience implementing data integration projects.
Reltio or MDM or Kafka or Integration engineer

Similar jobs