Data Engineer (Mid)

Overview

We are seeking a mid-level Data Engineer to join a growing Enterprise AI team. This team is focused on building modern data platforms and AI-enabled solutions that drive business value, operational efficiency, and informed decision-making.

If you enjoy solving complex data challenges, building scalable data products, and working at the intersection of data engineering and AI, this is an excellent opportunity to make a meaningful impact.

About the Role

You will join a multidisciplinary team of data engineers, AI/ML engineers, data scientists, and product stakeholders building enterprise data and AI solutions.

In this role, you will build reliable data foundations that power AI-enabled products, operational intelligence, and business decision-making. You'll integrate enterprise data with AI-generated insights and contribute reusable data capabilities across a variety of AI and analytics initiatives.

This is a hands-on individual contributor role. You'll own well-defined data engineering components from development through production, partnering with senior engineers and architects on solution design and technical direction.

Key Responsibilities

• Build, maintain, and enhance scalable batch and near real-time data pipelines using Python, SQL, and Spark/PySpark.
• Integrate data from enterprise applications, operational systems, APIs, event streams, and AI inference outputs into reliable, analysis-ready datasets.
• Develop transformations that combine structured and semi-structured data across sources using consistent identifiers and timestamps, including handling time zones, duplicate records, and late-arriving events.
• Create and maintain data models and curated datasets for business analytics, AI model development and evaluation, and AI-enabled application workflows.
• Partner with AI/ML engineers to deliver data interfaces for inference workflows and capture model outputs, confidence scores, and user feedback to support continuous improvement.
• Implement data quality checks, reconciliation processes, monitoring, and alerting to ensure completeness, accuracy, freshness, and traceability.
• Troubleshoot pipeline failures and performance issues while implementing reliable retries, backfills, and incremental processing strategies.
• Collaborate with product managers, data scientists, analysts, and engineering partners to translate business requirements into practical data solutions.
• Follow data security, privacy, governance, access control, and retention requirements when handling enterprise and AI-generated data.
• Write maintainable, tested code and participate in code reviews, CI/CD practices, and documentation of schemas, data lineage, dependencies, and operational procedures.

Required Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Information Systems, Engineering, Mathematics, Statistics, or a related quantitative discipline, or equivalent practical experience.
• 3-5 years of experience in data engineering or a closely related software engineering role.
• Strong hands-on proficiency with Python and SQL, including complex queries, transformations, optimization, and troubleshooting.
• Experience with Spark/PySpark and distributed data processing frameworks.
• Experience designing, building, and supporting ETL/ELT pipelines on cloud platforms such as Azure or AWS.
• Understanding of data modeling, relational databases, data lakes, data warehouses, and incremental processing patterns.
• Experience with workflow orchestration tools such as Apache Airflow or Databricks Workflows.
• Familiarity with event-driven and streaming data processing concepts including event time management, deduplication, and reliable delivery.
• Experience with Git, automated testing frameworks, monitoring tools, and production support.
• Strong communication skills and the ability to collaborate effectively with both technical and non-technical stakeholders.

Preferred Qualifications

• Hands-on experience with Databricks, Delta Lake, or lakehouse architectures.
• Experience with Databricks Asset Bundles (DABs) or similar deployment automation frameworks supporting CI/CD processes.
• Experience with streaming technologies such as Kafka, Azure Event Hubs, or Spark Structured Streaming.
• Experience working with customer, inventory, operational, financial, or transactional datasets.
• Experience preparing datasets for machine learning, anomaly detection, predictive analytics, or AI model evaluation.
• Familiarity with multimodal data, unstructured data processing, and time-based event correlation.
• Exposure to edge-to-cloud integration patterns, containerized workloads, Kubernetes, Docker, or infrastructure-as-code tools such as Terraform or Bicep.