Technology

Sr. Data Engineer with ETL

LiveMindzFort Mill, SC🇺🇸United StatesPosted Oct 21, 2026

Quick Overview

Seniority

Mid Senior

Work mode

Hybrid

Location

Fort Mill, SC, United States

Posted

22 hours ago

DockerSQLGitApache SparkAWSETLSnowflakeTableauAirflowCloudFormationDatabricksHadoopHiveJenkinsKafkaPythonRedshiftTerraformPostgreSQLCI/CDDelta Lake

Job Description

Role: Data engineering

Location: Fort Mill, SC

Type: W2

KEY HIGHLIGHTS

Brings 12 years of data engineering experience spanning production ETL/ELT pipelines, Big Data processing, cloud migrations, data lakes, and analytics platforms.
Builds and supports AWS Glue ETL pipelines using PySpark/Python, including ingestion, transformations, schema validation, data-quality checks, and curated data layers.
Develops event-driven and near-real-time architectures using AWS Lambda, EventBridge, Step Functions, Kafka/MSK, SNS, and SQS.
Has hands-on experience with Glue Catalog and Glue Crawlers for metadata discovery, schema registration, governance, and downstream querying through Athena.
Uses CloudWatch logs, metrics, monitoring, and alerts to troubleshoot Glue, Lambda, and Step Functions and maintain pipeline reliability and SLA compliance.
Has strong Python, PySpark, SQL, PostgreSQL, Redshift, Hive, Snowflake, and Databricks experience across large structured and semi-structured datasets.
Implements CDC, incremental processing, schema enforcement, error handling, retry logic, data-quality validation, and scalable batch/streaming pipelines using PySpark, Delta Lake, and Delta Live Tables.
Builds orchestration solutions using Step Functions, Airflow, Lambda, Control-M, and AWS services, managing workflow dependencies, scheduling, retries, and cross-service execution.
Automates infrastructure with Terraform and CloudFormation and supports CI/CD through Git, Jenkins, AWS Code Pipeline, and Docker.
Has extensive cloud-migration experience, including moving large on-premise Hadoop/Teradata workloads to AWS and designing scalable S3-based data lake architectures.
Brings hands-on performance tuning experience across Spark/PySpark jobs, SQL queries, Hive workloads, Redshift clusters, partitioning, and large-scale data pipelines.
Works effectively with business, BI, analytics, and downstream data consumers and has supported Tableau, Snowflake, Redshift, and Athena-based analytics environments.