Start searching
Back to Jobs
Recently postedTechnology

Senior Data Engineer

Long Finch TechnologiesAtlanta, GA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationAtlanta, GA, United States
SAFeSQLScalaGitApache SparkAWSETLScrumTableauApacheBigQueryDatabricksGoogle CloudHadoopHiveKafkaKanbanPythonUnityAgile

Job description

Job Description

We are looking for an experienced Senior Data Engineer with strong expertise in Databricks, Python, PySpark, Apache Spark, and large-scale data engineering. The ideal candidate will be responsible for designing and developing scalable ETL pipelines, implementing data processing solutions, managing data ingestion frameworks, and optimizing cloud-based data platforms. The role also involves technical leadership, data governance, and collaboration with analytics and business intelligence teams.

Required Technical Skills

  • Strong hands-on experience with Databricks, Delta Lake, Delta Live Tables, and Unity Catalog.

  • Proficiency in Python, PySpark, Spark SQL, and Apache Spark.

  • Strong experience in ETL development, data pipeline design, and data transformation.

  • Knowledge of Hadoop, Kafka, Hive, HDFS, and Sqoop.

  • Expertise in SQL, data modeling, and data warehousing.

  • Experience with batch and real-time data processing using Kafka and Spark Streaming.

  • Cloud experience with AWS and/or Google Cloud Platform.

  • Strong understanding of data quality, validation, cleansing, and governance.

  • Experience working in Agile environments (Scrum, Kanban, SAFe).

Preferred Skills

  • Scala / Scala-Spark, IBM DataStage, TIBCO.

  • AWS S3, Glue, Lambda, Kinesis/MSK.

  • Tableau, Power BI, Spotfire, OAC.

  • Teradata, DB2, BigQuery, Mainframe.

  • Git, data governance, and retention modeling.

Roles & Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL solutions.

  • Build data ingestion and transformation frameworks for structured and unstructured data.

  • Develop and optimize batch and streaming data processing solutions.

  • Implement data quality, validation, and governance standards.

  • Integrate heterogeneous data sources into enterprise data platforms.

  • Support analytics, reporting, BI, and machine learning initiatives.

  • Collaborate with Data Scientists, BI teams, and business stakeholders.

  • Lead technical discussions, mentor junior engineers, and drive data engineering best practices.

Similar jobs

Apply