Start searching
Back to Jobs
Recently postedTechnology

Data Engineer

Yochana IT SolutionsCT🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationCT, United States
Apache SparkAWSDatabricksKafkaPythondbtAzure

Job description

Data Engineer

Connecticut (Remote)

Contract - 1 Year

Job Description:

  • Review and translate requirements to end-end technical design with a goal of building data products
  • Develop pipelines in Databricks using PySpark and DBT to onboard various sources
  • Working knowledge in Kafka to stream the data into databricks platform
  • Identify Common approaches and best practices to data integration such as metadata drive ingestion, data validations, complex data type handling
  • Drive standards for reusability, while developing pipelines
  • Create or update documentation associated to key decisions on requirements, design and development
  • Liaise with architects, data owners and users for reviews on day-day basis to ensure the quality of deliverables are high.

Skill Proficiency Requirements:

  • 8+ years of overall IT experience with 6 years as a Data Engineer
  • Experience in providing data engineering solutions
  • Working knowledge of below technology toolsets:
    • Cloud (Azure/AWS)
    • Databricks, Data Mesh, Data as a Product
    • Python
    • Data streaming (Kafka, EventHub, Kinesis, SNS)
    • Cloud-based API management good to have
    • Immuta good to have
  • Multi-cloud and on-prem application integration good to have

Similar jobs

Apply