Start searching
Back to Jobs
Recently postedTechnology

Data engineer

Cosmic-I LLC DBA Northern BaseUnited States🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationUnited States
DockerSQLGitApache SparkETLDatabricksPythonTerraformRESTAzureKubernetesCI/CDAzure DevOpsAzure Data FactoryDelta Lake

Job description

Role: Data Engineer

Loc: Remote
Duration: 8 weeks
Skill: Python, CI/CD, Azure

Job Title: Data Engineer
Required Skills: Python, Microsoft Azure, CI/CD
Job Type: Contract / Full-Time
Experience: 5+ years (preferred)

Job Summary

We are looking for an experienced Data Engineer with strong expertise in Python, Microsoft Azure cloud services, and CI/CD pipelines. The ideal candidate will design, develop, and maintain scalable data pipelines, build reliable data integration solutions, and implement automated deployment processes to support enterprise data engineering initiatives.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Python.

  • Build and optimize ETL/ELT workflows for processing structured and unstructured data.

  • Develop cloud-based data engineering solutions using Microsoft Azure services.

  • Work with Azure Data Factory, Azure Databricks, Azure Data Lake Storage, and Azure Synapse Analytics.

  • Implement and manage CI/CD pipelines using Azure DevOps, Git, and automated deployment tools.

  • Automate data processing, validation, testing, and deployment workflows.

  • Optimize data pipelines for performance, scalability, reliability, and cost efficiency.

  • Integrate data from APIs, databases, files, and other enterprise systems.

  • Implement data quality checks, monitoring, logging, and error-handling mechanisms.

  • Collaborate with data analysts, data scientists, architects, and development teams.

  • Follow best practices for data security, governance, access control, and version management.

Required Skills

  • Strong programming experience in Python.

  • Hands-on experience with Microsoft Azure cloud services.

  • Experience building ETL/ELT pipelines and data integration solutions.

  • Strong understanding of SQL and relational databases.

  • Experience with Azure Data Factory and Azure Data Lake Storage.

  • Experience implementing CI/CD pipelines using Azure DevOps.

  • Proficiency with Git, branching strategies, and version control.

  • Knowledge of automated testing, deployment, and infrastructure management.

  • Understanding of data modeling, distributed processing, and cloud architecture.

  • Strong troubleshooting, analytical, and problem-solving skills.

Preferred Qualifications

  • Experience with Azure Databricks and PySpark.

  • Knowledge of Azure Synapse Analytics and Delta Lake.

  • Experience with Docker, Kubernetes, or infrastructure as code.

  • Familiarity with Terraform or ARM/Bicep templates.

  • Experience working with REST APIs and event-driven data processing.

  • Knowledge of data governance, security, and compliance practices.

Key Technical Skills

Python | Microsoft Azure | Azure Data Factory | Azure Databricks | PySpark | SQL | Azure DevOps | CI/CD | Git | ETL/ELT | Azure Data Lake Storage | Azure Synapse Analytics

Similar jobs

Apply