Python Data Engineer

Hi,     

This is Ravi from eCom Solutions Inc. I was reviewing your resume online and would like to talk to you regarding an exciting opportunity we have with one of eCom''s premier clients. Your experience looks like a good fit for the position and I wanted to know if you would be interested in exploring this opportunity.  

Job Title: Python Data Engineer

Location: Remote

Duration: Long Term

Job Description

We are seeking a hands-on Data Engineer to build scalable data ingestion and processing pipelines for scientific and imaging datasets. The role will focus heavily on Python-based data transformation, validation, metadata processing, and workflow automation, with a good understanding of Java-based applications and integrations.

Key Responsibilities

Develop high-performance data pipelines using Python, Pandas, PySpark, and SQL.

Ingest and process structured and unstructured data from APIs, databases, file systems, and cloud storage.

Implement data cleansing, transformation, validation, reconciliation, and data-quality controls.

Build reusable frameworks for batch and incremental data processing.

Integrate Python pipelines with Java-based services, APIs, and enterprise applications.

Implement logging, exception handling, monitoring, testing, and auditability.

Optimize pipeline performance, memory utilization, and processing time.

Required Skills

Strong hands-on Python experience for data engineering and large-scale data processing.

Experience with Pandas, PySpark, NumPy, SQL, REST APIs, and ETL/ELT pipelines.

Good understanding of Java, object-oriented programming, and Java-based application integration.

Experience with AWS services such as S3, Lambda, Glue, and Step Functions, or equivalent cloud services.

Knowledge of CSV, JSON, XML, Parquet, and scientific or imaging metadata formats.

Experience with Temporal, Git, CI/CD, unit testing, and production-quality coding standards.