Data Engineer

Job Description:

Role: Senior Data Engineer

Location: Remote

Duration: Full Time

Job Description

Seeking a Senior Data Engineer with strong hands-on experience in Databricks, Spark, and AWS to build scalable, secure data pipelines and lakehouse solutions supporting analytics, AI/ML, and healthcare data.

Key Responsibilities

• Design and build data pipelines using Databricks, Delta Lake, Spark, Unity Catalog, and Workflows.
• Develop batch and real-time pipelines using AWS S3, Glue, Redshift, Athena, Lambda, Step Functions, and Kinesis.
• Build and optimize lakehouse architecture, data models, and ETL/ELT processes.
• Support ML/AI initiatives including feature stores, LLM/RAG data pipelines, and data integration.
• Implement data security, governance, and HIPAA-compliant data handling, including access controls and auditing.
• Partner with Data Science, Engineering, Product, and business teams.
• Review code, establish engineering best practices, and mentor junior engineers.

Requirements

• 8+ years of Data Engineering experience.
• 4+ years working with cloud-based data lake/lakehouse architectures.
• Deep hands-on experience with Databricks, Apache Spark/PySpark, Delta Lake, Unity Catalog, and Workflows.
• Strong AWS experience with S3 and data/analytics services.
• Strong Python and SQL skills.
• Experience with Airflow/dbt, Kafka/Kinesis, CI/CD, Docker/Kubernetes, and data quality practices.
• Healthcare or regulated-data experience with HIPAA required.
• Experience supporting ML/AI data pipelines and feature stores.
• Strong communication and technical leadership skills.

Thanks,

Suresh- Sr. Technical Recruiter