Quick overview
Job description
Responsibilities:
- Assist with leading the team's transition to the Databricks platform and utilize the newer features of Delta Live Tables, Workflows etc.
- Design and develop data pipelines that extract data from Oracle, load it into the data lake, transform it into the desired format, and load it into Databricks data lakehouse.
- Optimize data pipelines and data processing workflows for performance, scalability, and efficiency.
- Implement data quality checks and validations within data pipelines to ensure the accuracy, consistency, and completeness of data.
- Help create and maintain documentation for data mappings, data definitions, architecture and data flow diagrams.
- Build proof-of-concepts to determine viability of possible new processes and technologies.
- Deploy and manage code in non-prod and prod environments.
- Investigate and troubleshoot data related issues and fix or provide solutions to fix defects.
- Identify and resolve performance bottlenecks, which could include suggesting ways to optimize and performance tune databases and queries to enhance query performance.
Requirements:
- Bachelor's Degree in Computer Science, Data Science, Software Engineering, Information Systems, or related quantitative field.
- 4+ years of experience working as a Data Engineer, ETL Engineer, Data/ETL Architect or similar roles.
- Current/active Databricks Data Engineer/Analyst certification a BIG plus.
- 3+ years working with Databricks with knowledge and expertise of data structures, data storage and change data capture gained from prior production implementations of data pipelines, optimizations, and best practices.
- Solid continuous experience in Python.
- 3+ years of experience in Kimball dimensional modeling (star-schema comprising of facts, type1 and type2 dimensions, aggregates, etc.) with solid understanding of ELT/ETL.
- 3+ years of solid experience writing SQL and PL/SQL code.
- 2+ years of experience with Airflow.
- 3+ years of experience working with relational databases (Oracle preferred).
- 2+ years of experience working with NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar.
- 2+ years of cloud experience (Azure preferred).
- Experience with CI/CD utilizing git/Azure DevOps.
- Experience with storage formats including Parquet/Arrow/Avro.
#RecruitPS
Similar jobs
- BR

Data Engineer/ArchitectRecently posted
Blue Rose Technologies LLC · Fremont, CA · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLApache SparkETL+3TechnologySalary not listed - MA

Snowflake AdminRecently posted
Magicforce · United States · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLGitAWS+7TechnologySalary not listed - ID
Sr AWS Data Engineer (with ETL, Python, AWS Glue, Lambda, Node.js & Microservices exp)Recently posted
Info Dinamica Inc · Malvern, PA · On-site · Posted 16 Nov 2026 🇺🇸
DynamoDBMicroservicesOracle+18TechnologySalary not listed - PG

Data Engineer – Databricks, PySpark, Cloud | Hybrid - 71701Recently posted
PRIMUS Global Services Inc. · Princeton, NJ · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLScalaApache Spark+4TechnologySalary not listed - PH

Data EngineerRecently posted
Photon · Sunnyvale, CA · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLAWSETL+6TechnologySalary not listed - GE

SQL Server DBARecently posted
Genesis10 · Charlotte, NC · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLShellMicrosoft Office+3Technology$53 - $61/hr - NT
Oracle DBA with Security ClearanceRecently posted
NorthHill Technology Resources · Fairfax, VA · Hybrid · Posted 17 Nov 2026 🇺🇸
MariaDBMongoDBOracle+6TechnologySalary not listed - IT

ETL LeadRecently posted
ITSYNTAX INC · Dallas, TX · Hybrid · Posted 16 Nov 2026 🇺🇸
OraclePL/SQLSQL+6Salary not listed - VI

Sr. Data EngineerRecently posted
Visa · Foster City, California · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLScalaApache Spark+14Technology$131.9k - $204k/yr - VI

Staff Data EngineerRecently posted
Visa · Foster City, California · Hybrid · Posted 17 Nov 2026 🇺🇸
SQLScalaApache Spark+13Technology$156.8k - $251.1k/yr - FA

Systems Engineer (Data Flow/NiFi) Level 2Recently posted Auto Apply
Fantom Corporation · Fort Meade · On-site · Posted 17 Nov 2026 🇺🇸
ApacheComplianceComputer Vision+5TechnologySalary not listed - LS

Senior Database EngineerRecently posted
London Stock Exchange Group · St Louis, MO · Hybrid · Posted 17 Nov 2026 🇺🇸
OracleShellAWS+5TechnologySalary not listed