Quick Overview
Job Description
• Must have strong hands-on experience working with Databricks, SQL, and Python to build, support, and optimize enterprise data pipelines in production environments.
• Proven expertise in writing advanced SQL queries using joins, window functions, CTEs, and performance tuning techniques to deliver high-quality and scalable data solutions.
• Strong Python development experience using PySpark and pandas for data processing, automation, transformation, and pipeline development.
• Hands-on experience designing and supporting Medallion Architecture, including Bronze, Silver, and Gold data layers to ensure trusted and well-governed analytical datasets.
• Experience with data warehouse and database modeling, including dimensional modeling, normalized schemas, and designing structures that support reporting and analytics.
• Strong understanding of data engineering best practices including data quality validation, monitoring, automated testing, troubleshooting, documentation, and production support.
• Experience developing, maintaining, and optimizing data pipelines while performing root-cause analysis and resolving data issues in complex environments.
• Ability to work closely with Data Engineers, Analysts, and business stakeholders to translate business requirements into scalable data solutions and trusted datasets.
• Experience working with retail, wholesale distribution, supply chain, inventory, merchandising, pricing, logistics, or sales data domains is highly preferred.
• Familiarity with modern data engineering tools and technologies such as Databricks, Delta Lake, Spark, Unity Catalog, Airflow, Azure Data Factory, Snowflake, Redshift, or BigQuery is a strong plus.
Similar jobs
- CI
AWS Data Engineer with AI Skills - Charlotte, NC (Hybrid 2 Days/Week Onsite)
NewCuboid IT Solutions
Charlotte, NC🇺🇸On-site19 hours agoSQLAWSETL+8Technology - GI
Data Engineer
NewGeorgia IT
Boston, MA🇺🇸Hybrid19 hours agoSQLSQL ServerETL+7Technology - NI
Hadoop and PySpark Data Engineer
NewNityo Infotech Corporation
Jersey City, NJ🇺🇸Hybrid19 hours agoSQLShellAgile+4Technology - DT
Sr. Data Engineer
NewDonato Technologies Inc
United States🇺🇸Hybrid19 hours agoOracleSQLETL+3Technology - SD
AI Data Engineer
NewStanley David and Associates
United States🇺🇸Hybrid19 hours agoAWSETLSnowflake+2Technology - LS
kafka engineer
NewLincoln Softtech LLC
Irvine, CA🇺🇸Hybrid19 hours agoSAFeAgileEngineering