Why This Role Stands Out
This hybrid Data Tech Lead role offers a fantastic opportunity to design and develop cutting-edge cloud-based data platforms and streaming pipelines, leveraging your Java and Databricks expertise. You'll thrive here if you're a proactive problem-solver passionate about building scalable data solutions and contributing to impactful cloud migration initiatives. Apply today to join a dynamic team and advance your skills in a collaborative environment.
Quick Overview
Seniority
Mid Senior
Work mode
Hybrid
Location
Alpharetta, GA, United States
Posted
4 weeks ago
SQLETLSnowflakeApacheAzureCassandraDatabricksGitLab CIJavaKafkaPostgreSQLPower BIPython
Job Description
| Data Tech Lead with Java Alpharetta GA (or) Berkley Hills NJ 1. Role Overview Experienced Data Technical Lead to design, develop, and support scalable cloud-based data platforms and streaming data pipelines. The ideal candidate should possess strong expertise in Databricks, PySpark, Python, Java-based streaming technologies, GitLab CI/CD pipelines, and cloud migration initiatives. 2. Key Responsibilities - Design and develop scalable Databricks ETL/ELT pipelines (Lakeflow & LakeBase) using Azure Databricks, PySpark, and Python. - Implement real-time and batch data ingestion frameworks using Kafka and Java-based streaming solutions. - Develop and optimize data processing workflows in Azure Databricks. - Integrate and manage data movement between PostgreSQL, YugabyteDB (Cassandra-based NoSQL), and cloud platforms. - Build reusable frameworks for data ingestion, transformation, validation, and orchestration. - Develop SQL-based data transformations, reporting datasets, and performance optimization solutions. - Design and implement GitLab CI/CD pipelines for automated deployment, testing, and release management of Databricks notebooks, jobs, and data pipelines. - Support Snowflake on-premises to Azure cloud migration initiatives. - Ensure coding standards, performance tuning, monitoring, and operational stability of data pipelines. - Develop Power BI dashboards and reports for business intelligence and analytics reporting. - Develop API automation and integration solutions for data exchange between enterprise systems. 3. Required Skills. - Strong experience in Python and PySpark development. - Hands-on experience with Azure Databricks and databricks SQL. - Experience in Java-based streaming and ingestion frameworks. - Strong knowledge of Apache Kafka streaming concepts. - Experience working with PostgreSQL databases. - Experience with YugabyteDB or Cassandra-based NoSQL databases. - Strong SQL development and query optimization skills. - Hands-on experience with GitLab CI/CD pipeline development and deployment automation. - Understanding cloud-based data engineering and distributed processing concepts. - Experience in data migration projects, especially Snowflake on-prem to Azure cloud migration. - Experience designing enterprise-scale data lake or lakehouse architectures. - Knowledge of streaming architectures and real-time analytics. - Familiarity with cloud monitoring and observability tools. |
--
Similar jobs
- IG
Model-Based Systems Engineer with Security Clearance
NewInsight Global, Inc.
San Antonio, TX🇺🇸HybridYesterdayTechnology - RC
Python Application Developer
NewR Cube Creative Consulting Inc
Plano, TX🇺🇸HybridYesterdayDjangoDockerFastAPI+13Technology - AT
Guidewire PC Tech Lead (ACE certified)
Aroha Technologies
San Antonio, TX🇺🇸On-site6 weeks ago - AA
Engineer 1- Full Time, $42.15/Hour
NewAulani, A Disney Resort & Spa
Kapolei, Hawaii🇺🇸$42/hrHybrid32 minutes agoHVACMicrosoft OfficeEngineering - BO
Senior Software Engineer Mission Systems Software Developer
NewBoeing
Daytona Beach, Florida🇺🇸$184.4k - $249.6k/yrHybrid32 minutes agoAgileC#Confluence+6Technology - AI
Robotics Software Engineer with Security Clearance
NewAnduril Industries
Atlanta, GA🇺🇸$165k - $218k/yrHybrid2 days agoDroneRustEmbedded Systems+6Technology