Quick overview
Job description
Senior Databricks Developer (AWS)
We are seeking a Senior Databricks Developer with 7+ years of Data Engineering experience and deep expertise in Databricks, AWS, PySpark, Python, and SQL. This is a highly technical, hands-on engineering role within a data engineering team focused primarily on building, enhancing, and supporting enterprise data pipelines. The ideal candidate will have extensive experience developing cloud-native data solutions in AWS, implementing Delta Lake and Medallion architectures, and delivering scalable, production-grade data pipelines using Databricks. This is not a consulting role; we are looking for a developer who enjoys building and optimizing data platforms and solving complex data engineering challenges.
Responsibilities
- Design, develop, and maintain large-scale ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
- Build scalable and highly reliable data ingestion, transformation, and orchestration frameworks within AWS.
- Develop and support Delta Lake implementations and Medallion Architecture (Bronze, Silver, Gold) data pipelines.
- Create and optimize batch and streaming pipelines that process large volumes of structured and unstructured data.
- Leverage AWS services including S3, Glue, Lambda, EMR, IAM, EventBridge, CloudWatch, Kinesis, and Redshift to support end-to-end data workflows.
- Optimize Spark jobs, Delta tables, and Databricks workloads for performance, scalability, and cost efficiency.
- Partner with Data Engineers, Architects, Analytics teams, and business stakeholders to deliver high-quality data solutions.
- Implement data quality controls, monitoring, alerting, and operational support processes.
- Troubleshoot and resolve complex production issues across Databricks and AWS environments.
- Contribute to engineering best practices, code reviews, automation, and platform modernization initiatives.
Required Qualifications
- 7+ years of Data Engineering experience building enterprise-scale data platforms and pipelines.
- 5+ years of hands-on Databricks development experience in production environments.
- Strong expertise in Python, PySpark, and SQL with a focus on pipeline development.
- Extensive experience developing and supporting data pipelines within AWS.
- Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion Architecture (Bronze, Silver, Gold).
- Hands-on experience with AWS services such as S3, Glue, Lambda, EMR, Redshift, Kinesis, IAM, and CloudWatch.
- Deep understanding of Spark architecture, distributed data processing, performance tuning, partitioning, and optimization.
- Experience building highly scalable batch and streaming data processing solutions.
- Strong understanding of data modeling, data warehousing concepts, and modern data architectures.
- Experience supporting production environments, monitoring data pipelines, and troubleshooting complex data issues.
Preferred Qualifications
- Experience with Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, and Structured Streaming.
- Experience with Kafka, Airflow, or cloud-native orchestration frameworks.
- Experience implementing CI/CD pipelines using GitHub Actions, Jenkins, or Azure DevOps.
- Infrastructure as Code experience utilizing Terraform or CloudFormation.
- Databricks Certified Data Engineer Associate or Professional certification.
- Experience working within large enterprise AWS environments supporting mission-critical data platforms.
Key Skills
Databricks, AWS, PySpark, Python, SQL, Delta Lake, Medallion Architecture, Lakehouse Architecture, ETL/ELT Pipelines, Spark Optimization, Delta Live Tables (DLT), Databricks Workflows, Unity Catalog, S3, Glue, Lambda, EMR, Kinesis, Redshift, CloudWatch, Kafka, Airflow, GitHub, CI/CD, Terraform.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- 2C
Full Stack Software Engineer with Security ClearanceRecently posted
22nd Century Technologies, Inc. · Washington, DC · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerAWSLoad Balancing+18TechnologySalary not listed - ST

Lead Radar Tracking Algorithm Developer with Security ClearanceRecently posted
STR · Atlanta, GA · Hybrid · Posted 18 Nov 2026 🇺🇸
MATLABPythonMachine LearningTechnology$174k - $225k/yr - SA
Sr. Systems Engineer (Angular-NodeJS FullStack Developer)Recently posted
SAIC · United States · Remote · Posted 18 Nov 2026 🇺🇸
DockerGitAWS+16Technology$80.0k - $120k/yr - B&

Web Developer or SharePointRecently posted
Bits & Bytes Technology Solutions · Albany, NY · Hybrid · Posted 18 Nov 2026 🇺🇸
PowerShellAzureJira+2TechnologySalary not listed - RS
Full Stack AI Architect (Python/React/Azure) - Remote PositionRecently posted
R Systems, Inc. · United States · Remote · Posted 17 Nov 2026 🇺🇸
LLMPythonVault+5Technology$60/hr - MA

Full Stack Software Engineer (Java/Spring Boot/Angular/PostgreSQ with Security ClearanceRecently posted
Markon · Annapolis Junction, MD · Remote · Posted 17 Nov 2026 🇺🇸
DockerMicroservicesSpring+16TechnologySalary not listed - MA

Senior Java Developer with Security ClearanceRecently posted
Markon · Annapolis Junction, MD · Hybrid · Posted 17 Nov 2026 🇺🇸
JavaJavaScriptMavenTechnology$120k - $200k/yr - CE

“Sr. .Net Developer” // “.Net AWS Lead” // “.Net Lead with OpenSearch”Recently posted
Centraprise Corp · Fort Mill, SC · On-site · Posted 17 Nov 2026 🇺🇸
Stakeholder ManagementDockerMicroservices+17TechnologySalary not listed - PS
Sr. Blue Prism RPA DeveloperRecently posted
Precision System Design Inc. · United States · Remote · Posted 17 Nov 2026 🇺🇸
SQLHTMLSplunk+3TechnologySalary not listed - PS
Full Stack Accessibility DeveloperRecently posted
Precision System Design Inc. · United States · Remote · Posted 14 Nov 2026 🇺🇸
HTMLAWSSnowflake+2TechnologySalary not listed - CA

Foundry Developer with Security ClearanceRecently posted
CACI · Ashburn, VA · Hybrid · Posted 17 Nov 2026 🇺🇸
Microsoft OfficeApache SparkScrum+11Technology$86.6k - $181.8k/yr - DV
Instrument Interface Developer / Senior Business Analyst – HL7 :: 100% RemoteRecently posted
DMS Vision Inc. · United States · Remote · Posted 17 Nov 2026 🇺🇸
SQLScrumJava+2TechnologySalary not listed