Senior Databricks Engineer

Position: Senior Databricks Data Engineer
Location: Remote / Hybrid
Duration: Long Term

Responsibilities

• Develop scalable data pipelines using Databricks, PySpark and SQL.
• Build and maintain Delta Lake tables using the Medallion Architecture.
• Develop batch and real-time data processing pipelines.
• Implement data ingestion using Auto Loader, APIs, Kafka or cloud storage.
• Optimize Spark jobs, SQL queries and Delta tables.
• Implement data governance and security using Unity Catalog.
• Develop Databricks Workflows/Jobs and monitor production pipelines.
• Integrate Databricks with AWS/Azure/Google Cloud Platform data services.
• Implement CI/CD and source-code management using Git and DevOps tools.
• Collaborate with data architects, analysts, ML engineers and business teams.

Sincerely,

Sr. Manager

nFolks Data Solutions LLC 

Phone:  
Email: