Haystack
← Back to Jobs
Other
PI

Databricks Platform Manager/Engineer

Prama Innovations India Pvt. Ltd.New York, NY🇺🇸United StatesPosted 21 Aug 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
New York, NY, United States
Posted
Yesterday
AWSSSOSplunkAzureComplianceDatabricksDatadogGitGitHub ActionsJenkinsTerraformUnity

Job Description

Job Title: Databricks Platform Manager/Engineer
Location: NY (Hybrid Onsite)
Type: Contract
 
Job Summary
We are looking for a Databricks Platform Manager/Engineer with strong hands-on experience administering enterprise Databricks environments on AWS. The role focuses on platform administration, workspace and cluster management, Unity Catalog, AWS IAM, security/governance, Terraform/IaC, CI/CD, monitoring, and cost optimization. Ideal candidates will have 3–5+ years of production Databricks platform administration experience and a strong AWS/cloud infrastructure background.
Key Responsibilities
  • Administer and support Databricks workspaces across Dev, Test, UAT, and Production.
  • Manage clusters, cluster policies, instance pools, job clusters, and Databricks Runtime upgrades.
  • Administer Unity Catalog, catalogs, schemas, permissions, and RBAC.
  • Manage AWS IAM integration, SSO, service principals, secrets, and access controls.
  • Implement and support Terraform/IaC and CI/CD for Databricks platform deployments.
  • Manage Git/Databricks Repos and automated deployment processes.
  • Monitor platform health, performance, jobs, clusters, and production environments.
  • Troubleshoot platform issues, incidents, and performance problems.
  • Support audit logging, security, governance, and compliance requirements.
  • Identify opportunities for Databricks and AWS cost optimization.
  • Partner with Data Engineering, DevOps, Cloud, Security, and Infrastructure teams.
Required Skills
  • Strong hands-on Databricks Platform Administration
  • Databricks Workspace Management
  • Cluster Management & Cluster Policies
  • Unity Catalog
  • Databricks Runtime
  • Service Principals, Secrets & PATs
  • AWS IAM
  • AWS S3, VPC, EC2, KMS, Security Groups
  • Terraform / Infrastructure as Code
  • Git and CI/CD
  • Azure DevOps, GitHub Actions, or Jenkins
  • Platform Monitoring & Troubleshooting
  • Production Support
  • Cost Optimization
Preferred
  • Experience supporting enterprise Databricks environments with 100+ users
  • Experience with AWS PrivateLink / VPC Endpoints
  • AWS Organizations / Control Tower
  • CloudWatch, Datadog, or Splunk
  • Delta Lake / Lakehouse architecture
  • Exposure to Spark internals
  • Experience supporting Databricks AI/ML workloads
  • Databricks or AWS certifications

Similar jobs

Browse related jobs

Apply now