Azure Databricks Admin

Scope of Work – Azure Databricks Platform Administration

Location: Wilmington, DE- Onsite

Mode: Fulltime

The responsibilities for the Azure Databricks platform administration should include, but are not limited to:

· Platform Setup & Configuration

o Provisioning and configuring Azure Databricks workspaces

o Managing clusters, pools, and job configurations

o Integration with Azure services (ADLS, Snowflake, Monte Carlo, Solidatus, Atlan, Key Vault, Kafka, etc.)

· User Access & Security Management

o Managing role-based access control (RBAC) and permissions

o Integration with Azure Active Directory (AAD)

o Implementing security best practices, including data encryption and secrets management

· Cluster & Job Management

o Monitoring cluster performance, uptime, and cost optimization

o Troubleshooting failed jobs and workflows

o Scheduling and managing automated jobs

· Performance Optimization

o Tuning Spark jobs and clusters for optimal performance

o Managing autoscaling and resource utilization

o Identifying and resolving performance bottlenecks

· Monitoring & Incident Management

o Setting up monitoring, alerting, and logging (Azure Monitor, Log Analytics)

o Proactive issue detection and resolution

o Incident response and root cause analysis

· Cost Management & Optimization

o Tracking usage and optimizing costs across environments

o Providing recommendations for resource efficiency

· CI/CD & Release Management

o Supporting deployment pipelines for Databricks notebooks and code

o Version control integration (e.g., Git)

o Managing environment promotion (Dev, Test, Prod)

· Backup, Recovery & Compliance

o Ensuring backup strategies and disaster recovery readiness

o Supporting compliance requirements and audit activities

· Documentation & Support

o Maintaining operational runbooks and documentation

o Providing periodic reporting and status updates

o Offering L2/L3 support for platform-related issues