Quick Overview
Job Description
7+ MONTH
This is a temp to perm role
Skills (EXPERT/ADVANCED/NONE)
Databricks on AWS Platform Engineer Platform Architecture
Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.
Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
Questions (must answer YES to ALL)
Do you have Databricks on AWS Platform Engineer Platform Architecture
Are you local to one of the locations listed?
Do you have certification on DataBricks Platform Administrator /Databricks Certified Data Engineer/
Do you have certification on AWS Certification (DevOps/Solution Architect)
Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.
Do you have Databricks, AWS S3,
Do you have Delta Lake
Do you have Unity Catalog, Spark/PySpark, Terraform, GitHub, CI/CD, CloudWatch, Mosaic AI, Genie, Vector Search, MLflow, Kafka, Airflow, Kubernetes.
Description:
UPDATED NOTES ON REQ FROM 8/21
Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.
Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.
Core Technologies: Databricks, AWS S3, Delta Lake, Unity Catalog, Spark/PySpark, Terraform, GitHub, CI/CD, CloudWatch, Mosaic AI, Genie, Vector Search, MLflow, Kafka, Airflow, Kubernetes.
Certification :
DataBricks Platform Administrator /Databricks Certified Data Engineer/
AWS Certification (DevOps/Solution Architect)
Terraform
Similar jobs
- UG
Per Diem Registration Representative
UnitedHealth Group
PORTLAND, ME🇺🇸$16 - $29/hrOn-site1 week ago401kMedical TerminologyMicrosoft Office+1 - RT
Partner BDR Opportunity | AI-Driven Partner Programs
NewRussell, Tobin & Associates
San Francisco, CA🇺🇸$100 - $110/hrHybridYesterdaySalesforceBusiness DevelopmentCRM+5 - OR
Oracle EBS Technical Lead
NewOraapps Inc
United States🇺🇸RemoteYesterdayOracleAccounts PayableAccounts Receivable+4 - KG
Loan IQ (Fulltime - New York City, NY)
NewK&K Global Talent Solutions
New York, NY🇺🇸On-siteYesterdaySQLBusiness AnalysisRequirements Gathering+2 - PT
ETL Tester - Claims
NewPeterson Technology Partners
Rosemont, IL🇺🇸$65 - $75/hrHybridYesterdaySQLETLScrum+7 - PG
Azure Engineer, Hybrid - 69800
NewPRIMUS Global Services Inc.
United States🇺🇸HybridYesterdayLoad BalancingActive DirectoryAnsible+4