Back to Jobs
Other

Azure Databricks Architect

SUNRAY INFORMATICSDallas, TX🇺🇸United States

Why this role stands out

This Senior Azure Databricks Engineer role offers significant opportunities to advance your skills in a cutting-edge data and AI environment with a reputable company. You'll thrive here if you're a seasoned professional eager to optimize complex Databricks workloads and contribute to impactful data solutions. Apply now to join a dynamic team and drive innovation in cloud-based data engineering.

Quick overview

Work modeOn-site
LocationDallas, TX, United States
OracleSQLGitApache SparkAWSMLflowApacheDatabricksGitHub ActionsGoogle CloudHadoopHiveJenkinsKafkaLLMPythonReconciliationTerraformUnityVaultdbtGenerative AIAzureSQL ServerAzure DevOpsDelta LakeRAGIAM

Job description

Hi Professionals,

We have an immediate openings for the below role, If interested please apply.

Job Title: Azure Databricks Architect

Interview: Telephonic & Face to Face & In person Interview

JOB RESPONSIBILITIES

Type : C2C and Onsite

OVERALL EXPERIENCE -12 YRS PLUS

ROLE: SENIOR AZURE DATABRICKS ARCHITECT

OVERALL EXPEREINCE 10YRS+

Job Description:

ROLE: SENIOR AZURE DATABRICK ARCHITECT

OVERALL EXPEREINCE 12YRS+

Job Description:

We are hiring a hands-on Technical Architect to lead enterprise-scale migrations to the Databricks Lakehouse for our US customers. You will lead the architecture and technical execution of a Palantir Foundry to Databricks migration. The candidate will be responsible for assessing the existing Palantir application, defining the target Databricks architecture, mapping existing capabilities to Databricks services, and providing technical leadership throughout the migration.

AI / GenAI Architecture

  • Architect solutions for extracting information from structured and unstructured documents.
  • Design RAG and LLM-based solutions where applicable.
  • Evaluate appropriate Databricks capabilities for:
    • Document processing
    • Information extraction
    • Embeddings and vector search
    • LLM inference
    • Model serving
    • AI agents / conversational applications

Qualifications & Requirements:

  • 12+ years in data engineering, data warehousing or data architecture, with 5+ years hands-on on Databricks.
  • Bachelor's or Master's degree in Computer Science, Engineering, Information Technology or related discipline.
  • 5+ years of experience working with modern cloud data platforms.
  • Proven experience designing enterprise-scale data platforms.
  • Experience leading technical teams and architecture initiatives.
  • Strong communication and stakeholder-management skills.
  • Ability to translate business requirements into scalable technical architectures.
  • Has led at least one large-scale enterprise migration to Databricks end to end (multi-TB to PB data, hundreds of pipelines, phased cutover).
  • Deep hands-on skill in Apache Spark (PySpark, Spark SQL), SQL and Python.
  • Strong knowledge of Delta Lake, Unity Catalog, Databricks Workflows, Delta Live Tables and Databricks SQL.
  • Proven experience migrating from legacy platforms such as Teradata, Oracle, Netezza, SQL Server, Hadoop/Cloudera or Informatica.
  • Track record of owning architecture and technology decisions on enterprise programs.
  • Experience turning business requirements into technical designs and working directly with US customer stakeholders.
  • Experience mentoring and technically leading engineering teams, including onsite–offshore models.
  • Excellent communication and presentation skills.

Technology Skills

Area

Skills expected

Databricks platform

Lakehouse architecture, Delta Lake, Unity Catalog, Databricks SQL and SQL Warehouses, Workflows/Jobs, Delta Live Tables, Photon, Serverless compute

Migration sources

Teradata, Oracle, Netezza, SQL Server, Hadoop/Cloudera (Hive, Impala, Spark), Informatica, DataStage, SSIS, SAS

Migration tooling

Lakebridge / BladeBridge or similar code-conversion tools, automated data validation and reconciliation frameworks

Data engineering

PySpark, Spark SQL, Python, Structured Streaming, Auto Loader, Kafka / Event Hubs / Kinesis, medallion architecture, dbt (plus)

Data modelling

Dimensional modelling, Data Vault, Lakehouse modelling patterns, SCD handling

Performance

Spark UI analysis, AQE, partitioning, Liquid Clustering / Z-order, OPTIMIZE / VACUUM, cluster and warehouse sizing

Governance and security

Unity Catalog permissions, row/column-level security, lineage, data quality (DLT expectations, Great Expectations), PII handling

AI/ML on Databricks

MLflow, Feature Store, Model Serving, Mosaic AI, Vector Search, GenAI use cases

Cloud

Databricks on AWS, Azure or Google Cloud Platform: object storage, networking (VNet/VPC, Private Link), IAM

DevOps and FinOps

Databricks Asset Bundles, Git, Azure DevOps / GitHub Actions / Jenkins, Terraform, DBU cost monitoring

Best Wishes,

Dany|Recruiter|Sunray Informatics Inc

(Direct) | (E) Desk: EXT 253

Sunray Informatics Inc | 77 Milltown Road Suite C1, East Brunswick, NJ 08816 Web:

Will your resume get past this employer's screening? Check your resume against this job, free

Similar jobs