Azure Architect

Job Title: Senior Azure Databricks Architect

Location: Dallas, TX

Job Type: 3 to 6 months CTH role

Work: Onsite role 5 days a week

Interview: Three rounds - Video interview & In person

Job Description:

We are hiring a hands-on Technical Architect to lead enterprise-scale migrations to the Databricks Lakehouse for our US customers. You will lead the architecture and technical execution of a Palantir Foundry to Databricks migration. The candidate will be responsible for assessing the existing Palantir application, defining the target Databricks architecture, mapping existing capabilities to Databricks services, and providing technical leadership throughout the migration.

The solution involves migration of capabilities including data ingestion and transformation, AI-based information extraction from structured and unstructured documents, large-scale data comparison, dashboards and drill-down reporting, chatbot capabilities, and business workflows/applications.

The ideal candidate should have strong expertise in Databricks, Lakehouse architecture, data engineering, AI/ML workloads, and cloud-native data platforms, along with the ability to understand and decompose existing Palantir Foundry capabilities into equivalent or improved Databricks-based solutions. You will own the end-to-end technical solution: architecture, design, deployment, performance and every major technology decision. You will work directly with customer business and IT stakeholders, turn business requirements into clear technical requirements, and guide a delivery team of engineers. This is a builder role: you are expected to design, write and review code, not only draw diagrams

Roles & Responsibilities:

• Architecture ownership
• Design and technology decisions
• Deployment
• Performance and cost
• Business-to-technical translation
• AI / GenAI Architecture
• Technical leadership and mentoring

Qualifications & Requirements:

12+ years in data engineering, data warehousing or data architecture, with 5+ years hands-on on Databricks.

Bachelor's or Master's degree in Computer Science, Engineering, Information Technology or related discipline.

5+ years of experience working with modern cloud data platforms.

Proven experience designing enterprise-scale data platforms.

Experience leading technical teams and architecture initiatives.

Strong communication and stakeholder-management skills.

Ability to translate business requirements into scalable technical architectures.

Has led at least one large-scale enterprise migration to Databricks end to end (multi-TB to PB data, hundreds of pipelines, phased cutover).

Deep hands-on skill in Apache Spark (PySpark, Spark SQL), SQL and Python.

Strong knowledge of Delta Lake, Unity Catalog, Databricks Workflows, Delta Live Tables and Databricks SQL.

Proven experience migrating from legacy platforms such as Teradata, Oracle, Netezza, SQL Server, Hadoop/Cloudera or Informatica.

Track record of owning architecture and technology decisions on enterprise programs.

Experience turning business requirements into technical designs and working directly with US customer stakeholders.

Experience mentoring and technically leading engineering teams, including onsite offshore models.

Excellent communication and presentation skills.

Technology Skills

Area	Skills expected
Databricks platform	Lakehouse architecture, Delta Lake, Unity Catalog, Databricks SQL and SQL Warehouses, Workflows/Jobs, Delta Live Tables, Photon, Serverless compute
Migration sources	Teradata, Oracle, Netezza, SQL Server, Hadoop/Cloudera (Hive, Impala, Spark), Informatica, DataStage, SSIS, SAS
Migration tooling	Lakebridge / BladeBridge or similar code-conversion tools, automated data validation and reconciliation frameworks
Data engineering	PySpark, Spark SQL, Python, Structured Streaming, Auto Loader, Kafka / Event Hubs / Kinesis, medallion architecture, dbt (plus)
Data modelling	Dimensional modelling, Data Vault, Lakehouse modelling patterns, SCD handling
Performance	Spark UI analysis, AQE, partitioning, Liquid Clustering / Z-order, OPTIMIZE / VACUUM, cluster and warehouse sizing
Governance and security	Unity Catalog permissions, row/column-level security, lineage, data quality (DLT expectations, Great Expectations), PII handling
AI/ML on Databricks	MLflow, Feature Store, Model Serving, Mosaic AI, Vector Search, GenAI use cases
Cloud	Databricks on AWS, Azure or Google Cloud Platform: object storage, networking (VNet/VPC, Private Link), IAM
DevOps and FinOps	Databricks Asset Bundles, Git, Azure DevOps / GitHub Actions / Jenkins, Terraform, DBU cost monitoring