Databricks Data Architect

Databricks Data Architect
New Jersey (Hybrid)

1. Data Engineering Execution
1. Build and maintain ingestion frameworks (ADF / Databricks / Spark)

1. Implement Bronze -> Silver transformations aligned to architecture

1. Architect data quality checks, schema validation, and contract rules

1. Optimize pipelines for performance and cost

2. DevOps & Automation
1. Own CI/CD pipelines (Azure DevOps)

1. Automate onboarding (repos, pipelines, policies, templates)

1. Maintain IaC and scripting (Terraform, PowerShell / Python)

1. Troubleshoot pipeline failures and permission issues

2. Data Modeling - Design and implement curated data models in Gold layer aligned to business use cases
1. Build:
1. Dimensional models (star/snowflake schemas)

1. Fact tables, dimensions, surrogate keys, SCD handling

2. Translate Silver datasets into:
1. Analytics-ready models
2. Consistent KPI definitions and business logic
3. Ensure:
1. Consistency across domains (common dimensions, conformed models)
2. Reusability and scalability of models
3. Alignment with enterprise semantic layer (Power BI / Fabric / downstream tools)
4. Handle changes such as:
1. Column derivation logic changes (not just schema changes)
2. Backward compatibility and impact analysis
5. Partner with business / data stewards to:
1. Define business rules
2. Validate metrics and edge cases
3. Support regression scenarios

3. Governance & Data Quality Implementation
1. Implement governance patterns defined by architecture
2. Enforce:
1. Unity Catalog standards
2. naming conventions, metadata policies
3. access controls (RBAC/ABAC)

4. Architecture Translation
1. Automation first thinking and design approach.
2. Convert high-level architecture into:
1. low-level designs
2. reusable templates
3. engineering playbooks
3. Ensure consistency across projects and teams    

5. Team Enablement
1. Support delivery teams with:
1. onboarding

1. debugging

1. design reviews

2. Provide hands-on guidance
3. Create reusable assets to reduce repeat questions

Required Skills

1. Core Platform
1. Azure Data Platform (ADF, ADLS, Azure SQL)

1. Databricks (DLT, Delta Lake, Spark, Workflows, Unity Catalog)

1. Microsoft Fabric (preferred)

2. Engineering
1. PySpark / Python (advanced)
2. SQL (strong)
3. Pipeline design (batch + streaming)

3. DevOps / Automation
1. Azure DevOps (pipelines, repos, policies)
2. PowerShell automation (must-have for your setup)
3. CI/CD + Infrastructure-as-Code concepts

4. Governance / Data Quality
1. Schema enforcement, data contracts, DQ frameworks
2. Metadata-driven design understanding

 
Thank you for your time!
 

CONFIDENTIALITY NOTICE: This e-mail message including attachments, if any, is intended only for the person or entity to which it is addressed and may contain confidential and /or privileged material. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. If you are the intended recipient but do not wish to receive communications through this medium, please so advise the sender immediately.  
When printed, this document is considered uncontrolled.