Senior Data Architect / AI Data Architect

Role: Senior Data Architect / AI Data Architect
Experience: 15+ years preferred, with strong hands-on experience in cloud data engineering and AI integration
Location: Remote
Employment Type: Contract

Mandatory Skills : Claude AI & Claude AI-Native Development

Job Summary

We are seeking an experienced Senior Data Architect with strong expertise in Microsoft Azure, Claude AI, Claude AI-native application development, Data bricks, and modern cloud data platforms. The ideal candidate will design and implement scalable, secure, and high-performance data architectures that integrate enterprise data engineering solutions with generative AI capabilities.

The candidate should have hands-on experience with Apache Spark, PySpark, Azure and AWS cloud services, and modern data architecture principles. Experience leveraging Anthropic's Claude AI models to build AI-native applications, intelligent data workflows, and enterprise-grade AI solutions is highly desirable.

Key Responsibilities
Design, develop, and implement scalable enterprise data architectures using Azure, AWS, and Data bricks.
Build and optimize large-scale data processing pipelines using Apache Spark and PySpark.
Design modern lakehouse and cloud data platforms supporting batch and real-time data processing.
Integrate Claude AI and Claude AI-native capabilities into enterprise applications, data platforms, and intelligent workflows.
Develop AI-powered solutions using Claude models, APIs, prompt engineering, structured outputs, tool use, and retrieval-augmented generation (RAG), where applicable.
Architect secure and scalable integrations between enterprise data sources, cloud platforms, and generative AI applications.
Implement data ingestion, transformation, orchestration, and data quality frameworks.
Optimize Spark workloads, Databricks jobs, cluster configurations, and cloud resource utilization.
Establish data governance, access controls, security, lineage, and compliance standards.
Collaborate with data engineers, AI engineers, cloud architects, and business stakeholders to translate requirements into technical solutions.
Follow best practices for monitoring, testing, CI/CD, performance optimization, and production support.
Ensure AI solutions follow enterprise security, privacy, responsible AI, and applicable healthcare data protection requirements.
Required Technical Skills

1. Azure Cloud

Strong experience with Microsoft Azure data services and cloud architecture.
Azure Data Factory, Azure Data bricks, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
Experience with cloud security, identity management, networking, and data integration.

2. Claude AI & Claude AI-Native Development

Hands-on experience integrating Anthropic Claude AI models through APIs or supported enterprise platforms.
Experience building Claude AI-native applications, agents, and intelligent workflows.
Strong understanding of prompt engineering, context management, tool calling, and structured outputs.
Experience with RAG, embeddings, vector databases, and enterprise knowledge retrieval is preferred.
Ability to integrate Claude AI with enterprise data platforms and business applications.
Understanding of AI security, guardrails, evaluation, observability, and responsible AI practices.

3. Data bricks & Data Architecture

Strong experience with Data bricks architecture and lakehouse solutions.
Knowledge of Delta Lake, data modeling, ETL/ELT, and medallion architecture.
Experience designing scalable data ingestion, transformation, and serving layers.
Understanding of data governance, access controls, lineage, and performance optimization.

4. Apache Spark & PySpark

Strong hands-on experience with Apache Spark and PySpark.
Expertise in distributed data processing, Spark SQL, DataFrames, and performance tuning.
Experience developing and maintaining complex data pipelines.
Knowledge of batch processing and streaming data architectures.

5. AWS Cloud

Experience with AWS cloud architecture and data services.
Familiarity with Amazon S3, AWS Glue, Amazon EMR, and related data processing services.
Experience integrating cloud data services with enterprise analytics and AI platforms.
Preferred Qualifications

Experience working with enterprise-scale or healthcare-related data platforms.
Knowledge of HIPAA-related data privacy and security requirements.
Experience integrating LLMs with enterprise data pipelines and analytics platforms.
Familiarity with CI/CD, Git, Terraform, and infrastructure as code.
Experience with API development, microservices, containerization, and production AI deployments.
Strong communication, analytical, problem-solving, and stakeholder collaboration skills.