Senior Azure Data Engineer - Elasticsearch (W2 Only)

Job Title: Azure Data/Search Engineer

Experience: Min 9+ Years

Location: Irving, TX (Hybrid – 3 days (Work from Office)

Contract Duration: Long Term

Job Description

Looking for a strong Azure Data Engineer / Senior Software Engineer with strong expertise in Python, Databricks, SQL, Elasticsearch, Kafka, and Azure, responsible for building scalable search platforms, REST APIs, event-driven data pipelines, and cloud-native distributed systems supporting high-volume enterprise applications.

Roles and Responsibilities: -

· Resource will be responsible to design, build, and support client, our enterprise search and intelligent matching platform.

· This role will develop scalable APIs, high-performance search capabilities, event-driven data pipelines, and cloud-native services supporting millions of transactions daily.

· The consultant will work across Azure, Elasticsearch, Kafka, Databricks, Python, and distributed systems to build highly available, low-latency applications

· The ideal candidate has strong backend engineering experience with cloud-native architectures, search technologies, REST APIs, event streaming, and large-scale data processing.

· Design and develop scalable backend services using Python. Build and maintain REST APIs supporting enterprise applications. Design and optimize Elasticsearch indexes, queries, mappings, and search relevance.

· Develop intelligent search and matching capabilities for large-scale datasets. Build event-driven architectures using Kafka and Azure Event Hub. Design high-throughput data ingestion and indexing pipelines. Integrate applications with Databricks and enterprise data platforms.

· Develop secure cloud-native applications on Microsoft Azure. Implement CI/CD pipelines and automated deployment processes.

· Monitor application health, logging, observability, and production performance. Optimize API response time, indexing performance, and infrastructure utilization. Participate in architecture, design reviews, code reviews, and production support.

· Collaborate with product managers, architects, data scientists, and engineering teams. 

Educational Qualifications: -

Engineering Degree – BE/ME/BTech/MTech/BSc/MSc.

Technical certification in multiple technologies is desirable.

Required Experience / Skills :

- Programming Python (required) SQL REST APIs JSON FastAPI (preferred) Async Python Cloud Microsoft Azure Azure App Services Azure Linux VMs Azure Container Apps / AKS (preferred) Azure Managed Identity, Azure Key Vault, Azure Storage, Azure Monitor, Azure Event Hub Search

- Technologies Elasticsearch Query DSL Relevance tuning Mapping design Aggregations Multi-search (msearch) Bulk indexing Search optimization Index lifecycle management Event Streaming Apache Kafka Confluent Cloud Producers / Consumers Kafka Connect Schema Registry

- Event-driven architecture Data Engineering Databricks PySpark Delta Lake Data ingestion pipelines ETL/ELT DevOps Git Azure DevOps Azure Pipelines CI/CD Docker Infrastructure as Code (Terraform or Pulumi preferred) - Databases PostgreSQL SQL Server Relational Databases Required

- Experience Min 9+ years of software engineering experience.

- Strong Python backend development experience.

- Experience designing and operating production REST APIs.

- Strong experience with Elasticsearch in production environments.

- Experience building scalable search platforms. Experience with Kafka or enterprise event-streaming platforms.

- Experience deploying applications on Microsoft Azure experience with CI/CD pipelines and automated deployments.

- Experience troubleshooting production systems and performance issues.

- Strong understanding of distributed systems and microservices.

Preferred Qualifications:

· Experience with AI-assisted search or intelligent matching platforms. Experience with Databricks and Delta Lake.

· Experience with machine learning integration. Experience with semantic search or vector search.

· Experience with Azure OpenAI or LLM integration. Healthcare or supply chain experience. Experience using GitHub Copilot or Codex.