Sr. Databricks Engineer

Required Qualifications Bachelor's or Master's degree in Computer Engineering, Computer Science, Information Systems, or a related field. 8+ years of experience in software engineering, data engineering, or big data platform development. Strong experience designing and implementing enterprise-scale data pipelines. Hands-on expertise with: Python PySpark Spark SQL SQL Kafka Databricks Delta Lake Snowflake Hive Experience building data solutions on AWS cloud platforms. Strong understanding of distributed computing, data modeling, and large-scale data processing. Experience with Git-based development workflows and CI/CD practices. Excellent analytical, troubleshooting, and problem-solving skills.

Preferred Qualifications Experience with real-time streaming architectures and event-driven systems. Knowledge of data governance, metadata management, and data quality frameworks. Experience with generative AI technologies including: LLMs RAG Vector Databases AI Agents MCP integrations Experience developing developer productivity tools and AI-assisted engineering workflows. Exposure to enterprise supply chain, retail, healthcare, or manufacturing data domains. AWS certifications are highly preferred. Technical Skills Programming Languages Python Java SQL Shell Scripting C/C++ Big Data & Data Engineering PySpark Spark SQL Hive Databricks Delta Lake Snowflake Kafka HBase Sqoop Workflow & Orchestration Apache Airflow Databricks Workflows Oozie Cloud Technologies AWS S3 EMR EC2 Athena Redshift RDS IAM Lambda SNS SQS AI & Modern Engineering Generative AI Large Language Models (LLMs) Retrieval Augmented Generation (RAG) Agentic AI Systems Model Context Protocol (MCP) Vector Databases Visualization & Tools Tableau Git Docker Splunk IntelliJ IDEA PyCharm Cursor

Preferred Certifications AWS Certified Solutions Architect – Associate AWS Certified Cloud Practitioner Databricks Certifications (preferred) What Success Looks Like Deliver highly scalable and reliable data pipelines. Improve platform performance, efficiency, and cost optimization. Enable enterprise-wide analytics and AI initiatives through trusted data products. Drive modernization of data platforms and adoption of cloud-native architectures. Leverage AI technologies to enhance engineering efficiency, automation, and innovation.

Ideal Candidate Profile: A senior-level data engineer with extensive experience in Databricks, Spark, AWS, Kafka, Snowflake, and Lakehouse architectures, who is equally passionate about modern AI technologies and building intelligent data platforms for the future.