Senior Azure Databricks Engineer - Jersey City, NJ 1 of 2

job summary:

Seeking experienced Databricks / Azure Consultants to support the design, implementation, and ongoing development of the Databricks data platform. The consultants will work with technology and data teams to build and enhance scalable data solutions using Databricks and Microsoft Azure, with a strong focus on data engineering, integration, security, automation, and platform best practices.

location: Jersey City, New Jersey
job type: Contract to Perm
salary: $75 - 80 per hour
work hours: 8am to 5pm
education: Bachelors
responsibilities:

Ideal candidates will have strong hands-on experience across the Databricks ecosystem, including Databricks Workspace, Unity Catalog, Delta Lake, Spark, and MLflow, along with experience developing solutions in Microsoft Azure. Candidates should be comfortable working in a large enterprise environment and collaborating across application development, infrastructure, cloud, security, architecture, and data teams.

Key Skills and Experience

Databricks Ecosystem: Hands-on experience with Databricks Workspace, Unity Catalog, Delta Lake, Spark UI, and MLflow.

Programming & Data Engineering: Strong development experience with Python, PySpark, SQL, and/or Java, including the development and optimization of data processing and ETL/ELT pipelines.

Cloud Platforms: Strong Microsoft Azure experience, particularly Azure Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), and Azure Data Factory. Experience with AWS, including S3 and IAM, is also desirable.

Apache Spark: Experience developing, troubleshooting, and performance-tuning Spark-based workloads and data processing pipelines.

DevOps & MLOps: Experience with Git, CI/CD pipelines, OpenShift, Kubernetes, and Terraform / Infrastructure as Code (IaC).

Security & Governance: Experience implementing or supporting Databricks security, access controls, secrets management, identity integration, data governance, and Unity Catalog.

Automation & Operations: Experience with Databricks workflows, job orchestration, monitoring, troubleshooting, performance optimization, and production support.

Enterprise Delivery: Ability to collaborate across technical teams and operate within established enterprise architecture, security, change-management, and governance standards.

Knowledge of Apache Airflow

Preferred Experience

Experience designing and delivering production-grade Databricks solutions in a large enterprise environment, particularly where security, data governance, resiliency, automation, and operational support are key requirements.

qualifications:

- Databricks Ecosystem: Hands-on experience with Databricks Workspace, Unity Catalog, Delta Lake, Spark UI, and MLflow.

- Programming & Data Engineering: Strong development experience with Python, PySpark, SQL, and/or Java, including the development and optimization of data processing and ETL/ELT pipelines.

- Cloud Platforms: Strong Microsoft Azure experience, particularly Azure Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), and Azure Data Factory. Experience with AWS, including S3 and IAM, is also desirable.

- Apache Spark: Experience developing, troubleshooting, and performance-tuning Spark-based workloads and data processing pipelines.

- DevOps & MLOps: Experience with Git, CI/CD pipelines, OpenShift, Kubernetes, and Terraform / Infrastructure as Code (IaC).

skills:

Apache Airflow,S3,Experience with AWS,Spark,Apache Spark,Azure Data Factory,Azure Data Lake Storage,ADLS Gen2,cloud,Programming,CI/CD pipelines,data governance,Delta Lake,ELT pipelines,data processing pipelines,data processing,Databricks,Azure Databricks,DevOps,ETL,Git,IAM,data engineering,Infrastructure as Code (IaC),Java,secrets management,Kubernetes,MLflow,MLOps,Azure,Microsoft Azure,OpenShift,performance-tuning,performance optimization,PySpark,Python,SQL,application development,Terraform,troubleshooting,access controls,architecture,automation,change-management,enterprise architecture,Governance,infrastructure,operational support,production support,security,workflows

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.
At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact
Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).
This posting is open for thirty (30) days.