Quick overview
Job description
Must Have Skills:
· Databricks - Spark (PySpark or Spark SQL, Delta Lake (medallion architecture), Unity Catalog (Multi-tenant)
· Python & SQL
· Experience owning multi-tenant data architecture
· Experience in Higher Education technology (CRM/SIS/Degree Audit platforms, etc.)
· Working knowledge of FERPA, GLBA, SOC 2
· Data Security and Governance
· GitHub, working with a CI/CD
· Architecture Reviews, decisions
· Evaluating emerging tools, models and frameworks
Nice to Have Skills
· Exposure to agentic AI systems or LLM adjacent data pipelines.
· Experience with independently standing up net-new Unity Catalog structures for new tenants from scratch.
Job Responsibilities
· Design, build and maintain Databricks pipelines across bronze/silver/gold layers for multiple higher education clients unifying data across the institution, each within its own Unity Catalog.
· Create and implement strategies for moving client data from their CRM/SIS platforms via API calls or into drop zones (SFTP) for ingestion into Databricks. Examples include Slate SFTP feeds, Banner derived warehouses, EAB Navigate 360 and Stellic, all core higher education platforms, with attention to schema drift and data quality.
· Own the thin, tenant scoped physical ingest versus a Lakehouse Federation decision for new client onboarding.
· Implement Data governance controls including Unity Catalog lineage and audit logging. Enforce PII de-identification gates and FERPA compliance across patterns.
· Be the resident expert on changing compliance requirements and plan as needed while keeping all the stakeholders informed.
· Partner with the AI Solutions team to translate agent data requirement into concrete schemas and feature tables.
· Support the onboarding of new institutional clients, each with their distinct CRM/SIS stacks and data dictionaries.
· Contribute and suggest reusability for components, especially with connectors so that patterns build for one client can be reused across the portfolio.
· Work with engineering as the primary engineering/infrastructure partner on platform level Databricks and AWS decisions
Required Experience:
· At least 3 years of building production data pipelines, with hands-on Databricks experience (Spark, Delta Lake, Unity Catalog, etc.).
· Strong SQL and Python skills with familiarity working across batch ETL/ELT patterns.
· Experience with multi-tenant data architecture and a deep understanding of catalog isolation reasoning.
· Working knowledge of FERPA, GLBA, SOC 2 or equivalent regulated-data experience (higher education, healthcare, financial services).
· Experience with AWS (our deployment model is single-tenant private AWS per client).
· Comfortable working with a geographically dispersed team across the world.
Educational Qualifications:
· Bachelor's/Master's in Computer Science, AI, Engineering or related field.
Similar jobs
- IS
Sr Microsoft Fabric Data EngineerRecently posted
IFLOWSOFT Solutions Inc. · United States · Hybrid · Posted 16 Nov 2026 🇺🇸
SQLT-SQLGit+10TechnologySalary not listed - ME

Data EngineerRecently posted
MetaSense, Inc. · Dallas, TX · On-site · Posted 16 Nov 2026 🇺🇸
MicroservicesApache SparkDatabricks+3TechnologySalary not listed - IQ
Cybersecurity data engineer with Security ClearanceRecently posted
iQuasar LLC · Adelphi, MD · On-site · Posted 16 Nov 2026 🇺🇸
GCPOracleRuby+9TechnologySalary not listed - RS
Data Engineer MidRecently posted
ResolveTech Solutions Inc. · Irving, TX · Hybrid · Posted 16 Nov 2026 🇺🇸
SQLGitApache Spark+7TechnologySalary not listed - CS

Lead Snowflake and Hadoop Data EngineerRecently posted
Cynet Systems · Alpharetta, GA · On-site · Posted 16 Nov 2026 🇺🇸
SQLApache SparkSOC 2+7Technology$50 - $55/hr - IS

Data EngineerRecently posted
IRIS Software, Inc. · Parsippany-Troy Hills, NJ · Hybrid · Posted 16 Nov 2026 🇺🇸
SQLGitApache Spark+12TechnologySalary not listed - NA

Senior Bioinformatics & Genomics Data System EngineerRecently posted Auto Apply
Natera · San Carlos · Hybrid · Posted 16 Nov 2026 🇺🇸
MySQLGitAWS+6Technology$125.8k/yr - NA

Senior Bioinformatics & Genomics Data System EngineerRecently posted Auto Apply
Natera · US Remote · Remote · Posted 16 Nov 2026 🇺🇸
MySQLGitAWS+6Technology$113.6k - $142k/yr - CI

Senior Data EngineerRecently posted
ComTec Information Systems · New Orleans, LA · On-site · Posted 16 Nov 2026 🇺🇸
SQLMicrosoft ExcelMicrosoft Office+7TechnologySalary not listed - CT

Java Developer with data engineerRecently posted
CIS Technologies Inc. · Plano, TX · Hybrid · Posted 16 Nov 2026 🇺🇸
DynamoDBSQLAWS+8TechnologySalary not listed - PB
ETL EngineerRecently posted
Pebal Bright Technologies LLC · Chicago, IL · On-site · Posted 16 Nov 2026 🇺🇸
SQLApache SparkETL+5TechnologySalary not listed - LI

Data & AI Platform, Intern - Summer 2027Recently posted Auto Apply
Lightmatter · Boston · Hybrid · Posted 16 Nov 2026 🇺🇸
SQLGitSnowflake+2$50 - $55/hr
