Senior Data Engineer (Databricks)

Required Qualifications

·       Data Engineering: Proven track record of architectural ownership and end-to-end system design.

·       Production Databricks Depth: Extensive experience with Delta Lake, Spark (PySpark & Spark SQL), Unity Catalog, job orchestration, and performance tuning (partitioning, clustering, Z-ordering, file compaction).

·       Financial Time-Series Domain Expertise: Strong understanding of structured finance instruments (ABS/MBS, loan-level data, cash flow analytics) and time-series correctness (bitemporal processing, point-in-time alignment, late-arriving data).

·       Legacy Code Discovery: Demonstrated ability to read unfamiliar, monolithic production codebases (e.g., legacy SQL/stored procedures, C++, Java, or custom scripts) to recover underlying business logic without relying on specs.

·       Demonstrated Impact: Applicants must be able to quantify the scale and business outcomes of their past initiatives (e.g., performance gains, cost reductions, data parity metrics, scale handled).

·       Languages & Tooling: Advanced Python, SQL, shell scripting, and CI/CD-driven infrastructure delivery (e.g., Git, Terraform).

Strongly Preferred Qualifications

·       Conference Presentation or Published Work: Prior speaking engagement at major industry events (e.g., Databricks Data + AI Summit, AWS re:Invent, Snowflake Summit, FINOS, or regional Data Engineering conferences) OR published technical blogs/whitepapers on complex data engineering or financial modeling.

·       Legacy Migration Cutover Experience: Proven history of taking a legacy platform completely off-line through rigorous parity testing and controlled cutover.

·       Governance & Cost Control: Hands-on execution with Delta Live Tables, Unity Catalog governance, and FinOps/cost management.