Quick overview
Job description
Technologies We Use
- Java, Scala, Rust, and Python
- Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution
- Apache Iceberg for table management and catalog interoperability
- Apache Arrow and Apache Parquet for in-memory data processing and columnar storage
- Industry-standard build tooling, including Gradle, Cargo, and GitHub
Core Responsiblities
- Designing and implementing query planning and optimization capabilities that turn complex computations into efficient execution plans
- Extending execution engines with new capabilities and improving query operators, parallelism, memory management, and data movement
- Developing Foundry’s Iceberg catalog and engine integrations, including table metadata, transactions, and efficient reads and writes
- Building shared transformation semantics and execution interfaces for workloads across Foundry’s products and platform services
- Improving incremental processing so pipelines can reuse previous results and process new data efficiently while preserving correctness
- Evaluating and integrating advances in open-source data systems, validating their behavior and performance against real-world workloads
- Investigating correctness and performance issues across planning, execution, and storage, and building tests and benchmarks that prevent regressions
- Working with product teams and customers to translate operational needs into engine capabilities that integrate with Foundry’s security, data management, and build infrastructure
What We Value
- Ownership mindset and a high bar for correctness. Our systems support decisions and operations that customers depend on.
- Curiosity about how data systems work, from query optimizers and execution operators to table formats and distributed processing.
- Strong debugging skills and motivation to follow a problem across languages, services, and layers of the stack.
- A practical approach to performance, grounded in profiling, representative workloads, and measurable improvements.
- Interest in applying deep systems engineering to real-world problems, with empathy for the people who use and depend on our software.
- Experience building or extending systems such as Spark, DataFusion, Iceberg, or comparable technologies, and an interest in learning across the stack.
- Ability to collaborate across teams and work effectively with the open-source projects we build on. Experience contributing to open-source projects is valued, but not required.
What We Require
- 4+ years of professional software engineering experience developing database engines, distributed data processing systems, storage systems, or comparable infrastructure, with depth in areas such as query planning, execution, or performance optimization. Operating on top of these technologies alone does not qualify.
- Experience diagnosing correctness and performance problems in complex systems.
- Strong written and verbal communication skills and the ability to work effectively across teams, incorporate feedback, and hold a high bar for quality.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- KA
Staff Engineer - Release Engineering Auto Apply
Kaluza · Edinburgh · Hybrid 🇬🇧
Talent AcquisitionGitAWS+11TechnologySalary not listed - SB
Software EngineerRecently posted Auto Apply
Standard Bank Group · Dunfermline · Hybrid · Posted 17 Nov 2026 🇬🇧
GitHTMLCSS+9TechnologySalary not listed - TA
Software EngineerRecently posted Auto Apply
The Adaptavist Group · London · Hybrid · Posted 17 Nov 2026 🇬🇧
MicroservicesPHPAWS+12TechnologySalary not listed - TT
Lead Embedded Software EngineerRecently posted Auto Apply
TTP · Cambridge · On-site · Posted 17 Nov 2026 🇬🇧
Embedded SystemsTechnologySalary not listed - WI
Senior Software Engineer II - Wise for BanksRecently posted Auto Apply
Wise · London · Hybrid · Posted 17 Nov 2026 🇬🇧
JavaSpring BootSpring+2Technology£111k - £145k/yr - GD
Senior Systems EngineerRecently posted Auto Apply
General Dynamics Missions System International · Hastings · Remote · Posted 16 Nov 2026 🇬🇧
Microsoft ExcelTechnology£45.6k - £60k/yr - GD
Principal Systems EngineerRecently posted Auto Apply
General Dynamics Missions System International · Hastings · Remote · Posted 16 Nov 2026 🇬🇧
Technology£59.2k - £80k/yr - MG
Senior Python EngineerRecently posted Auto Apply
Man Group · London · On-site · Posted 17 Nov 2026 🇬🇧
MongoDBGitJava+4TechnologySalary not listed - CO
Software EngineerRecently posted Auto Apply
Cogna · London · Hybrid · Posted 17 Nov 2026 🇬🇧
RustGitGPT+7TechnologySalary not listed - BJ
Lead Engineer - Stock Trading App Auto Apply
Bjakcareer · United Kingdom · Hybrid · Posted 29 Oct 2026 🇬🇧
System DesignEngineeringSalary not listed - BJ
Lead Engineer - Travel FX App Auto Apply
Bjakcareer · United Kingdom · Hybrid · Posted 29 Oct 2026 🇬🇧
System DesignEngineeringSalary not listed - PE

Member of Technical Staff (Index Engineer, Search Crawler)Recently posted Auto Apply
Perplexity · London · Hybrid · Posted 17 Nov 2026 🇬🇧
RustC++EngineeringSalary not listed