HPC, Big Data & Data Science

Spack v1.0 and Beyond: Managing HPC Software Stacks

H.1308 (Rolin)
Harmen Stoppels
<h2>Abstract</h2> <p>Spack is a flexible multi-language package manager for HPC, Data Science, and AI, designed to support multiple versions, configurations, and compilers of software on the same system. Since the last FOSDEM, the Spack community has reached a major milestone with the release of Spack v1.0, followed closely by v1.1. This talk will provide a comprehensive overview of the "What's New" in these releases, highlighting the changes that improve robustness, performance, and user experience. We will cover among other things the shift to modeling compilers as dependencies, the package repository split, and the new jobserver-aware parallel installer.</p> <h2>Description</h2> <p>With the release of Spack v1.0 in July 2025 and v1.1 in November 2025, the project has introduced significant architectural changes and new features requested by the community. In this talk, we will dive into the key features introduced across these releases:</p> <ul> <li><strong>Compilers as dependencies.</strong> Spack has fulfilled an old promise from FOSDEM 2018. Compilers are modeled as first-class dependencies, dependency resolution is more accurate, and binary distribution and ABI compatibility checks are more robust.</li> <li>The <strong>separation of the package repository</strong> from the core tool and the introduction of a versioned Package API allows users to pin the package repository version independently from Spack itself and enables regular package repository releases.</li> <li><strong>Parallel builds with a new user interface.</strong> Spack has a new scheduler that coordinates parallel builds using the POSIX jobserver protocol, allowing efficient resource sharing across all build processes. The decades-old jobserver protocol is experiencing a major renaissance, adopted recently by Ninja v1.13 (July 2025) and the upcoming LLVM 22 release. We’ll talk about how this enables composable parallelism across make, ninja, cargo, GCC, LLVM, Spack, and other tools.</li> </ul> <h2>Expected Prior Knowledge / Intended Audience</h2> <p>This talk is aimed at Research Software Engineers (RSEs), HPC system administrators, and Data Scientists who use or manage software stacks. Familiarity with Spack is helpful but not strictly required; the talk will be accessible to anyone interested in package management and software reproducibility in scientific computing.</p> <h2>Links</h2> <ul> <li>Spack Website: https://spack.io</li> <li>Spack GitHub: https://github.com/spack/spack</li> <li>Spack Packages: https://github.com/spack/spack-packages</li> </ul>

Additional information

Live Stream https://live.fosdem.org/watch/h1308
Type devroom
Language English

More sessions

2/1/26
HPC, Big Data & Data Science
H.1308 (Rolin)
<p>Scientific models are today limited by compute resources, forcing approximations driven by feasibility rather than theory. They consequently miss important physical processes and decision-relevant regional details. Advances in AI-driven supercomputing — specialized tensor accelerators, AI compiler stacks, and novel distributed systems — offer unprecedented computational power. Yet, scientific applications such as ocean models, often written in Fortran, C++, or Julia and built for ...
2/1/26
HPC, Big Data & Data Science
Jan-Patrick Lehr
H.1308 (Rolin)
<p>ROCm™ has been AMD’s software foundation for both high-performance computing (HPC) and AI workloads and continues to support the distinct needs of each domain. As these domains increasingly converge, ROCm™ is evolving into a more modular and flexible platform. Soon, the distribution model shifts to a core SDK with domain-specific add-ons—such as HPC—allowing users to select only the components they need. This reduces unnecessary overhead while maintaining a cohesive and ...
2/1/26
HPC, Big Data & Data Science
Thomas Breuer
H.1308 (Rolin)
<p>Wherever research software is developed and used, it needs to be installed, tested in various ways, benchmarked, and set up within complex workflows. Typically, in order to perform such tasks, either individual solutions are implemented - imposing significant restrictions due to the lack of portability - or the necessary steps are performed manually by developers or users, a time-consuming process, highly susceptible to errors. Furthermore, particularly in the field of high-performance ...
2/1/26
HPC, Big Data & Data Science
Boris Martin
H.1308 (Rolin)
<h2>Content</h2> <p>High-frequency wave simulations in 3D (with e.g. Finite Elements) involve systems with hundreds of millions unknowns (up to 600M in our runs), prompting the use of massively parallel algorithms. In the harmonic regime, we favor Domain Decomposition Methods (DDMs) where local problems are solved in smaller regions (subdomains) and the full solution of the PDE is recovered iteratively. This requires each rank to own a portion of the mesh and to have a view on neighboring ...
2/1/26
HPC, Big Data & Data Science
Jade Abraham
H.1308 (Rolin)
<p>As the computing needs of the world have grown, the need for parallel systems has grown to match. However, the programming languages used to target those systems have not had the same growth. General parallel programming targeting distributed CPUs and GPUs is frequently locked behind low-level and unfriendly programming languages and frameworks. Programmers must choose between parallel performance with low-level programming or productivity with high-level languages.</p> <p><a ...
2/1/26
HPC, Big Data & Data Science
Mahendra Paipuri
H.1308 (Rolin)
<p>With the rapid acceleration of ML/AI research in the last couple of years, the already energy-hungry HPC platforms have become even more demanding. A major part of this energy consumption is due to users’ workloads and it is only by the participation of end users that it is possible to reduce the overall energy consumption of the platforms. However, most of the HPC platforms do not provide any sort of metrics related to energy consumption, nor the performance metrics out of the box, which ...
2/1/26
HPC, Big Data & Data Science
Tobias Kremer
H.1308 (Rolin)
<p><a href="www.ecmwf.int">ECMWF</a> manages petabytes of meteorological data critical for weather and climate research. But traditional storage formats pose challenges for machine learning, big-data analytics, and on-demand workflows. </p> <p>We propose a solution which introduces a Zarr store implementation for creating virtual views of ECMWF’s Fields Database (FDB), enabling users to access GRIB data as if it were a native Zarr dataset. Unlike existing approaches such as VirtualiZarr or ...