JUBE: An Environment for systematic benchmarking and scientific workflows

<p>Wherever research software is developed and used, it needs to be installed, tested in various ways, benchmarked, and set up within complex workflows. Typically, in order to perform such tasks, either individual solutions are implemented - imposing significant restrictions due to the lack of portability - or the necessary steps are performed manually by developers or users, a time-consuming process, highly susceptible to errors. Furthermore, particularly in the field of high-performance computing (HPC), where large amounts of data are processed and the computer systems used are unique worldwide, not only performance, scalability, and efficiency of the applications are important, but so are modern research software engineering (RSE) principles such as reproducibility, reusability, and documentation.</p> <p>With these challenges and requirements in mind, JUBE [1] (Jülich Benchmarking Environment) has been developed at the Jülich Supercomputing Centre (JSC), enabling automated and transparent scientific workflows. JUBE is a generic, lightweight, configurable environment to run, monitor and analyze application execution in a systematic way. It is a free, open-source software implemented in Python that operates on a "definition-based" paradigm where the “experiment” is described declaratively in a configuration file (XML or YAML). The JUBE engine is responsible for translating this definition into shell scripts, job submission files, and directory structures. Due to its standardized configuration format, it simplifies collaboration and usability of research software. JUBE also complements the Continuous Integration and Continuous Delivery (CI/CD) capabilities, leading to Continuous Benchmarking.</p> <p>To introduce and facilitate JUBE’s usage, the documentation includes a tutorial with simple and advanced examples, an FAQ page, a description of the command line interface, and a glossary with all accepted keywords [2]. In addition, a dedicated Carpentries course offers an introduction to the JUBE framework [3] (basic knowledge of the Linux shell and either XML or YAML are beneficial when getting started with JUBE). A large variety of scientific codes and standard HPC benchmarks have already been automated using JUBE and are also available open-source [4].</p> <p>In this presentation, an overview of JUBE will be provided, including its fundamental concepts, current status, and roadmap of future developments (external code contributions are welcome). Additionally, three illustrative use cases will be introduced to offer a comprehensive understanding of JUBE's practical applications: - benchmarking as part of the procurement of JUPITER, Europe’s first exascale supercomputer; - a complex scientific workflow for energy system modelling [5]; - continuous insight into HPC system health by regular execution of applications, and the subsequent graphical presentation of their results.</p> <p>JUBE is a well-established software, which has already been used in several national and international projects and on numerous and diverse HPC systems [6-13]. Besides being available via EasyBuild [14] and Spack [15], further software has been built up based on JUBE [16,17]. Owing to its broad scope and range of applications, JUBE is likely to be of interest to audiences in the HPC sector, as well as those involved in big data and data science.</p> <p>[1] https://github.com/FZJ-JSC/JUBE [2] https://apps.fz-juelich.de/jsc/jube/docu/index.html [3] https://carpentries-incubator.github.io/hpc-workflows-jube/ [4] https://github.com/FZJ-JSC/jubench [5] https://elib.dlr.de/196232/1/2023-09_UNSEEN-Compendium.pdf [6] MAX CoE: https://max-centre.eu/impact-outcomes/key-achievements/benchmarking-and-profiling/ [7] RICS2: https://risc2-project.eu/?p=2251 [8] EoCoE: https://www.eocoe.eu/technical-challenges/programming-models/ [9] DEEP: https://deep-projects.eu/modular-supercomputing/software/benchmarking-and-tools/ [10] DEEP-EST: https://cordis.europa.eu/project/id/754304/reporting [11] IO-SEA: https://cordis.europa.eu/project/id/955811/results [12] EPICURE: https://epicure-hpc.eu/wp-content/uploads/2025/07/EPICURE-BEST-PRACTICE-GUIDE-Power-measurements-in-EuroHPC-machines_v1.0.pdf [13] UNSEEN: https://juser.fz-juelich.de/record/1007796/files/UNSEEN_ISC_2023_Poster.pdf [14] EasyBuild: https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/j/JUBE [15] Spack: https://packages.spack.io/package.html?name=jube [16] https://github.com/edf-hpc/unclebench [17] https://dl.acm.org/doi/10.1145/3733723.3733740</p>