Software Defined Storage
Implementing S3-fronted cold storage at CERN
<p>The <a href="https://github.com/cern-cta/CTA">CERN Tape Archive (CTA)</a> is the open source solution developed at CERN to store more than 1 Exabyte of data from CERN’s experimental programmes. CTA interfaces with two disk systems widely used by the High-Energy Physics (HEP) community, <a href="https://github.com/cern-eos/eos">EOS</a> and <a href="https://github.com/dCache/dcache">dCache</a>. However, until now there has been no integration with systems used outside of HEP.</p>
<p>Looking at current industry standards, the leading interface for file and object storage is S3, which includes cold storage extensions for data archival. The CTA team are investigating whether CTA can be fronted by an S3 API. During this talk, we’ll review a proof-of-concept implementation, and look at alternative solutions to explore along with their respective trade-offs.</p>
Additional information
| Live Stream | https://live.fosdem.org/watch/ub4136 |
|---|---|
| Type | devroom |
| Language | English |
More sessions
| 1/31/26 |
<p><a href="https://github.com/opencloud-eu/">OpenCloud</a> has the design goal to not use a relational database. This requires a deeper integration with the underlying storage system, ie. through extensive use of extended file attributes. Since features like file revisions, trash and shares are inevitable nowadays, OpenCloud makes use of SDS native supported storage aspects to build these advanced features in an efficient way.</p> <p>In this talk we will give an overview of the storage aspects ...
|
| 1/31/26 |
<p>Ceph storage: Enterprise meets Community. Our traditional Ceph storage roadmap session starts with everything that is happening in the upstream project this year and what we have planned for the future, and closes with the state of what is backed by vendor-supported products. A 360-degree look at the state of Ceph integration with OpenStack and what is planned going forward in the broader storage space, in particular in regards to features relevant to container workloads.</p> <p>Architectural ...
|
| 1/31/26 |
<p>Garage (<a href="https://garagehq.deuxfleurs.fr/">project website</a>) is a versatile object storage software, focused on decentralized and geo-distributed deployments. The software has been developed under the AGPL for more than 5 years and is now reaching maturity.</p> <p>This talk will cover development and new features of the 2.x releases since the last FOSDEM talk (2024), best practices for administrators, available UIs, and a small tutorial on how to migrate from minio.</p>
|
| 1/31/26 |
<p>The CernVM File System (CVMFS) is a scalable, high-performance distributed filesystem developed at CERN to efficiently deliver software and static data across global computing infrastructures, primarily designed for high-energy physics (HEP). For the Large Hadron Collider (LHC) only, CVMFS is serving around 4 billion files (~2PB of data). CVMFS uses a content-addressable storage model, where files are stored in the form of cryptographic hashes, ensuring integrity and enabling deduplication. ...
|
| 1/31/26 |
<p>With cost and performance requirements becoming more and more relevant in today’s storage products, technologies that leverage algorithmic driven improvements are getting a lot of attention. Erasure coding is the most prominent algorithm and a meanwhile well established standard for saving on-disk space requirements in storage. It is built upon mathematical techniques. In my talk I want to explain and explore these techniques, and thereby the mathematical reasoning underlying these ...
|
| 1/31/26 |
<p>Have you ever found your CephFS setup mysteriously broken and had no clue how it got there? Maybe someone ran a CLI command in haste, or a misstep happened weeks ago. We have suspicions, but can’t really recall what might've splintered the system. That changes now.</p> <p>In this talk, we introduce a robust command history logging mechanism for CephFS: a persistent log of CephFS commands and standalone tool invocations, backed by LibCephSQLite. Think of it as “shell history,” but ...
|
| 1/31/26 |
<p>Starting with the Tentacle release, Ceph introduces mgmt-gateway: a modular, nginx-based service that provides a secure, highly available entry point to the entire management and monitoring stack. This talk will cover its architecture and deployment, how it centralizes access to the dashboard and observability tools, and how OIDC-based Single Sign-On streamlines authentication. We’ll also show how mgmt-gateway enhances security and access control while delivering full HA for Prometheus, ...
|
