Software Defined Storage
The history of error correction and detection and how it led to Ceph’s Erasure Coding Techniques
70 years of academic innovation in the development of error correction codes have led to the advanced erasure coding techniques that we use in Ceph. Learn more about how these came about, the different types, how they work, and how we use them in distributed storage today.
Erasure Coding is the latest in a long line of error detection and correction approaches over the last 70 years which have all had an impact on the way we approach storing and recovering data in sensible and efficient ways. I’ll give an overview of the main approaches over the years, including the parity bit, the hamming codes, RAID, reed-solomon, and how they have impacted media storage, distributed storage, and their usage in other unexpected ways. I’ll then provide an overview of erasure coding across distributed storage and specifically Ceph.
Additional information
| Type | devroom |
|---|
More sessions
| 2/2/20 |
Ceph is an open source, highly scalable, software defined storage that provides object, file and block interfaces under a unified system. Ceph Object Storage Gateway (RGW) provides a RESTful object storage interface to the Ceph Storage cluster. It provides an interface that is compatible with a large subset of AWS S3 APIs. In this talk we discuss the implementation of a subset of the APIs of AWS Secure Token Service (STS). AWS STS is a web service which enables identity federation and ...
|
| 2/2/20 |
NFS-Ganesha is an extensible user-space NFS server that supports NFS v3, v4, v4.1, v4.2, pNFS, and 9P protocol. It has an easily pluggable architecture called FSAL (File System Abstraction Layer), which enables seamless integration with many filesystem backends (GlusterFS, Ceph, etc.). There will be a discussion on the components along with an architectural explanation of NFS Ganesha with a detailed look at how a request flows through the various layers of NFS Ganesha and see some critical ...
|
| 2/2/20 |
As data is becoming more and more important in the world, we can't afford to lose it even if there is a natural calamity. We will see how Geo-Replication came in to solve this problem for us and how it evolved over the days. Through this session, the users will learn how easy it is to set up Georep for Gluster to use it for their storage and back up their data with minimal understanding of storage and linux. Having a basic Gluster knowledge will make it even more easy
|
| 2/2/20 |
While running in user space ZFS utilizes a user space binary called ztest. In cStor, we followed a similar approach to create a binary called ‘zrepl’ that is part of cStor. It has been built using the libraries similar to what is used for ztest and contains transactional, pooled storage layers. cStor uses ZFS behind the scenes by running it in the user space. This talk we will discuss in detail how we used ZFS in userspace for storage engine cStor and highlight a few challenges that our team ...
|
| 2/2/20 |
The presentation will give an overview of all the changes happening in the Samba project code, from the fileserver virtual filesystem (VFS) rewrite, the new features in the SMB3 code, the quest to remove the old SMB1 protocol and much more. Improvements in Samba scalability, clustering and the Active Directory code will be discussed. The intended audience is anyone who uses the Samba code, creates products with Samba or is interested in the SMB protocol.
|
| 2/2/20 |
Metadata-heavy workloads are often the bane of networked and clustered filesystems. Directory operations (create and unlink, in particular) usually involve making a synchronous request to a server on the network, which can be very slow. CephFS however has a novel mechanism for delegating the ability for clients to do certain operations locally. While that mechanism has mostly been used to delegate capabilities on normal files in the past, it's possible to extend this to cover certain types of ...
|
| 2/2/20 |
What is Rook and the architecture of Rook the storage run in Kubernetes. We'll also take a look at new features added to Rook.
|
