A Programming Language Perspective on Replication

<p>Distributed systems replicate data to improve availability, scalability, and fault tolerance. Ensuring that replicas remain eventually consistent is difficult. Current practices advocate for the use of Replicated Data Types (RDTs) which guarantee convergence out-of-the-box, e.g. CRDTs. However, programming distributed systems using these RDTs is non-trivial due to the lack of appropriate abstractions for replica discovery, update propagation, etc.</p> <p>At our research lab at the Vrije Universiteit Brussel, we look at what it means for developers to build and utilise eventually consistent data types in their applications. The majority of current RDT approaches are ad-hoc, they require a dedicated implementation for each data type. However, building advanced collaborative applications requires custom RDTs that are tailored to the needs of the application. That implies extending or composing existing RDTs, or designing new ones.</p> <p>Our goal is to develop abstractions and methodologies for the systematic construction of applications requiring replicated state. In this talk, we particularly focus on two main aspects: (1) non-ad hoc approaches to efficient RDT implementations, and (2) simplifying the development of application-specific RDTs:</p> <p>Many approaches use ad-hoc solutions to track causality and ensure eventual convergence, e.g. keeping meta-data in the implementation. In a lot of cases, their design does not translate well into efficient implementations and is not suitable for resource-constrained runtimes. We present Flec[1, 2, 3], a framework that guides developers in making informed decisions during the development process, by providing an API that gives developers a uniform and structured way to deal with functional system requirements. This can range from network constraints to security and authorization aspects.</p> <p>To simplify the development of collaborative applications using RDTs, we investigate alternative approaches where RDTs can be automatically derived from their sequential implementation [4, 5]. By means of an analysis we can detect conflicting operations, and automatically derive functional CRDTs. In some datatypes, certain application invariants would be impossible to guarantee with CRDTs. For these cases, we support automatically detecting where an application would have to synchronise and output an RDT with mixed consistency. </p> <p>[1] <a href="https://cris.vub.be/ws/portalfiles/portal/116156754/Jim_Bauwens_PhD_thesis.pdf">Jim Bauwens, 2024. Flexible CRDTS for a demanding world. PhD Thesis</a> [2] Jim Bauwens and Elisa Gonzalez Boix. 2020. Flec: a versatile programming framework for eventually consistent systems. Proceedings of the 7th Workshop on Principles and Practice of Consistency for Distributed Data. Association for Computing Machinery, New York, NY, USA, Article 12, 1–4. <a href="https://doi.org/10.1145/3380787.3393685">DOI</a> [3] <a href="https://gitlab.soft.vub.ac.be/jimbauwens/flec">https://gitlab.soft.vub.ac.be/jimbauwens/flec</a> [3] Kevin De Porre, Carla Ferreira, Nuno Preguiça, and Elisa Gonzalez Boix. 2021. ECROs: building global scale systems from sequential code. Proc. ACM Program. Lang. 5, OOPSLA, Article 107 (October 2021), 30 pages. <a href="https://doi.org/10.1145/3485484">DOI</a> [4] <a href="https://github.com/verifx-prover/verifx/tree/main">https://github.com/verifx-prover/verifx/tree/main</a></p>