Session
FOSDEM Schedule 2021
Continuous Integration and Continuous Deployment

MariaDB Buildbot, the journey of a reinvention

D.cicd
Vlad Bogolin
Recently, the MariaDB Foundation has been developing a new continuous integration framework for the MariaDB Server. The goal of buildbot.mariadb.org is to ensure that each change is properly tested on all supported platforms and operating systems. Our new CI uses almost exclusively latent workers, more exactly Docker latent workers. In this talk, I will present a main overview of the CI infrastructure, the advantages of using latent workers and talk about the challenges that we encountered along the way. This includes a broad range of aspects, ranging from misconfigurations to Buildbot code changes to ensure that everything runs smoothly.
In order to ensure that MariaDB runs smoothly, it needs to be tested on multiple platforms and configurations. In order to obtain a gain both in terms of speed and flexibility, we have decided to use Docker latent workers. In this way, each different environment is defined in a separate dockerfile. Besides having a clean and concise environment definition, using latent workers has the advantage of requiring minimal configurations on the worker machines. This makes the process of adding new hardware very easy, mainly involving installing Docker and configuring Docker remote access. Now that we have the build environments defined, we can start testing. The process starts by cloning the MariaDB server repo and creating a source tarball. Then, all other configurations are triggered, all using the same source tarball. While it seemed quite straightforward, the whole process turned out to be more challenging than we expected. This includes a quite long list of issues ranging from files mixups, weird sporadic failures where the main testing process was killed, grid view customizations to multi-master configuration and master-worker file transfer problems. In this talk, I will talk in more detail about these issues and tell you more about our experience and how we managed to overcome them.

Additional information

Type devroom

More sessions

2/7/21
Continuous Integration and Continuous Deployment
D.cicd
Introduction of the CI/CD devroom
2/7/21
Continuous Integration and Continuous Deployment
D.cicd
The emergence of virtualization, containers, and cloud native has resulted in tremendous advances in enabling organizations to develop new services and make them available to end users. In addition, new paradigms such as Continuous Integration (CI) and Continuous Delivery (CD) allow organisations to do this much faster than before, empowering them to go to market ahead of the competition. Despite its many advantages, the CI/CD ecosystem has its challenges. This session will discuss issues ...
2/7/21
Continuous Integration and Continuous Deployment
D.cicd
Three phrases keep popping up when talking about modern workflows and development and deployment techniques; CD, GitOps, and progressive delivery.
2/7/21
Continuous Integration and Continuous Deployment
Andrea Frittoli
D.cicd
Continuous integration and deployment (CI/CD) system are hardly ever ceaseless as the name would suggest; they do aim though to follow changes in code, configurations and versions. They often achieve that by both handling and generating events. For instance, a CD system receives an event that describes a new version of an application, and it runs a workflow in response. When the workflow starts or when it reaches completion, the CD system generates events for the benefit of other processes that ...
2/7/21
Continuous Integration and Continuous Deployment
D.cicd
When Everything as Code converges to automate/test your processes, in this talk we would like to discuss further our journey and our vision to handle our automation programmatically.
2/7/21
Continuous Integration and Continuous Deployment
Philipp Strube
D.cicd
Kubernetes, GitOps, and Infrastructure as Code are as powerful as they are popular and seem like the perfect match. Consequently, using Terraform to maintain Kubernetes clusters and resources is a very common use-case. And it requires careful integration of many moving parts, from Terraform providers and modules, to CI/CD pipelines and triggers. However, despite this being such a popular use-case, teams previously had no alternative than writing everything from scratch. On the software ...
2/7/21
Continuous Integration and Continuous Deployment
Tom Eyckmans
D.cicd
The Kubernetes tooling landscape is littered with template based solutions to deal with all the YAML needed to get things done. Kuberig takes a different approach that developers will love! No need to learn another template language or tool. With Kuberig you define your resources using Kotlin code and deploy them by executing Gradle tasks. Simple.