Go

Speed up the monolith

building a smart reverse proxy in Go
UB2.252A (Lameere)
Alessio Caiazza
GitLab is a ruby on rails application, but this didn’t prevent us from having fun with Go. Learn how we decomposed our monolith by writing a smart reverse proxy in Go that handles I/O intensive operations. A technique that every web app can use, regardless of the company stack.
We set a deadline for releasing a cloud-native version of GitLab and put a team of engineers to work planning the helm charts, splitting several components into independently scalable PODs. The team faced a few challenges. GitLab’s main codebase is written in Ruby, which has a global interpreter lock. We relied on NFS to asynchronously upload files from our workers fleet. Removing shared file system by uploading directly from the controller was not an option. We wanted to move to an object storage based solution, but that was a paid feature and we had to port it to the open-source codebase. Oh, we also needed to make sure the rest of our engineers could keep shipping new features at our regular monthly cadence. At the same time, we were planning our infrastructure migration from Azure to Google Cloud. Removing this intermediate state, where a file is on GitLab server NFS but not yet uploaded to the object storage, would have made the migration a lot easier. We had to remove the NFS dependency to make GitLab easily deployable on Kubernetes and we needed a performant multi-cloud object storage uploader viable also for on-prem installations, a solution that would work for a single server setup up to Ggitlab.com scale. Luckily we already had written workhorse, a smart reverse proxy written in Go for handling git operations. It was time to extend workhorse capabilities leveraging the full power of goroutines. We had a plan, but the devil is in the detail. Allow me to guide you through this journey. During the talk I’ll tell you how a ruby-on-rails company began to write Go code, how we implemented an object storage uploader inside our proxy, the problems we faced, and tradeoffs we took to deliver this in time.

Additional information

Type devroom

More sessions

2/2/20
Go
UB2.252A (Lameere)
Go 1.14 is planned to be released in February 2019 and this talk covers what's coming up with it. We'll talk about new features and fixes in Go, new proposals for Go 2. All of the new things you might have missed.
2/2/20
Go
Dylan Meeus
UB2.252A (Lameere)
Are you tired of seeing Object Oriented code everywhere, with mutations, side-effects and headaches? Luckily, writing Go does not have to be that way! Functional programming is perfectly possible within Go, hence we can leverage FP techniques to make our code more robust, testable and fun.
2/2/20
Go
Maya Rashish
UB2.252A (Lameere)
An introduction to calling conventions, thread-local storage, signal handling and how they relate to Go, in the context of my new port of Go to NetBSD/arm64.
2/2/20
Go
Daniel Esteban
UB2.252A (Lameere)
Learn the multiples and fun possibilities of using Go on microcontrollers like Arduino to make gaming related hardware.
2/2/20
Go
Ronna Steinberg
UB2.252A (Lameere)
What if we decided to solve, once and for all, the problem of underrepresentation in the Go community of women, gender minorities, people of color, or any other group the same way we handle our problems in production, by identifying "bugs" and then fixing them? Can it even be done? What if we took the engineering approach? Ronna is planning to convince you it's not a matter of if, but a matter of how, and we are going to analyze some of the statistics, find where the problems actually lay, and ...
2/2/20
Go
Stefan Schimanski
UB2.252A (Lameere)
Kubernetes is built using Golang. CustomResourceDefinitions are the primary extension points for bringing custom data into a Kubernetes cluster. This hands-on talk is about the workflow of API definitions in Golang, generation of OpenAPI schemas as part of the CRD, client and informer generation and how to use these to process data in real-time using logic implemented in Golang controllers.
2/2/20
Go
Derek Parker
UB2.252A (Lameere)
In this talk I will begin by delivering the "State of Delve" update. In similar fashion to the popular "State of Go" talk I will begin by discussing all of the exciting new features and changes that have happened over the past year, since last FOSDEM. Following that I will go into a live demo showcasing how Delve can leverage Mozilla RR in order to perform deterministic debugging. This talk will begin by introducing users to the concept of deterministic debugging and the power that comes with ...