Session
Schedule FOSDEM 2020
Go

Dragons of CGO

Hard-learned Lessons from Writing Go Wrappers
UB2.252A (Lameere)
K.S. Bhaskar
YottaDB is a mature, hierarchical key-value, free / open source NoSQL database whose is used in enterprise-scale mission-critical applications in banking and healthcare, and also scales down to fit on IoT devices like the Raspberry Pi Zero, as well as applications in-between (like the University of Antwerp library catalog system). When a customer funded us to develop a Go API to YottaDB, we thought it would be a straightforward project. But it was a very painful exercise for us. The presentation discusses the problems we faced, and how we overcame them.

Go is a popular language for writing highly concurrent software, and works well when used in isolation. Using Go alongside software written in other languages (such as C) can be done, but there are some hidden dragons to keep an eye out for. In addition to obvious problems, such as calling C variadic functions, other more subtle problems are hidden deep in the depths of Go documentation.

Consider callback functions; how does one pass a function pointer from Go to C to provide a callback? There are strict limitations enforced by Go on what pointers may be passed to C routines, intended to prevent faults resulting from Go structures being garbage collected without knowledge of the C code. Without being able to pass pointers, even function pointers, how do we “pass” a callback function to the C code to callback into? Furthermore, how do we pass data to the callback function, since we can’t pass Go structures?

Given that we can’t pass Go structures to C code, at some point we will need to allocate C structures to store data for the C code to operate on. Go promises one thing about memory allocated in C land; it will not keep track of it for you. The garbage collector will gladly clean up any Go structures no longer needed, but will not clean up the associated C memory. How can one write code which isn’t likely to result in memory leaks, using this model?

Perhaps the most difficult challenge to overcome is that Go makes no promises about what thread is running code. Go does its best to hide the identity of Go routines from the user, so they won’t rely on this metadata for handling code execution. This presents a problem for many C applications, which often use POSIX mutexes to control access, and the owner of a mutex needs to belong to a specific thread. How can one write Go applications that allow the concurrency Go users expect, without trashing the libraries they are calling?

Of course, none of the solutions we talk about here are any good unless you can compile your program. The hidden dragons of Go also lurk behind the “go build“ command; fitting in the required C flags requires knowledge of, among other things, pkg-config, a systems tool used behind the scenes by programs like CMake and autotools.

We had to tackle all these problems, and many more, during the development of the YottaDB Go wrapper (https://gitlab.com/YottaDB/Lang/YDBGo and https://yottadb.com). This presentation hopes to pass some of our hard-learned lessons to other programmers who will use Go to interface with non-Go libraries and utilities.

Additional information

Type devroom

More sessions

2/2/20
Go
UB2.252A (Lameere)
Go 1.14 is planned to be released in February 2019 and this talk covers what's coming up with it. We'll talk about new features and fixes in Go, new proposals for Go 2. All of the new things you might have missed.
2/2/20
Go
Dylan Meeus
UB2.252A (Lameere)
Are you tired of seeing Object Oriented code everywhere, with mutations, side-effects and headaches? Luckily, writing Go does not have to be that way! Functional programming is perfectly possible within Go, hence we can leverage FP techniques to make our code more robust, testable and fun.
2/2/20
Go
Maya Rashish
UB2.252A (Lameere)
An introduction to calling conventions, thread-local storage, signal handling and how they relate to Go, in the context of my new port of Go to NetBSD/arm64.
2/2/20
Go
Daniel Esteban
UB2.252A (Lameere)
Learn the multiples and fun possibilities of using Go on microcontrollers like Arduino to make gaming related hardware.
2/2/20
Go
Ronna Steinberg
UB2.252A (Lameere)
What if we decided to solve, once and for all, the problem of underrepresentation in the Go community of women, gender minorities, people of color, or any other group the same way we handle our problems in production, by identifying "bugs" and then fixing them? Can it even be done? What if we took the engineering approach? Ronna is planning to convince you it's not a matter of if, but a matter of how, and we are going to analyze some of the statistics, find where the problems actually lay, and ...
2/2/20
Go
Stefan Schimanski
UB2.252A (Lameere)
Kubernetes is built using Golang. CustomResourceDefinitions are the primary extension points for bringing custom data into a Kubernetes cluster. This hands-on talk is about the workflow of API definitions in Golang, generation of OpenAPI schemas as part of the CRD, client and informer generation and how to use these to process data in real-time using logic implemented in Golang controllers.
2/2/20
Go
Derek Parker
UB2.252A (Lameere)
In this talk I will begin by delivering the "State of Delve" update. In similar fashion to the popular "State of Go" talk I will begin by discussing all of the exciting new features and changes that have happened over the past year, since last FOSDEM. Following that I will go into a live demo showcasing how Delve can leverage Mozilla RR in order to perform deterministic debugging. This talk will begin by introducing users to the concept of deterministic debugging and the power that comes with ...