Session
FOSDEM 2021 Schedule
Containers

ML inference acceleration on K8s using kata containers & AWS Firecracker

D.containers
Orestis Lagkas Nikolos
The Serverless computing paradigm facilitates the use of cloud computing resources by developers without the burden of administering and maintaining infrastructure. This simplification of cloud programming appears ideal (in theory) but the catch is that when someone needs to perform a more complex task, things could get a bit more complicated. Hardware acceleration, for instance, has been a pain point, even for traditional cloud computing programming models: IaaS providers chose dedicated solutions to avoid interference and preserve tenant isolation (device passthrough), while losing one of the most important benefits of virtualization, flexibility in workload placement through live migration. Various solutions have been proposed to overcome this limitation (API remoting, hardware slicing etc.). In the Serverless world though, do we need users to interface with a hardware device directly? Most serverless deployments are backed by containers, however, the most popular (and used) one, AWS Lambda, uses a ligthweight VMM (AWS Firecracker) integrated in the container ecosystem, in order to ensure strict isolation, while maintaining scalability. To this end, enabling hardware acceleration on this kind of deployment incurs the same pain points with traditional cloud infrastructure. Kata containers evolved from clear containers and offer hypervisor support for popular orchestrators container deployments such as docker, Kubernetes etc. Through kata containers, AWS Firecracker VMs can be easily provisioned as Pods on a kubernetes system, serving workloads prepared as container images. We build on the kata container runtime and port the necessary components to support vAccel, a lightweight framework for hardware acceleration on VMs, on Firecracker. In this talk, we briefly go through vAccel, its design principles and implementation, while focusing on the integration with kata-containers and the end-to-end system applicability on ML inference workloads. We present a short patch for kata-containers to support AWS Firecracker v0.23.1, and go through the necessary patching to add the vAccel framework on k8s. Finally, we present a short demo that scales image classification purpose-built microVMs across a working K8s cluster with GPUs. Hardware acceleration for serverless deployments has never been more secure!

Additional information

Type devroom

More sessions

2/7/21
Containers
Daniel Borkmann
D.containers
BPF is becoming ubiquitous in today's modern container environments and thanks to the fast pace of innovations from Linux kernel developers in the BPF subsystem, cloud native networking software such as Cilium is able to bring these extensions to a mainstream user base for improving throughput, latency and reliability of workloads and services. This talk provides a deep dive on recently added BPF kernel as well as Cilium extensions for Kubernetes environments which significantly reduce ...
2/7/21
Containers
Jakub Dżon
D.containers
Operator SDK is a solid foundation for building robust applications for Kubernetes; one of such applications is the VM import operator (https://github.com/kubevirt/vm-import-operator) allowing Kubernetes administrators to easily import their oVirt-managed virtual machines to KubeVirt. In this talk, the speaker will show how his team used Operator SDK to build the VM import operator and how that operator can be used.
2/7/21
Containers
Vlad Bogolin
D.containers
Containers are a central point for the MariaDB buildbot (buildbot.mariadb.org). In fact, almost all our builds run in Docker containers. In this short presentation, I will talk about the container environment used in order to build MariaDB from source both on Linux and Windows. Then, I will present some of the challenges associated with running Windows in a Docker container and finally I will focus on some of the advantages of having a container based continuous integration infrastructure.
2/7/21
Containers
Viktor Farcic
D.containers
What are we going to do without Docker inside Kubernetes clusters?
2/7/21
Containers
akrem
D.containers
Hello, Our solution is developed from scratch using python django we aim to provide easy way and flexible to deploy virtual environment in enterprise , we use lxd as our core hypervisor and we developed a complete solution how to manage all machines and resources utilization with implementing all the lxd features And every thing is packaged one script it will do all the handy work we use NoVnc as Xserver to visualize the desktop monitoring tools grafana and ELK stack
2/7/21
Containers
Christian Brauner
D.containers
On most POSIX systems including Linux file ownership can only be changed globally, i.e. for all users through the chown*() syscall family. In this talk we will introduce idmapped mounts. Idmapped mounts allow to change the ownership of files under the mounts they appear in.
2/7/21
Containers
Peter Zaitsev
D.containers
DBaaS is the fastest growing way to deploy databases. It is fast and convenient and it helps to reduce toil a lot, yet it is typically done using proprietary software and tightly coupled to the cloud vendor. We believe Kubernetes finally allows us to build fully OpenSource DBaaS Solution capable to be deployed anywhere Kubernetes runs - on the Public Cloud or in your private data center. In this presentation, we will describe the most important user requirements and typical problems you would ...