Containers
ML inference acceleration on K8s using kata containers & AWS Firecracker
The Serverless computing paradigm facilitates the use of cloud computing resources by developers without the burden of administering and maintaining infrastructure. This simplification of cloud programming appears ideal (in theory) but the catch is that when someone needs to perform a more complex task, things could get a bit more complicated.
Hardware acceleration, for instance, has been a pain point, even for traditional cloud computing programming models: IaaS providers chose dedicated solutions to avoid interference and preserve tenant isolation (device passthrough), while losing one of the most important benefits of virtualization, flexibility in workload placement through live migration. Various solutions have been proposed to overcome this limitation (API remoting, hardware slicing etc.). In the Serverless world though, do we need users to interface with a hardware device directly?
Most serverless deployments are backed by containers, however, the most popular (and used) one, AWS Lambda, uses a ligthweight VMM (AWS Firecracker) integrated in the container ecosystem, in order to ensure strict isolation, while maintaining scalability. To this end, enabling hardware acceleration on this kind of deployment incurs the same pain points with traditional cloud infrastructure.
Kata containers evolved from clear containers and offer hypervisor support for popular orchestrators container deployments such as docker, Kubernetes etc. Through kata containers, AWS Firecracker VMs can be easily provisioned as Pods on a kubernetes system, serving workloads prepared as container images. We build on the kata container runtime and port the necessary components to support vAccel, a lightweight framework for hardware acceleration on VMs, on Firecracker.
In this talk, we briefly go through vAccel, its design principles and implementation, while focusing on the integration with kata-containers and the end-to-end system applicability on ML inference workloads. We present a short patch for kata-containers to support AWS Firecracker v0.23.1, and go through the necessary patching to add the vAccel framework on k8s. Finally, we present a short demo that scales image classification purpose-built microVMs across a working K8s cluster with GPUs.
Hardware acceleration for serverless deployments has never been more secure!
Additional information
| Type | devroom |
|---|
More sessions
| 2/7/21 |
BPF is becoming ubiquitous in today's modern container environments and thanks to the fast pace of innovations from Linux kernel developers in the BPF subsystem, cloud native networking software such as Cilium is able to bring these extensions to a mainstream user base for improving throughput, latency and reliability of workloads and services. This talk provides a deep dive on recently added BPF kernel as well as Cilium extensions for Kubernetes environments which significantly reduce ...
|
| 2/7/21 |
Operator SDK is a solid foundation for building robust applications for Kubernetes; one of such applications is the VM import operator (https://github.com/kubevirt/vm-import-operator) allowing Kubernetes administrators to easily import their oVirt-managed virtual machines to KubeVirt. In this talk, the speaker will show how his team used Operator SDK to build the VM import operator and how that operator can be used.
|
| 2/7/21 |
What are we going to do without Docker inside Kubernetes clusters?
|
| 2/7/21 |
Containers are a central point for the MariaDB buildbot (buildbot.mariadb.org). In fact, almost all our builds run in Docker containers. In this short presentation, I will talk about the container environment used in order to build MariaDB from source both on Linux and Windows. Then, I will present some of the challenges associated with running Windows in a Docker container and finally I will focus on some of the advantages of having a container based continuous integration infrastructure.
|
| 2/7/21 |
On most POSIX systems including Linux file ownership can only be changed globally, i.e. for all users through the chown*() syscall family. In this talk we will introduce idmapped mounts. Idmapped mounts allow to change the ownership of files under the mounts they appear in.
|
| 2/7/21 |
DBaaS is the fastest growing way to deploy databases. It is fast and convenient and it helps to reduce toil a lot, yet it is typically done using proprietary software and tightly coupled to the cloud vendor. We believe Kubernetes finally allows us to build fully OpenSource DBaaS Solution capable to be deployed anywhere Kubernetes runs - on the Public Cloud or in your private data center. In this presentation, we will describe the most important user requirements and typical problems you would ...
|
| 2/7/21 |
Although Kubernetes is the leading container orchestration solution, it does not necessarily solve all container management-related challenges that one might face. Leaving fashions aside, some other technologies may actually be a better solution for some use cases and projects. Kubernetes is actually a very complex technology, with limited support for multi-tenancy and lacking secure isolation between tenants. Kubernetes does not offer cloud-like self-service provision features for users either. ...
|
