RFC: support kuasar runtime with containerd
xiaods opened this issue · comments
as pure kubernetes distribution, k8e's feature is almost like k3s. we aimed to fast enterprise adoption with serverless technology.
- k8e containerd support devmapper plugin
- kuasar support
- Testing feature
Refs:
use kata to quickstart use vmm on k8s.
Proposed solution: provide another engine for K8E, Kata provides a one-click reference that can work directly with runc. And firecrack does not apply directly to K8S, this limits our design, so, after consideration, it is recommended to unify the management of VMM containers through Kata
follow kata-containers/kata-containers#2207 we need let k8e containerd support devmapper plugin.
[root@ip-172-31-28-183 ~]# k8e ctr plugins list
TYPE ID PLATFORMS STATUS
io.containerd.content.v1 content - ok
io.containerd.snapshotter.v1 aufs linux/amd64 skip
io.containerd.snapshotter.v1 btrfs linux/amd64 skip
io.containerd.snapshotter.v1 devmapper linux/amd64 error
- 首先安装Device Mapper，使用命令
yum install device-mapper-persistent-data - 配置Device Mapper，使用命令
systemctl enable devicemapper - 打开/etc/lvm/lvm.conf文件，添加以下内容：
devices {
dir = "/dev/mapper"
scan = [ "/dev/mapper" ]
filter = [ "a|/dev/loop|", "a|/dev/dm-|", "r/.*/" ]
}
- 保存并关闭文件
- 启动Device Mapper，使用命令
systemctl start devicemapper - 下载containerd的devmapper plugin，使用命令
wget https://github.com/containerd/devmapper/releases/download/v1.1.1/devmapper-1.1.1-linux-amd64.tar.gz - 解压文件，使用命令
tar -xvf devmapper-1.1.1-linux-amd64.tar.gz - 将devmapper plugin 安装到containerd，使用命令
containerd config default | sudo tee /etc/containerd/config.toml - 重启containerd，使用命令
systemctl restart containerd - 验证devmapper plugin的安装，使用命令
containerd plugin ls | grep devmapper
在/var/lib/k8e/agent/etc/containerd/confg.toml.tmpl 中配置模板, 然后重启服务后k8e就会加载devmapper plugin snapshotter
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = "/var/lib/k8e/agent/containerd"
pool_name = "devpool"
base_image_size = "8192MB"
discard_blocks = true
reference: https://github.com/containerd/containerd/blob/main/docs/snapshotters/devmapper.md
- sudo ctr plugin ls, make sure the devmapper plugin status is ok
- sudo dmsetup ls check devices exist.
- check /var/lib/k8e/agent/etc/containerd/confg.toml configuration is OK
report issue on kata community: kata-containers/kata-containers#6317
report issue on kata community: kata-containers/kata-containers#6317
no response from kata community.
又发现有的库来实现VMM
libkrun is a dynamic library that allows programs to easily acquire the ability to run processes in a partially isolated environment using KVM Virtualization.
libkrun的主要目标是在单个动态库中提供所有虚拟化过程隔离所需的功能（包括VMM、模拟设备和Guest内核等），以便可以轻松地在不同命名空间之间切换，并且具有最小可能的内存占用。
我们并不试图成为通用VMM，这使得我们能够减少一些负载，并且专注于此使用案例可以将对外部支持进程和shim的需求降至最低。
我们还正在开发实验性扩展Guest内核，以打破VM和容器之间的障碍，例如透明套接字模仿（Transparent Socket Impersonation），它提供了一定程度的网络命名空间透明度，并会有更多功能推出。
总而言之，libkrun仍处于早期开发阶段，尚未准备好投入生产。因此，如果您需要短期解决方案，则QEMU或Firecracker（与Kata或其他运行时组合）可能是您需要的。
https://kuasar.io/docs/architecture/why-kuasar/ 今天看到这个，直接替换 runc，提供多 sandbox 方案。可以啊。
kuasar 正在申请 cncf sandbox
k8e 替换了 containerd , PR #331
Expires, close it