krew-wasm
krew-wasm is an experimental project that demonstrates how kubectl plugins could be written using WebAssembly and WASI.
krew-wasm takes inspiration from the Krew project, the official plugin manager for kubectl. However, on top of being a completely different codebase, krew-wasm does not aim to replace Krew. It's a complementary tool that can be used alongside with Krew.
The sole purpose of krew-wasm is to manage kubectl plugins written using WebAssembly and WASI.
How it works
kubectl plugins are simple binaries that follow this naming convention:
kubectl-<name of the plugin>
.
Once placed in a known $PATH
, the plugin becames available by just invoking
kubectl <name of the plugin>
.
krew-wasm can download kubectl WebAssembly plugins and make them discoverable
to the kubectl tool.
This is achieved by creating a symbolic link for each managed plugin. This symbolic
link is named kubectl-<name of the plugin>
but, instead of pointing to the
WebAssembly module, it points to the krew-wasm
executable.
Once invoked, krew-wasm
determines its usage mode which could either be a
"direct invocation" (when the user invokes the krew-wasm
binary to manage plugins)
or it could be a "wrapper invocation" done via kubectl
.
When invoked in "wrapper mode", krew-wasm takes care of loading the WebAssembly plugin and invoking it. krew-wasm works as a WebAssembly host, and takes care of setting up the WASI environment used by the plugin.
The WebAssembly runtime
Thanks to WASI, the plugins are granted access to the following portions of the host:
- Standard input, output and error
- Environment variables
- The whole home directory of the user, with read and write privileges
Because of that, the plugins can read the default kubeconfig file, or the one
referenced by the KUBECONFIG
environment variable (as long as it's located
inside of the user home directory).
The plugins can also interact with the user like any regular cli application.
Network access
Currently, the WebAssembly WASI specification doesn't cover network access for guest modules. However, kubectl plugins need to interact with the Kubernetes API server.
Network access is granted via an experimental interface provided by the krew-wasm runtime.
The interface is defined via the WIT
format and can be found
here.
Currently, plugins are allowed to make http requests only against the Kubernetes API server that is defined inside of the default kubeconfig file.
Why?
Why would someone be interested in writing kubectl plugins in this way?
That's a legitimate question. I think the main advantages are:
- WebAssembly is portable: you don't have to build your plugin for all the possible operating systems and architectures the end users might want. Portability remains a problem of krew-wasm maintainers, not of plugin authors.
- Security: WebAssembly modules are executed inside of a dedicated sandbox. They cannot see other processes running on the host nor have access to the host filesystem. Note well: currently this POC gives access to the whole user home directory to each plugin. We have plans to change that and give access only to the kubeconfig file and all the resources referenced by it.
- Size: the majority of kubectl plugins are written using Go, which produces
big binaries (the average size of a kubectl plugin is around ~9Mb). A Rust
plugin compiled into WebAssembly is almost half the size of it (~4.2 Mb).
This can be even trimmed down a bit by running a WebAssembly binary optimizer
like
wasm-opt
.
Last but not least, this was fun! We have high expectations for WebAssembly and WASI, this is just another way to prove that to the world 🤓
Limitations
Currently the biggest pain point is network access. The network interface introduced by krew-wasm kinda solves the problem, but there are some limitations.
Regular Kubernetes client libraries cannot be used. With Rust the k8s-openapi
can be used, but just because it uses a sans-io
approach.
We don't think other languages feature a library similar to k8s-openapi
.
Finally, all the long polling requests issued via k8s-openapi
(like all the watch
operations) are not going to work because of limitations with how data
is exchanged between the WebAssembly host and the guest.
All these limitations should be solved once WASI implements the socket proposal. That, among other things, should allow the usage of regular Kubernetes clients.
Installation
Download the right pre-built binary from the GitHub Releases page and
install it in your $PATH
.
Usage
List plugins
The list of installed plugins can be listed with this command:
krew-wasm list
Download and install a plugin
Plugins are distributed via OCI registries, the same infrastructure used to distribute container images.
Plugins can be downloaded and made available with this command:
krew-wasm pull <OCI reference>
For example:
krew-wasm pull ghcr.io/flavio/krew-wasm-plugins/kubewarden:latest
This command downloads and installs the latest
version of the kubectl-kubewarden
that is published inside of the ghcr.io/flavio/krew-wasm-plugins/kubewarden
registry.
Uninstall plugins
Plugins can be removed from the system by using the following command:
krew-wasm rm <name of the plugin>
The name of the plugin can be obtained by using the list
command.
Writing a plugin
Note well: this is still a POC, the documentation is limited, but will be improved in the future
Plugins are written as regular WebAssembly modules leveraging the WASI interface.
This website has many examples about "Hello World" WASI programs.
A demo policy, that interacts with the API server, can be found here.
Examples
These are some kubectl plugins written using WebAssembly and WASI:
- kubectl-decoder: decode Kubernetes
Secret
s, provide detailed information about x509 certificates found inside of theSecret
- kubectl-kubewarden: display information about the kubewarden stack
Do you have other plugins made with WebAssembly and WASI? Open a PR!
Acknowledgements
The idea about writing kubectl plugins using WebAssembly was born by Rafael Fernández López during a brain storming session with Flavio Castelli.