Keynote: eBPF - Everything You Need to Know in 5 Minutes - Thomas Graf, CTO, Isovalent

Introduction to eBPF

In this section, Thomas Graf introduces eBPF and its significance in extending operating systems.

What is eBPF?

00:28 eBPF is a programming language and runtime designed to extend operating systems.
00:55 It can be likened to JavaScript or Lua for kernel developers, enabling running programs in the kernel when specific events occur.

Why eBPF?

01:49 Operating systems like Linux are challenging to change quickly, leading to long innovation cycles.
02:48 eBPF offers programmability, allowing rapid adaptation to changing requirements and fostering innovation.

03:16

Distinguishing Features of eBPF

This part delves into how eBPF differs from other languages like Lua or WebAssembly and explains its unique characteristics.

How is eBPF Different?

03:40 Unlike Lua or WebAssembly, eBPF is embedded into the operating system, interfacing with the Linux kernel securely.
04:08 The key uniqueness of eBPF lies in its integration into the operating system, enhancing security and efficiency.

04:32

Working Mechanism of eBPF

Graf elaborates on the functioning of eBPF through its language, runtime verification, and bytecode compilation process.

Operational Process

04:57 The language used for expressing programs can vary, with C being common; compiled into bytecode for runtime verification.
05:25 The runtime embedded in the OS verifies safety, compiles efficiently just-in-time, offering secure execution at specified hook points.

05:49

eBPF Implementation and Usage

Details about who controls the governance of eBFP and where it finds application today are discussed by Graf.

Governance and Application

05:49 The governance of eBFP lies with the EBFP Foundation and open-source communities.

06:50

Detailed Overview of eBPF Usage in Android Operating System

In this section, the speaker discusses the extensive use of eBPF (Extended Berkeley Packet Filter) in the Android operating system for various purposes such as traffic accounting, memory usage tracking, and observability.

Android's Utilization of eBPF

06:50 Android OS employs eBPF for tasks like traffic accounting, CPU and memory usage tracking for applications, and even GPU memory accounting.

Multi-Cloud Networking with eBPF

07:18 Enterprises are leveraging eBPF for multi-cloud networking. An example is S&P Global using eBPF with Sodium to redefine networking paradigms.

Innovations in Observability through Grafana and I Surveillance Partnership

07:44 The partnership between Grafana and I Surveillance aims to enhance observability tools by combining Grafana's expertise with Loki, Tempo, etc., and I Surveillance's sodium tetragon Hubble.

Enhanced Observability Capabilities Enabled by eBPF

00:28 The collaboration introduces new tools for observability. Notable features include embedding Grafana dashboards into Psyllium's Hubble UI for network metrics visualization.

Security Layer Visibility Enhancement via Tetragon Project

09:01 Psyllium's Tetragon project enhances security visibility by displaying process ancestry and application behavior through Grafana toolsets.

Transcription

00:00:02 ➜ all right good morning everybody my name
00:00:05 ➜ is Thomas craft i'm co-founder and CTO
00:00:08 ➜ of I surveillance but probably more
00:00:10 ➜ importantly one of the creators of
00:00:12 ➜ psyllium and currently the chair of the
00:00:14 ➜ evpf governing board or ebpf Foundation
00:00:17 ➜ governing board I'm trying to give an
00:00:20 ➜ introduction to psyllium to ebpf not
00:00:23 ➜ selling ebpf in five minutes let's see
00:00:26 ➜ what that gets us
00:00:28 ➜ first of all what is ebpf how many of
00:00:30 ➜ you
00:00:31 ➜ um this is the first ebpf day how many
00:00:34 ➜ of you have been to an ebpf day before
00:00:37 ➜ first ebpf day
00:00:39 ➜ all right quite a few excellent so you
00:00:42 ➜ may be wondering what is actually BPF
00:00:44 ➜ and this is the the closest accurate
00:00:46 ➜ definition I could come to ebpf is a
00:00:49 ➜ programming language and run time to
00:00:52 ➜ extend operating systems
00:00:55 ➜ that sounds very abstract
00:00:57 ➜ a more practical comparison is ebpf is
00:01:02 ➜ like JavaScript or Lua
00:01:04 ➜ but for kernel Developers
00:01:07 ➜ now maybe you're even more confused so
00:01:09 ➜ let's look at this
00:01:11 ➜ on the left we have JavaScript an
00:01:13 ➜ example of JavaScript how it's commonly
00:01:15 ➜ used right you use JavaScript to run
00:01:17 ➜ some code when some event happens for
00:01:20 ➜ example most common example when the
00:01:22 ➜ website user clicks on a button it and
00:01:24 ➜ submits a form
00:01:25 ➜ on the right side we have ebpf very very
00:01:28 ➜ similar but you run ebpf programs in the
00:01:32 ➜ kernel in the operating system when some
00:01:34 ➜ event happens for example when a system
00:01:36 ➜ call is being invoked when a network
00:01:38 ➜ packet is being processed when a file is
00:01:40 ➜ being accessed and so on so very
00:01:42 ➜ different layers operating system for
00:01:44 ➜ ebpf JavaScript lure higher up in the
00:01:47 ➜ stack but conceptually they're actually
00:01:49 ➜ very similar they're making systems
00:01:51 ➜ programmable
00:01:56 ➜ why ebpf that's the second question I
00:01:59 ➜ usually get
00:02:01 ➜ operating systems in particular Linux
00:02:04 ➜ has become incredibly hard to change it
00:02:07 ➜ takes weeks or months to even get a
00:02:10 ➜ change upstream and then years for
00:02:12 ➜ consumers for users to consume these
00:02:15 ➜ Linux kernel versions these new versions
00:02:16 ➜ so the Innovation cycle is very very
00:02:20 ➜ very long very similar to hardware
00:02:23 ➜ this leads to this problem that's
00:02:25 ➜ similar to what we have for CPUs the
00:02:28 ➜ operating system is essentially using
00:02:31 ➜ building blocks because it has to
00:02:32 ➜ predict use cases it cannot adjust adopt
00:02:36 ➜ to changing requirements all the time
00:02:38 ➜ that's why the kernel the Linux kernel
00:02:40 ➜ typically has system subsystems that you
00:02:43 ➜ as a user can configure you cannot
00:02:45 ➜ program them for CPUs we have solved
00:02:48 ➜ these problems with higher level
00:02:49 ➜ programming languages like Java go rust
00:02:52 ➜ and so on
00:02:53 ➜ edpf is the same concept applied to
00:02:56 ➜ operating system so it gives us
00:02:58 ➜ programmability and thus allows us to
00:03:01 ➜ continuously adopt to changing
00:03:03 ➜ requirements and thus innovate very very
00:03:05 ➜ quickly that's the biggest fundamental
00:03:07 ➜ reason why ebpf exists
00:03:11 ➜ great so we have programmability how is
00:03:14 ➜ this different to things like Lua or
00:03:16 ➜ webassembly because there's lots of
00:03:18 ➜ other languages
00:03:19 ➜ much much more widely known ones that
00:03:22 ➜ have been around for ages that give
00:03:24 ➜ programmability
00:03:26 ➜ why and how is ebpf different ebpf is
00:03:30 ➜ specifically designed to be embedded
00:03:32 ➜ into the operating system Linux and
00:03:34 ➜ windows it can interface with the Linux
00:03:37 ➜ kernel so it can call into kernel apis
00:03:40 ➜ and it is restricted to run safely in
00:03:43 ➜ the kernel context
00:03:45 ➜ but it is aimed for kernel developers
00:03:47 ➜ and thus traditionally very hard to
00:03:50 ➜ learn and use Lua webassembly on the
00:03:52 ➜ other hand are designed to be embedded
00:03:54 ➜ into arbitrary applications or system
00:03:56 ➜ their general purpose and their aimed
00:03:58 ➜ application Developers
00:04:00 ➜ much easier to learn but they cannot run
00:04:04 ➜ in the context of your of the operating
00:04:06 ➜ system so the single unique feature of
00:04:08 ➜ ebpf is that it is embedded into the
00:04:10 ➜ operating system that what makes our
00:04:12 ➜ ebpf unique we could for example have
00:04:15 ➜ chosen Lua as a runtime for the Lewis
00:04:18 ➜ kernel and give that a name that would
00:04:20 ➜ be similar to what ebpf is today
00:04:23 ➜ how does ebpf work there is a language
00:04:26 ➜ and there is a runtime
00:04:28 ➜ the language can be expressed in a
00:04:30 ➜ variety of different languages sudo C
00:04:32 ➜ code is the the most common one where we
00:04:34 ➜ use a compiler like llvm or clang to
00:04:37 ➜ compile that into bytecode but there's
00:04:39 ➜ several different language Frameworks IR
00:04:41 ➜ libpf BCC to express ebpf programs in
00:04:45 ➜ different higher language higher level
00:04:47 ➜ languages all of this eventually produce
00:04:49 ➜ so-called ebpf bytecode that is our
00:04:52 ➜ program
00:04:53 ➜ then we have a run time this is what's
00:04:55 ➜ embedded into the operating system the
00:04:57 ➜ runtime takes the bytecode verifies it
00:05:00 ➜ to make sure it is safe to run just in
00:05:02 ➜ time compiles it for efficiency and then
00:05:04 ➜ runs it at the requested hook points in
00:05:07 ➜ this example it is being run when system
00:05:09 ➜ calls are being invoked
00:05:12 ➜ so these three properties ebpf is secure
00:05:15 ➜ it has runtime verification it has a
00:05:18 ➜ Sandbox concept it has program signing
00:05:21 ➜ it is efficient because it is going
00:05:22 ➜ through a just in time compiler it is
00:05:25 ➜ embedded into the operating system and
00:05:27 ➜ it offers per CPU data structures and it
00:05:30 ➜ portable the bytecode is generic it gets
00:05:33 ➜ just in time compiled later on it has
00:05:35 ➜ data type Discovery BTF as well as a
00:05:38 ➜ stable API how to interface with the
00:05:40 ➜ operating system
00:05:42 ➜ who controls evpf evpf is controlled by
00:05:46 ➜ the ebpf foundation and the open source
00:05:49 ➜ communities around it if you are
00:05:51 ➜ interested in participating on the
00:05:53 ➜ governance side of ebpf foundation feel
00:05:55 ➜ free to go to ebpf.foundation
00:06:00 ➜ that's that's nice now we know about
00:06:02 ➜ ebpf is where is ebpf used today this
00:06:04 ➜ may actually be a surprise it's
00:06:06 ➜ incredibly widely used we could look at
00:06:09 ➜ the psyllium users file it's everywhere
00:06:10 ➜ but that's kind of boring lots of logos
00:06:12 ➜ let's look at some more extreme use
00:06:15 ➜ cases
00:06:16 ➜ Cloud native landscape we see a couple
00:06:18 ➜ of projects using evpf but let's dive
00:06:20 ➜ deeper and look at like heavy
00:06:22 ➜ applications of ebpf that you probably
00:06:24 ➜ use every day
00:06:26 ➜ facebook slash meta is using ebpr for
00:06:28 ➜ load balancing it's called project
00:06:30 ➜ katron it's been used in production from
00:06:32 ➜ for for several years all ebpf Facebook
00:06:35 ➜ is using this exclusively for all load
00:06:38 ➜ balancing needs we have all three big
00:06:41 ➜ cloud providers AWS Azure Google Cloud
00:06:44 ➜ using Solium in their managed kuberous
00:06:46 ➜ offerings many of you probably use a
00:06:48 ➜ managed kubernetes offering gke eks AKs
00:06:50 ➜ at some point they are using psyllium
00:06:52 ➜ and thus ebpf underneath
00:06:55 ➜ but even smartphones this is the evpf
00:06:58 ➜ page of the Android operating system
00:07:00 ➜ Android uses ebpf it has an ebpf loader
00:07:03 ➜ and it uses ebpf pro ebpf programs to do
00:07:06 ➜ traffic accounting it uses it to account
00:07:08 ➜ how much CPU memory each app uses and
00:07:11 ➜ can even do GPU memory accounting and so
00:07:13 ➜ on it's all done using eppf so when you
00:07:16 ➜ look at the traffic statistic of your
00:07:18 ➜ app how much traffic volume how much
00:07:19 ➜ Network volume it it consumed that was
00:07:22 ➜ counted using BPF
00:07:24 ➜ multi-cloud networking Enterprises there
00:07:27 ➜ was a fantastic presentation at ebpf
00:07:29 ➜ Summit a couple of weeks back where s p
00:07:32 ➜ Global presented how they're using ebpf
00:07:34 ➜ with sodium for a multi-cloud networking
00:07:37 ➜ layer to essentially redefine how they
00:07:39 ➜ think about networking the talk was
00:07:41 ➜ absolutely fantastic a road to invisible
00:07:44 ➜ Network smp's Global Network
00:07:45 ➜ transformation Journey you can go to
00:07:47 ➜ ebpf.io click on watch Summit recordings
00:07:50 ➜ and you will find a talk the talk
00:07:52 ➜ recording
00:07:54 ➜ but there is one more thing because the
00:07:56 ➜ biggest aspect of eppf where I think
00:07:59 ➜ we'll see a lot more Innovation going
00:08:00 ➜ forward is observability
00:08:03 ➜ so really happy to announce a absolutely
00:08:06 ➜ fantastic new uh partnership between
00:08:08 ➜ grafana and I surveillance grafana is
00:08:12 ➜ bringing their entire observability
00:08:13 ➜ knowledge and tools at Loki grafana
00:08:16 ➜ Tempo and so on and I surveillance is
00:08:18 ➜ bringing sodium tetragon Hubble for for
00:08:21 ➜ observability and all of our ebpf
00:08:24 ➜ knowledge and together we're building a
00:08:26 ➜ new set of tools that you can use for
00:08:28 ➜ observability
00:08:30 ➜ a couple of examples Network
00:08:32 ➜ observability where you we can now embed
00:08:35 ➜ grafana dashboards directly into
00:08:37 ➜ psyllium's Hubble UI for amazing Network
00:08:39 ➜ observability metrics for traffic volume
00:08:42 ➜ Network policy drops traffic volume and
00:08:46 ➜ so on
00:08:47 ➜ but there is more we can also do ebpf
00:08:49 ➜ based HTTP tracing and HTTP
00:08:51 ➜ observability and for example build
00:08:53 ➜ golden signal dashboards that we see on
00:08:55 ➜ the right side of the screen as well as
00:08:57 ➜ integration in the grafana tempo where
00:08:59 ➜ we can trace individual HTTP requests
00:09:01 ➜ all thanks to ebpf and those without
00:09:04 ➜ sidecars without proxies at all very
00:09:06 ➜ very fascinating and last but not least
00:09:09 ➜ because of because of tilium's tetragon
00:09:11 ➜ project we also have excellent
00:09:13 ➜ visibility into the security layer and
00:09:15 ➜ can for example display process ancestry
00:09:19 ➜ how how applications behave and what
00:09:22 ➜ commands they execute and visualize that
00:09:24 ➜ in uh in a grafana tool sets as well as
00:09:27 ➜ well as things like TLS handshake
00:09:29 ➜ monitoring
00:09:32 ➜ this was a five minute introduction to
00:09:34 ➜ to ebpf and um I tried to to use a
00:09:37 ➜ couple of real world examples as well
00:09:39 ➜ ebpf is already everywhere and we're
00:09:41 ➜ using it every day but there's lots of
00:09:43 ➜ more exciting things coming and I'm
00:09:45 ➜ looking forward to the to a whole day of
00:09:47 ➜ um presentations thanks a lot everybody

Laugharne / Keynote__eBPF_-_Everything_You_Need_to_Know_in_5_Minutes