Check out my first novel, midnight's simulacra!

EBPF: Difference between revisions

From dankwiki
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
eBPF (Enhanced [https://en.wikipedia.org/wiki/Berkeley_Packet_Filter Berkeley Packet Filter]) is a powerful toolchain capable of compiling high-level languages into a BPF bytecode, which is JITted into local machine code, and can be inserted into a running kernel. It builds atop kprobes, and is in the same family of tools as SystemTap and DTrace. It is driven through the [http://man7.org/linux/man-pages/man2/bpf.2.html <tt>bpf(2)</tt>] system call, though it is usually more convenient to employ the libbpf library and <tt>bpftool</tt> binary.
eBPF (Enhanced [https://en.wikipedia.org/wiki/Berkeley_Packet_Filter Berkeley Packet Filter]) is a powerful Linux kernel mechanism allowing bytecode to be attached to dynamic points in kernel and userspace, and implementing JIT of said bytecode to the host ISA, all on the fly using a running kernel. It builds atop [[kprobes]], and is in the same family of tools as SystemTap and DTrace. It is driven through the [http://man7.org/linux/man-pages/man2/bpf.2.html <tt>bpf(2)</tt>] system call, though it is usually more convenient to employ the libbpf library and <tt>bpftool</tt> binary. eBPF supports its own BTF debugging information, a simplified form of [[DWARF]].


eBPF supports its own BTF debugging information, a simplified form of [[DWARF]].
The BCC (BPF Compiler Collection) toolchain is capable of compiling high-level languages (a restricted C, Lua, etc.) into eBPF bytecode, and provides a high-level Python infrastructure around eBPF. <tt>bpftrace</tt> provides an awk-like language geared towards eBPF "one-liners." The [[XDP|eXpress Data Path (XDP)]] is built atop eBPF.


==Tools==
==Tools==
* <tt>bpftool</tt> can be built in <tt>tools/bpf</tt> of the installed kernel's source.
* <tt>bpftrace</tt> provides a terse DSL that looks an awful lot like awk, allowing simple eBPF programs to be instantiated and attached directly from the command line.
* <tt>bpftrace</tt> provides a terse DSL that looks an awful lot like awk, allowing simple eBPF programs to be instantiated and attached directly from the command line.
* <tt>llvm-readelf</tt> can analyze an ELF object, including those targeting eBPF
* <tt>llvm-objdump</tt> can disassemble an ELF object to eBPF bytecode
* <tt>llvm-objdump</tt> can disassemble an ELF object to eBPF bytecode
===bpftool===
<tt>bpftool</tt> can be built in <tt>tools/bpf</tt> of the installed kernel's source.
{|class="wikitable"
! Subcommand !! Role
|-
| feature || examines the running kernel, and enumerates all capabilities present related to eBPF (program types, map types, kernel config options, eBPF helpers, etc.)
|-
| prog || enumerates attached eBPF programs, loads and attaches programs, and can dump loaded programs' bytecode (or JITted host code)
|-
| perf || lists [[kprobes]] and other tracepoints with attached programs
|-
| map || enumerates and manipulates maps
|-
|}
===[[perf]]===
Kernel 4.4 added general events to the [[perf]] infrastructure. Kernel 4.9 added the ability for eBPF to register perf events. These are the preferred method for streaming events to userspace from an eBPF program.


==Compiling eBPF==
==Compiling eBPF==
Line 15: Line 32:
LLVM has enjoyed <tt>bpf</tt> backend support since 3.7. Compile using <tt>-target bpf</tt> to generate BPF bytecode, adding <tt>-g</tt> to generate BTF information.
LLVM has enjoyed <tt>bpf</tt> backend support since 3.7. Compile using <tt>-target bpf</tt> to generate BPF bytecode, adding <tt>-g</tt> to generate BTF information.


<tt>readelf</tt> on the resulting object ought indicate a <tt>Machine</tt> of "Linux BPF" or "EM_BPF". The resulting object can be loaded into the kernel with <tt>bpftool prog load</tt> or libbpf's <tt>bpf_object__open()</tt>.
<tt>readelf</tt> on the resulting object ought indicate a <tt>Machine</tt> of "Linux BPF" or "EM_BPF". The resulting object can be loaded into the kernel with <tt>bpftool prog load</tt> or libbpf's <tt>bpf_object__open()</tt>. When using <tt>bpftool prog load</tt>, you must specify a PATH within a mounted <tt>bpffs</tt> filesystem.


===Kernel JIT===
===Kernel JIT===
Line 25: Line 42:
* zoidbergwill's [https://github.com/zoidbergwill/awesome-ebpf awesome-ebpf] list
* zoidbergwill's [https://github.com/zoidbergwill/awesome-ebpf awesome-ebpf] list
* BCC [https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md reference guide]
* BCC [https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md reference guide]
* [https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md Kernel versions] supporting various features

Latest revision as of 22:23, 6 January 2023

eBPF (Enhanced Berkeley Packet Filter) is a powerful Linux kernel mechanism allowing bytecode to be attached to dynamic points in kernel and userspace, and implementing JIT of said bytecode to the host ISA, all on the fly using a running kernel. It builds atop kprobes, and is in the same family of tools as SystemTap and DTrace. It is driven through the bpf(2) system call, though it is usually more convenient to employ the libbpf library and bpftool binary. eBPF supports its own BTF debugging information, a simplified form of DWARF.

The BCC (BPF Compiler Collection) toolchain is capable of compiling high-level languages (a restricted C, Lua, etc.) into eBPF bytecode, and provides a high-level Python infrastructure around eBPF. bpftrace provides an awk-like language geared towards eBPF "one-liners." The eXpress Data Path (XDP) is built atop eBPF.

Tools

  • bpftrace provides a terse DSL that looks an awful lot like awk, allowing simple eBPF programs to be instantiated and attached directly from the command line.
  • llvm-readelf can analyze an ELF object, including those targeting eBPF
  • llvm-objdump can disassemble an ELF object to eBPF bytecode

bpftool

bpftool can be built in tools/bpf of the installed kernel's source.

Subcommand Role
feature examines the running kernel, and enumerates all capabilities present related to eBPF (program types, map types, kernel config options, eBPF helpers, etc.)
prog enumerates attached eBPF programs, loads and attaches programs, and can dump loaded programs' bytecode (or JITted host code)
perf lists kprobes and other tracepoints with attached programs
map enumerates and manipulates maps

perf

Kernel 4.4 added general events to the perf infrastructure. Kernel 4.9 added the ability for eBPF to register perf events. These are the preferred method for streaming events to userspace from an eBPF program.

Compiling eBPF

BCC

The BPF Compiler Collection automates much of the process of turning eBPF source into a kernel object, but much of this (as of 2019-09) requires Python. The BPF object of bcc.py can take raw eBPF text, and return an object which can be easily attached to a variety of eBPF targets.

LLVM

LLVM has enjoyed bpf backend support since 3.7. Compile using -target bpf to generate BPF bytecode, adding -g to generate BTF information.

readelf on the resulting object ought indicate a Machine of "Linux BPF" or "EM_BPF". The resulting object can be loaded into the kernel with bpftool prog load or libbpf's bpf_object__open(). When using bpftool prog load, you must specify a PATH within a mounted bpffs filesystem.

Kernel JIT

eBPF bytecode was designed to have one-to-one correspondences with most instruction sets. The kernel, when configured appropriately, will JIT the bytecode input into host machine code. JIT requires the net.core.bpf_jit_enable sysctl to be set.

See Also