diff options
author | Jakub Kicinski <kuba@kernel.org> | 2022-11-29 02:14:01 +0100 |
---|---|---|
committer | Jakub Kicinski <kuba@kernel.org> | 2022-11-29 04:42:17 +0100 |
commit | d6dc62fca6b6a1e75430618bf6d84cf135f1b3d8 (patch) | |
tree | 2e955738844b6091fb36d509e1ef7aaed3a67392 /Documentation | |
parent | Revert "net: stmmac: use sysfs_streq() instead of strncmp()" (diff) | |
parent | docs/bpf: Add BPF_MAP_TYPE_XSKMAP documentation (diff) | |
download | linux-d6dc62fca6b6a1e75430618bf6d84cf135f1b3d8.tar.xz linux-d6dc62fca6b6a1e75430618bf6d84cf135f1b3d8.zip |
Daniel Borkmann says:
====================
bpf-next 2022-11-25
We've added 101 non-merge commits during the last 11 day(s) which contain
a total of 109 files changed, 8827 insertions(+), 1129 deletions(-).
The main changes are:
1) Support for user defined BPF objects: the use case is to allocate own
objects, build own object hierarchies and use the building blocks to
build own data structures flexibly, for example, linked lists in BPF,
from Kumar Kartikeya Dwivedi.
2) Add bpf_rcu_read_{,un}lock() support for sleepable programs,
from Yonghong Song.
3) Add support storing struct task_struct objects as kptrs in maps,
from David Vernet.
4) Batch of BPF map documentation improvements, from Maryam Tahhan
and Donald Hunter.
5) Improve BPF verifier to propagate nullness information for branches
of register to register comparisons, from Eduard Zingerman.
6) Fix cgroup BPF iter infra to hold reference on the start cgroup,
from Hou Tao.
7) Fix BPF verifier to not mark fentry/fexit program arguments as trusted
given it is not the case for them, from Alexei Starovoitov.
8) Improve BPF verifier's realloc handling to better play along with dynamic
runtime analysis tools like KASAN and friends, from Kees Cook.
9) Remove legacy libbpf mode support from bpftool,
from Sahid Orentino Ferdjaoui.
10) Rework zero-len skb redirection checks to avoid potentially breaking
existing BPF test infra users, from Stanislav Fomichev.
11) Two small refactorings which are independent and have been split out
of the XDP queueing RFC series, from Toke Høiland-Jørgensen.
12) Fix a memory leak in LSM cgroup BPF selftest, from Wang Yufen.
13) Documentation on how to run BPF CI without patch submission,
from Daniel Müller.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
====================
Link: https://lore.kernel.org/r/20221125012450.441-1-daniel@iogearbox.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/bpf/bpf_design_QA.rst | 11 | ||||
-rw-r--r-- | Documentation/bpf/bpf_devel_QA.rst | 27 | ||||
-rw-r--r-- | Documentation/bpf/btf.rst | 7 | ||||
-rw-r--r-- | Documentation/bpf/index.rst | 1 | ||||
-rw-r--r-- | Documentation/bpf/kfuncs.rst | 54 | ||||
-rw-r--r-- | Documentation/bpf/libbpf/index.rst | 3 | ||||
-rw-r--r-- | Documentation/bpf/libbpf/program_types.rst | 203 | ||||
-rw-r--r-- | Documentation/bpf/map_array.rst | 22 | ||||
-rw-r--r-- | Documentation/bpf/map_bloom_filter.rst | 174 | ||||
-rw-r--r-- | Documentation/bpf/map_cpumap.rst | 59 | ||||
-rw-r--r-- | Documentation/bpf/map_devmap.rst | 238 | ||||
-rw-r--r-- | Documentation/bpf/map_hash.rst | 33 | ||||
-rw-r--r-- | Documentation/bpf/map_lpm_trie.rst | 24 | ||||
-rw-r--r-- | Documentation/bpf/map_of_maps.rst | 6 | ||||
-rw-r--r-- | Documentation/bpf/map_queue_stack.rst | 36 | ||||
-rw-r--r-- | Documentation/bpf/map_xskmap.rst | 192 | ||||
-rw-r--r-- | Documentation/bpf/programs.rst | 3 | ||||
-rw-r--r-- | Documentation/bpf/redirect.rst | 81 |
18 files changed, 1107 insertions, 67 deletions
diff --git a/Documentation/bpf/bpf_design_QA.rst b/Documentation/bpf/bpf_design_QA.rst index 17e774d96c5e..cec2371173d7 100644 --- a/Documentation/bpf/bpf_design_QA.rst +++ b/Documentation/bpf/bpf_design_QA.rst @@ -332,13 +332,14 @@ avoid defining types with 'bpf\_' prefix to not be broken in future releases. In other words, no backwards compatibility is guaranteed if one using a type in BTF with 'bpf\_' prefix. -Q: What is the compatibility story for special BPF types in local kptrs? ------------------------------------------------------------------------- -Q: Same as above, but for local kptrs (i.e. pointers to objects allocated using -bpf_obj_new for user defined structures). Will the kernel preserve backwards +Q: What is the compatibility story for special BPF types in allocated objects? +------------------------------------------------------------------------------ +Q: Same as above, but for allocated objects (i.e. objects allocated using +bpf_obj_new for user defined types). Will the kernel preserve backwards compatibility for these features? A: NO. Unlike map value types, there are no stability guarantees for this case. The -whole local kptr API itself is unstable (since it is exposed through kfuncs). +whole API to work with allocated objects and any support for special fields +inside them is unstable (since it is exposed through kfuncs). diff --git a/Documentation/bpf/bpf_devel_QA.rst b/Documentation/bpf/bpf_devel_QA.rst index 761474bd7fe6..03d4993eda6f 100644 --- a/Documentation/bpf/bpf_devel_QA.rst +++ b/Documentation/bpf/bpf_devel_QA.rst @@ -44,6 +44,33 @@ is a guarantee that the reported issue will be overlooked.** Submitting patches ================== +Q: How do I run BPF CI on my changes before sending them out for review? +------------------------------------------------------------------------ +A: BPF CI is GitHub based and hosted at https://github.com/kernel-patches/bpf. +While GitHub also provides a CLI that can be used to accomplish the same +results, here we focus on the UI based workflow. + +The following steps lay out how to start a CI run for your patches: + +- Create a fork of the aforementioned repository in your own account (one time + action) + +- Clone the fork locally, check out a new branch tracking either the bpf-next + or bpf branch, and apply your to-be-tested patches on top of it + +- Push the local branch to your fork and create a pull request against + kernel-patches/bpf's bpf-next_base or bpf_base branch, respectively + +Shortly after the pull request has been created, the CI workflow will run. Note +that capacity is shared with patches submitted upstream being checked and so +depending on utilization the run can take a while to finish. + +Note furthermore that both base branches (bpf-next_base and bpf_base) will be +updated as patches are pushed to the respective upstream branches they track. As +such, your patch set will automatically (be attempted to) be rebased as well. +This behavior can result in a CI run being aborted and restarted with the new +base line. + Q: To which mailing list do I need to submit my BPF patches? ------------------------------------------------------------ A: Please submit your BPF patches to the bpf kernel mailing list: diff --git a/Documentation/bpf/btf.rst b/Documentation/bpf/btf.rst index cf8722f96090..7cd7c5415a99 100644 --- a/Documentation/bpf/btf.rst +++ b/Documentation/bpf/btf.rst @@ -1062,4 +1062,9 @@ format.:: 7. Testing ========== -Kernel bpf selftest `test_btf.c` provides extensive set of BTF-related tests. +The kernel BPF selftest `tools/testing/selftests/bpf/prog_tests/btf.c`_ +provides an extensive set of BTF-related tests. + +.. Links +.. _tools/testing/selftests/bpf/prog_tests/btf.c: + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/tools/testing/selftests/bpf/prog_tests/btf.c diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst index 1b50de1983ee..1088d44634d6 100644 --- a/Documentation/bpf/index.rst +++ b/Documentation/bpf/index.rst @@ -29,6 +29,7 @@ that goes into great technical depth about the BPF Architecture. clang-notes linux-notes other + redirect .. only:: subproject and html diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst index 0f858156371d..90774479ab7a 100644 --- a/Documentation/bpf/kfuncs.rst +++ b/Documentation/bpf/kfuncs.rst @@ -72,6 +72,30 @@ argument as its size. By default, without __sz annotation, the size of the type of the pointer is used. Without __sz annotation, a kfunc cannot accept a void pointer. +2.2.2 __k Annotation +-------------------- + +This annotation is only understood for scalar arguments, where it indicates that +the verifier must check the scalar argument to be a known constant, which does +not indicate a size parameter, and the value of the constant is relevant to the +safety of the program. + +An example is given below:: + + void *bpf_obj_new(u32 local_type_id__k, ...) + { + ... + } + +Here, bpf_obj_new uses local_type_id argument to find out the size of that type +ID in program's BTF and return a sized pointer to it. Each type ID will have a +distinct size, hence it is crucial to treat each such call as distinct when +values don't match during verifier state pruning checks. + +Hence, whenever a constant scalar argument is accepted by a kfunc which is not a +size parameter, and the value of the constant matters for program safety, __k +suffix should be used. + .. _BPF_kfunc_nodef: 2.3 Using an existing kernel function @@ -137,22 +161,20 @@ KF_ACQUIRE and KF_RET_NULL flags. -------------------------- The KF_TRUSTED_ARGS flag is used for kfuncs taking pointer arguments. It -indicates that the all pointer arguments will always have a guaranteed lifetime, -and pointers to kernel objects are always passed to helpers in their unmodified -form (as obtained from acquire kfuncs). - -It can be used to enforce that a pointer to a refcounted object acquired from a -kfunc or BPF helper is passed as an argument to this kfunc without any -modifications (e.g. pointer arithmetic) such that it is trusted and points to -the original object. - -Meanwhile, it is also allowed pass pointers to normal memory to such kfuncs, -but those can have a non-zero offset. - -This flag is often used for kfuncs that operate (change some property, perform -some operation) on an object that was obtained using an acquire kfunc. Such -kfuncs need an unchanged pointer to ensure the integrity of the operation being -performed on the expected object. +indicates that the all pointer arguments are valid, and that all pointers to +BTF objects have been passed in their unmodified form (that is, at a zero +offset, and without having been obtained from walking another pointer). + +There are two types of pointers to kernel objects which are considered "valid": + +1. Pointers which are passed as tracepoint or struct_ops callback arguments. +2. Pointers which were returned from a KF_ACQUIRE or KF_KPTR_GET kfunc. + +Pointers to non-BTF objects (e.g. scalar pointers) may also be passed to +KF_TRUSTED_ARGS kfuncs, and may have a non-zero offset. + +The definition of "valid" pointers is subject to change at any time, and has +absolutely no ABI stability guarantees. 2.4.6 KF_SLEEPABLE flag ----------------------- diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst index 3722537d1384..f9b3b252e28f 100644 --- a/Documentation/bpf/libbpf/index.rst +++ b/Documentation/bpf/libbpf/index.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +.. _libbpf: + libbpf ====== @@ -7,6 +9,7 @@ libbpf :maxdepth: 1 API Documentation <https://libbpf.readthedocs.io/en/latest/api.html> + program_types libbpf_naming_convention libbpf_build diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst new file mode 100644 index 000000000000..ad4d4d5eecb0 --- /dev/null +++ b/Documentation/bpf/libbpf/program_types.rst @@ -0,0 +1,203 @@ +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +.. _program_types_and_elf: + +Program Types and ELF Sections +============================== + +The table below lists the program types, their attach types where relevant and the ELF section +names supported by libbpf for them. The ELF section names follow these rules: + +- ``type`` is an exact match, e.g. ``SEC("socket")`` +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")`` + with a '``/``' separator between ``type`` and ``extras``. + +When ``extras`` are specified, they provide details of how to auto-attach the BPF program. The +format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")`` +for tracepoints or ``SEC("usdt/<path>:<provider>:<name>")`` for USDT probes. The extras are +described in more detail in the footnotes. + + ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| Program Type | Attach Type | ELF Section Name | Sleepable | ++===========================================+========================================+==================================+===========+ +| ``BPF_PROG_TYPE_CGROUP_DEVICE`` | ``BPF_CGROUP_DEVICE`` | ``cgroup/dev`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_CGROUP_SKB`` | | ``cgroup/skb`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET_EGRESS`` | ``cgroup_skb/egress`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET_INGRESS`` | ``cgroup_skb/ingress`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_CGROUP_SOCKOPT`` | ``BPF_CGROUP_GETSOCKOPT`` | ``cgroup/getsockopt`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_SETSOCKOPT`` | ``cgroup/setsockopt`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_CGROUP_SOCK_ADDR`` | ``BPF_CGROUP_INET4_BIND`` | ``cgroup/bind4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET4_CONNECT`` | ``cgroup/connect4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET4_GETPEERNAME`` | ``cgroup/getpeername4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET4_GETSOCKNAME`` | ``cgroup/getsockname4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET6_BIND`` | ``cgroup/bind6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET6_CONNECT`` | ``cgroup/connect6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET6_GETPEERNAME`` | ``cgroup/getpeername6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET6_GETSOCKNAME`` | ``cgroup/getsockname6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_UDP4_RECVMSG`` | ``cgroup/recvmsg4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_UDP4_SENDMSG`` | ``cgroup/sendmsg4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_UDP6_RECVMSG`` | ``cgroup/recvmsg6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_UDP6_SENDMSG`` | ``cgroup/sendmsg6`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_CGROUP_SOCK`` | ``BPF_CGROUP_INET4_POST_BIND`` | ``cgroup/post_bind4`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET6_POST_BIND`` | ``cgroup/post_bind6`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET_SOCK_CREATE`` | ``cgroup/sock_create`` | | ++ + +----------------------------------+-----------+ +| | | ``cgroup/sock`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_CGROUP_INET_SOCK_RELEASE`` | ``cgroup/sock_release`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_CGROUP_SYSCTL`` | ``BPF_CGROUP_SYSCTL`` | ``cgroup/sysctl`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_EXT`` | | ``freplace+`` [#fentry]_ | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_FLOW_DISSECTOR`` | ``BPF_FLOW_DISSECTOR`` | ``flow_dissector`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_KPROBE`` | | ``kprobe+`` [#kprobe]_ | | ++ + +----------------------------------+-----------+ +| | | ``kretprobe+`` [#kprobe]_ | | ++ + +----------------------------------+-----------+ +| | | ``ksyscall+`` [#ksyscall]_ | | ++ + +----------------------------------+-----------+ +| | | ``kretsyscall+`` [#ksyscall]_ | | ++ + +----------------------------------+-----------+ +| | | ``uprobe+`` [#uprobe]_ | | ++ + +----------------------------------+-----------+ +| | | ``uprobe.s+`` [#uprobe]_ | Yes | ++ + +----------------------------------+-----------+ +| | | ``uretprobe+`` [#uprobe]_ | | ++ + +----------------------------------+-----------+ +| | | ``uretprobe.s+`` [#uprobe]_ | Yes | ++ + +----------------------------------+-----------+ +| | | ``usdt+`` [#usdt]_ | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_TRACE_KPROBE_MULTI`` | ``kprobe.multi+`` [#kpmulti]_ | | ++ + +----------------------------------+-----------+ +| | | ``kretprobe.multi+`` [#kpmulti]_ | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LIRC_MODE2`` | ``BPF_LIRC_MODE2`` | ``lirc_mode2`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LSM`` | ``BPF_LSM_CGROUP`` | ``lsm_cgroup+`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_LSM_MAC`` | ``lsm+`` [#lsm]_ | | ++ + +----------------------------------+-----------+ +| | | ``lsm.s+`` [#lsm]_ | Yes | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LWT_IN`` | | ``lwt_in`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LWT_OUT`` | | ``lwt_out`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LWT_SEG6LOCAL`` | | ``lwt_seg6local`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_LWT_XMIT`` | | ``lwt_xmit`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_PERF_EVENT`` | | ``perf_event`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE`` | | ``raw_tp.w+`` [#rawtp]_ | | ++ + +----------------------------------+-----------+ +| | | ``raw_tracepoint.w+`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_RAW_TRACEPOINT`` | | ``raw_tp+`` [#rawtp]_ | | ++ + +----------------------------------+-----------+ +| | | ``raw_tracepoint+`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SCHED_ACT`` | | ``action`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SCHED_CLS`` | | ``classifier`` | | ++ + +----------------------------------+-----------+ +| | | ``tc`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SK_LOOKUP`` | ``BPF_SK_LOOKUP`` | ``sk_lookup`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SK_MSG`` | ``BPF_SK_MSG_VERDICT`` | ``sk_msg`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SK_REUSEPORT`` | ``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE`` | ``sk_reuseport/migrate`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_SK_REUSEPORT_SELECT`` | ``sk_reuseport`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SK_SKB`` | | ``sk_skb`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_SK_SKB_STREAM_PARSER`` | ``sk_skb/stream_parser`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_SK_SKB_STREAM_VERDICT`` | ``sk_skb/stream_verdict`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SOCKET_FILTER`` | | ``socket`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SOCK_OPS`` | ``BPF_CGROUP_SOCK_OPS`` | ``sockops`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_STRUCT_OPS`` | | ``struct_ops+`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_SYSCALL`` | | ``syscall`` | Yes | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_TRACEPOINT`` | | ``tp+`` [#tp]_ | | ++ + +----------------------------------+-----------+ +| | | ``tracepoint+`` [#tp]_ | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_TRACING`` | ``BPF_MODIFY_RETURN`` | ``fmod_ret+`` [#fentry]_ | | ++ + +----------------------------------+-----------+ +| | | ``fmod_ret.s+`` [#fentry]_ | Yes | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_TRACE_FENTRY`` | ``fentry+`` [#fentry]_ | | ++ + +----------------------------------+-----------+ +| | | ``fentry.s+`` [#fentry]_ | Yes | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_TRACE_FEXIT`` | ``fexit+`` [#fentry]_ | | ++ + +----------------------------------+-----------+ +| | | ``fexit.s+`` [#fentry]_ | Yes | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_TRACE_ITER`` | ``iter+`` [#iter]_ | | ++ + +----------------------------------+-----------+ +| | | ``iter.s+`` [#iter]_ | Yes | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_TRACE_RAW_TP`` | ``tp_btf+`` [#fentry]_ | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ +| ``BPF_PROG_TYPE_XDP`` | ``BPF_XDP_CPUMAP`` | ``xdp.frags/cpumap`` | | ++ + +----------------------------------+-----------+ +| | | ``xdp/cpumap`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_XDP_DEVMAP`` | ``xdp.frags/devmap`` | | ++ + +----------------------------------+-----------+ +| | | ``xdp/devmap`` | | ++ +----------------------------------------+----------------------------------+-----------+ +| | ``BPF_XDP`` | ``xdp.frags`` | | ++ + +----------------------------------+-----------+ +| | | ``xdp`` | | ++-------------------------------------------+----------------------------------------+----------------------------------+-----------+ + + +.. rubric:: Footnotes + +.. [#fentry] The ``fentry`` attach format is ``fentry[.s]/<function>``. +.. [#kprobe] The ``kprobe`` attach format is ``kprobe/<function>[+<offset>]``. Valid + characters for ``function`` are ``a-zA-Z0-9_.`` and ``offset`` must be a valid + non-negative integer. +.. [#ksyscall] The ``ksyscall`` attach format is ``ksyscall/<syscall>``. +.. [#uprobe] The ``uprobe`` attach format is ``uprobe[.s]/<path>:<function>[+<offset>]``. +.. [#usdt] The ``usdt`` attach format is ``usdt/<path>:<provider>:<name>``. +.. [#kpmulti] The ``kprobe.multi`` attach format is ``kprobe.multi/<pattern>`` where ``pattern`` + supports ``*`` and ``?`` wildcards. Valid characters for pattern are + ``a-zA-Z0-9_.*?``. +.. [#lsm] The ``lsm`` attachment format is ``lsm[.s]/<hook>``. +.. [#rawtp] The ``raw_tp`` attach format is ``raw_tracepoint[.w]/<tracepoint>``. +.. [#tp] The ``tracepoint`` attach format is ``tracepoint/<category>/<name>``. +.. [#iter] The ``iter`` attach format is ``iter[.s]/<struct-name>``. diff --git a/Documentation/bpf/map_array.rst b/Documentation/bpf/map_array.rst index b2cceb6c696b..f2f51a53e8ae 100644 --- a/Documentation/bpf/map_array.rst +++ b/Documentation/bpf/map_array.rst @@ -32,7 +32,11 @@ Usage Kernel BPF ---------- -.. c:function:: +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) Array elements can be retrieved using the ``bpf_map_lookup_elem()`` helper. @@ -40,7 +44,11 @@ This helper returns a pointer into the array element, so to avoid data races with userspace reading the value, the user must use primitives like ``__sync_fetch_and_add()`` when updating the value in-place. -.. c:function:: +bpf_map_update_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags) Array elements can be updated using the ``bpf_map_update_elem()`` helper. @@ -53,7 +61,7 @@ To clear an array element, you may use ``bpf_map_update_elem()`` to insert a zero value to that index. Per CPU Array -~~~~~~~~~~~~~ +------------- Values stored in ``BPF_MAP_TYPE_ARRAY`` can be accessed by multiple programs across different CPUs. To restrict storage to a single CPU, you may use a @@ -63,7 +71,11 @@ When using a ``BPF_MAP_TYPE_PERCPU_ARRAY`` the ``bpf_map_update_elem()`` and ``bpf_map_lookup_elem()`` helpers automatically access the slot for the current CPU. -.. c:function:: +bpf_map_lookup_percpu_elem() +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_percpu_elem(struct bpf_map *map, const void *key, u32 cpu) The ``bpf_map_lookup_percpu_elem()`` helper can be used to lookup the array @@ -119,7 +131,7 @@ This example BPF program shows how to access an array element. index = ip.protocol; value = bpf_map_lookup_elem(&my_map, &index); if (value) - __sync_fetch_and_add(&value, skb->len); + __sync_fetch_and_add(value, skb->len); return 0; } diff --git a/Documentation/bpf/map_bloom_filter.rst b/Documentation/bpf/map_bloom_filter.rst new file mode 100644 index 000000000000..c82487f2fe0d --- /dev/null +++ b/Documentation/bpf/map_bloom_filter.rst @@ -0,0 +1,174 @@ +.. SPDX-License-Identifier: GPL-2.0-only +.. Copyright (C) 2022 Red Hat, Inc. + +========================= +BPF_MAP_TYPE_BLOOM_FILTER +========================= + +.. note:: + - ``BPF_MAP_TYPE_BLOOM_FILTER`` was introduced in kernel version 5.16 + +``BPF_MAP_TYPE_BLOOM_FILTER`` provides a BPF bloom filter map. Bloom +filters are a space-efficient probabilistic data structure used to +quickly test whether an element exists in a set. In a bloom filter, +false positives are possible whereas false negatives are not. + +The bloom filter map does not have keys, only values. When the bloom +filter map is created, it must be created with a ``key_size`` of 0. The +bloom filter map supports two operations: + +- push: adding an element to the map +- peek: determining whether an element is present in the map + +BPF programs must use ``bpf_map_push_elem`` to add an element to the +bloom filter map and ``bpf_map_peek_elem`` to query the map. These +operations are exposed to userspace applications using the existing +``bpf`` syscall in the following way: + +- ``BPF_MAP_UPDATE_ELEM`` -> push +- ``BPF_MAP_LOOKUP_ELEM`` -> peek + +The ``max_entries`` size that is specified at map creation time is used +to approximate a reasonable bitmap size for the bloom filter, and is not +otherwise strictly enforced. If the user wishes to insert more entries +into the bloom filter than ``max_entries``, this may lead to a higher +false positive rate. + +The number of hashes to use for the bloom filter is configurable using +the lower 4 bits of ``map_extra`` in ``union bpf_attr`` at map creation +time. If no number is specified, the default used will be 5 hash +functions. In general, using more hashes decreases both the false +positive rate and the speed of a lookup. + +It is not possible to delete elements from a bloom filter map. A bloom +filter map may be used as an inner map. The user is responsible for +synchronising concurrent updates and lookups to ensure no false negative +lookups occur. + +Usage +===== + +Kernel BPF +---------- + +bpf_map_push_elem() +~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + long bpf_map_push_elem(struct bpf_map *map, const void *value, u64 flags) + +A ``value`` can be added to a bloom filter using the +``bpf_map_push_elem()`` helper. The ``flags`` parameter must be set to +``BPF_ANY`` when adding an entry to the bloom filter. This helper +returns ``0`` on success, or negative error in case of failure. + +bpf_map_peek_elem() +~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + long bpf_map_peek_elem(struct bpf_map *map, void *value) + +The ``bpf_map_peek_elem()`` helper is used to determine whether +``value`` is present in the bloom filter map. This helper returns ``0`` +if ``value`` is probably present in the map, or ``-ENOENT`` if ``value`` +is definitely not present in the map. + +Userspace +--------- + +bpf_map_update_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + int bpf_map_update_elem (int fd, const void *key, const void *value, __u64 flags) + +A userspace program can add a ``value`` to a bloom filter using libbpf's +``bpf_map_update_elem`` function. The ``key`` parameter must be set to +``NULL`` and ``flags`` must be set to ``BPF_ANY``. Returns ``0`` on +success, or negative error in case of failure. + +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + int bpf_map_lookup_elem (int fd, const void *key, void *value) + +A userspace program can determine the presence of ``value`` in a bloom +filter using libbpf's ``bpf_map_lookup_elem`` function. The ``key`` +parameter must be set to ``NULL``. Returns ``0`` if ``value`` is +probably present in the map, or ``-ENOENT`` if ``value`` is definitely +not present in the map. + +Examples +======== + +Kernel BPF +---------- + +This snippet shows how to declare a bloom filter in a BPF program: + +.. code-block:: c + + struct { + __uint(type, BPF_MAP_TYPE_BLOOM_FILTER); + __type(value, __u32); + __uint(max_entries, 1000); + __uint(map_extra, 3); + } bloom_filter SEC(".maps"); + +This snippet shows how to determine presence of a value in a bloom +filter in a BPF program: + +.. code-block:: c + + void *lookup(__u32 key) + { + if (bpf_map_peek_elem(&bloom_filter, &key) == 0) { + /* Verify not a false positive and fetch an associated + * value using a secondary lookup, e.g. in a hash table + */ + return bpf_map_lookup_elem(&hash_table, &key); + } + return 0; + } + +Userspace +--------- + +This snippet shows how to use libbpf to create a bloom filter map from +userspace: + +.. code-block:: c + + int create_bloom() + { + LIBBPF_OPTS(bpf_map_create_opts, opts, + .map_extra = 3); /* number of hashes */ + + return bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, + "ipv6_bloom", /* name */ + 0, /* key size, must be zero */ + sizeof(ipv6_addr), /* value size */ + 10000, /* max entries */ + &opts); /* create options */ + } + +This snippet shows how to add an element to a bloom filter from +userspace: + +.. code-block:: c + + int add_element(struct bpf_map *bloom_map, __u32 value) + { + int bloom_fd = bpf_map__fd(bloom_map); + return bpf_map_update_elem(bloom_fd, NULL, &value, BPF_ANY); + } + +References +========== + +https://lwn.net/ml/bpf/20210831225005.2762202-1-joannekoong@fb.com/ diff --git a/Documentation/bpf/map_cpumap.rst b/Documentation/bpf/map_cpumap.rst index eaf57b38cafd..923cfc8ab51f 100644 --- a/Documentation/bpf/map_cpumap.rst +++ b/Documentation/bpf/map_cpumap.rst @@ -30,30 +30,35 @@ Usage Kernel BPF ---------- -.. c:function:: +bpf_redirect_map() +^^^^^^^^^^^^^^^^^^ +.. code-block:: c + long bpf_redirect_map(struct bpf_map *map, u32 key, u64 flags) - Redirect the packet to the endpoint referenced by ``map`` at index ``key``. - For ``BPF_MAP_TYPE_CPUMAP`` this map contains references to CPUs. +Redirect the packet to the endpoint referenced by ``map`` at index ``key``. +For ``BPF_MAP_TYPE_CPUMAP`` this map contains references to CPUs. - The lower two bits of ``flags`` are used as the return code if the map lookup - fails. This is so that the return value can be one of the XDP program return - codes up to ``XDP_TX``, as chosen by the caller. +The lower two bits of ``flags`` are used as the return code if the map lookup +fails. This is so that the return value can be one of the XDP program return +codes up to ``XDP_TX``, as chosen by the caller. -Userspace ---------- +User space +---------- .. note:: CPUMAP entries can only be updated/looked up/deleted from user space and not from an eBPF program. Trying to call these functions from a kernel eBPF program will result in the program failing to load and a verifier warning. -.. c:function:: - int bpf_map_update_elem(int fd, const void *key, const void *value, - __u64 flags); +bpf_map_update_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags); - CPU entries can be added or updated using the ``bpf_map_update_elem()`` - helper. This helper replaces existing elements atomically. The ``value`` parameter - can be ``struct bpf_cpumap_val``. +CPU entries can be added or updated using the ``bpf_map_update_elem()`` +helper. This helper replaces existing elements atomically. The ``value`` parameter +can be ``struct bpf_cpumap_val``. .. code-block:: c @@ -65,23 +70,29 @@ Userspace } bpf_prog; }; - The flags argument can be one of the following: +The flags argument can be one of the following: - BPF_ANY: Create a new element or update an existing element. - BPF_NOEXIST: Create a new element only if it did not exist. - BPF_EXIST: Update an existing element. -.. c:function:: +bpf_map_lookup_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + int bpf_map_lookup_elem(int fd, const void *key, void *value); - CPU entries can be retrieved using the ``bpf_map_lookup_elem()`` - helper. +CPU entries can be retrieved using the ``bpf_map_lookup_elem()`` +helper. + +bpf_map_delete_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c -.. c:function:: int bpf_map_delete_elem(int fd, const void *key); - CPU entries can be deleted using the ``bpf_map_delete_elem()`` - helper. This helper will return 0 on success, or negative error in case of - failure. +CPU entries can be deleted using the ``bpf_map_delete_elem()`` +helper. This helper will return 0 on success, or negative error in case of +failure. Examples ======== @@ -142,8 +153,8 @@ The following code snippet shows how to declare a ``BPF_MAP_TYPE_CPUMAP`` called return bpf_redirect_map(&cpu_map, cpu_dest, 0); } -Userspace ---------- +User space +---------- The following code snippet shows how to dynamically set the max_entries for a CPUMAP to the max number of cpus available on the system. diff --git a/Documentation/bpf/map_devmap.rst b/Documentation/bpf/map_devmap.rst new file mode 100644 index 000000000000..927312c7b8c8 --- /dev/null +++ b/Documentation/bpf/map_devmap.rst @@ -0,0 +1,238 @@ +.. SPDX-License-Identifier: GPL-2.0-only +.. Copyright (C) 2022 Red Hat, Inc. + +================================================= +BPF_MAP_TYPE_DEVMAP and BPF_MAP_TYPE_DEVMAP_HASH +================================================= + +.. note:: + - ``BPF_MAP_TYPE_DEVMAP`` was introduced in kernel version 4.14 + - ``BPF_MAP_TYPE_DEVMAP_HASH`` was introduced in kernel version 5.4 + +``BPF_MAP_TYPE_DEVMAP`` and ``BPF_MAP_TYPE_DEVMAP_HASH`` are BPF maps primarily +used as backend maps for the XDP BPF helper call ``bpf_redirect_map()``. +``BPF_MAP_TYPE_DEVMAP`` is backed by an array that uses the key as +the index to lookup a reference to a net device. While ``BPF_MAP_TYPE_DEVMAP_HASH`` +is backed by a hash table that uses a key to lookup a reference to a net device. +The user provides either <``key``/ ``ifindex``> or <``key``/ ``struct bpf_devmap_val``> +pairs to update the maps with new net devices. + +.. note:: + - The key to a hash map doesn't have to be an ``ifindex``. + - While ``BPF_MAP_TYPE_DEVMAP_HASH`` allows for densely packing the net devices + it comes at the cost of a hash of the key when performing a look up. + +The setup and packet enqueue/send code is shared between the two types of +devmap; only the lookup and insertion is different. + +Usage +===== +Kernel BPF +---------- +bpf_redirect_map() +^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + long bpf_redirect_map(struct bpf_map *map, u32 key, u64 flags) + +Redirect the packet to the endpoint referenced by ``map`` at index ``key``. +For ``BPF_MAP_TYPE_DEVMAP`` and ``BPF_MAP_TYPE_DEVMAP_HASH`` this map contains +references to net devices (for forwarding packets through other ports). + +The lower two bits of *flags* are used as the return code if the map lookup +fails. This is so that the return value can be one of the XDP program return +codes up to ``XDP_TX``, as chosen by the caller. The higher bits of ``flags`` +can be set to ``BPF_F_BROADCAST`` or ``BPF_F_EXCLUDE_INGRESS`` as defined +below. + +With ``BPF_F_BROADCAST`` the packet will be broadcast to all the interfaces +in the map, with ``BPF_F_EXCLUDE_INGRESS`` the ingress interface will be excluded +from the broadcast. + +.. note:: + - The key is ignored if BPF_F_BROADCAST is set. + - The broadcast feature can also be used to implement multicast forwarding: + simply create multiple DEVMAPs, each one corresponding to a single multicast group. + +This helper will return ``XDP_REDIRECT`` on success, or the value of the two +lower bits of the ``flags`` argument if the map lookup fails. + +More information about redirection can be found :doc:`redirect` + +bpf_map_lookup_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) + +Net device entries can be retrieved using the ``bpf_map_lookup_elem()`` +helper. + +User space +---------- +.. note:: + DEVMAP entries can only be updated/deleted from user space and not + from an eBPF program. Trying to call these functions from a kernel eBPF + program will result in the program failing to load and a verifier warning. + +bpf_map_update_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags); + +Net device entries can be added or updated using the ``bpf_map_update_elem()`` +helper. This helper replaces existing elements atomically. The ``value`` parameter +can be ``struct bpf_devmap_val`` or a simple ``int ifindex`` for backwards +compatibility. + + .. code-block:: c + + struct bpf_devmap_val { + __u32 ifindex; /* device index */ + union { + int fd; /* prog fd on map write */ + __u32 id; /* prog id on map read */ + } bpf_prog; + }; + +The ``flags`` argument can be one of the following: + - ``BPF_ANY``: Create a new element or update an existing element. + - ``BPF_NOEXIST``: Create a new element only if it did not exist. + - ``BPF_EXIST``: Update an existing element. + +DEVMAPs can associate a program with a device entry by adding a ``bpf_prog.fd`` +to ``struct bpf_devmap_val``. Programs are run after ``XDP_REDIRECT`` and have +access to both Rx device and Tx device. The program associated with the ``fd`` +must have type XDP with expected attach type ``xdp_devmap``. +When a program is associated with a device index, the program is run on an +``XDP_REDIRECT`` and before the buffer is added to the per-cpu queue. Examples +of how to attach/use xdp_devmap progs can be found in the kernel selftests: + +- ``tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c`` +- ``tools/testing/selftests/bpf/progs/test_xdp_with_devmap_helpers.c`` + +bpf_map_lookup_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + +.. c:function:: + int bpf_map_lookup_elem(int fd, const void *key, void *value); + +Net device entries can be retrieved using the ``bpf_map_lookup_elem()`` +helper. + +bpf_map_delete_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + +.. c:function:: + int bpf_map_delete_elem(int fd, const void *key); + +Net device entries can be deleted using the ``bpf_map_delete_elem()`` +helper. This helper will return 0 on success, or negative error in case of +failure. + +Examples +======== + +Kernel BPF +---------- + +The following code snippet shows how to declare a ``BPF_MAP_TYPE_DEVMAP`` +called tx_port. + +.. code-block:: c + + struct { + __uint(type, BPF_MAP_TYPE_DEVMAP); + __type(key, __u32); + __type(value, __u32); + __uint(max_entries, 256); + } tx_port SEC(".maps"); + +The following code snippet shows how to declare a ``BPF_MAP_TYPE_DEVMAP_HASH`` +called forward_map. + +.. code-block:: c + + struct { + __uint(type, BPF_MAP_TYPE_DEVMAP_HASH); + __type(key, __u32); + __type(value, struct bpf_devmap_val); + __uint(max_entries, 32); + } forward_map SEC(".maps"); + +.. note:: + + The value type in the DEVMAP above is a ``struct bpf_devmap_val`` + +The following code snippet shows a simple xdp_redirect_map program. This program +would work with a user space program that populates the devmap ``forward_map`` based +on ingress ifindexes. The BPF program (below) is redirecting packets using the +ingress ``ifindex`` as the ``key``. + +.. code-block:: c + + SEC("xdp") + int xdp_redirect_map_func(struct xdp_md *ctx) + { + int index = ctx->ingress_ifindex; + + return bpf_redirect_map(&forward_map, index, 0); + } + +The following code snippet shows a BPF program that is broadcasting packets to +all the interfaces in the ``tx_port`` devmap. + +.. code-block:: c + + SEC("xdp") + int xdp_redirect_map_func(struct xdp_md *ctx) + { + return bpf_redirect_map(&tx_port, 0, BPF_F_BROADCAST | BPF_F_EXCLUDE_INGRESS); + } + +User space +---------- + +The following code snippet shows how to update a devmap called ``tx_port``. + +.. code-block:: c + + int update_devmap(int ifindex, int redirect_ifindex) + { + int ret; + + ret = bpf_map_update_elem(bpf_map__fd(tx_port), &ifindex, &redirect_ifindex, 0); + if (ret < 0) { + fprintf(stderr, "Failed to update devmap_ value: %s\n", + strerror(errno)); + } + + return ret; + } + +The following code snippet shows how to update a hash_devmap called ``forward_map``. + +.. code-block:: c + + int update_devmap(int ifindex, int redirect_ifindex) + { + struct bpf_devmap_val devmap_val = { .ifindex = redirect_ifindex }; + int ret; + + ret = bpf_map_update_elem(bpf_map__fd(forward_map), &ifindex, &devmap_val, 0); + if (ret < 0) { + fprintf(stderr, "Failed to update devmap_ value: %s\n", + strerror(errno)); + } + return ret; + } + +References +=========== + +- https://lwn.net/Articles/728146/ +- https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=6f9d451ab1a33728adb72d7ff66a7b374d665176 +- https://elixir.bootlin.com/linux/latest/source/net/core/filter.c#L4106 diff --git a/Documentation/bpf/map_hash.rst b/Documentation/bpf/map_hash.rst index e85120878b27..8669426264c6 100644 --- a/Documentation/bpf/map_hash.rst +++ b/Documentation/bpf/map_hash.rst @@ -34,7 +34,14 @@ the ``BPF_F_NO_COMMON_LRU`` flag when calling ``bpf_map_create``. Usage ===== -.. c:function:: +Kernel BPF +---------- + +bpf_map_update_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags) Hash entries can be added or updated using the ``bpf_map_update_elem()`` @@ -49,14 +56,22 @@ parameter can be used to control the update behaviour: ``bpf_map_update_elem()`` returns 0 on success, or negative error in case of failure. -.. c:function:: +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) Hash entries can be retrieved using the ``bpf_map_lookup_elem()`` helper. This helper returns a pointer to the value associated with ``key``, or ``NULL`` if no entry was found. -.. c:function:: +bpf_map_delete_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_delete_elem(struct bpf_map *map, const void *key) Hash entries can be deleted using the ``bpf_map_delete_elem()`` @@ -70,7 +85,11 @@ For ``BPF_MAP_TYPE_PERCPU_HASH`` and ``BPF_MAP_TYPE_LRU_PERCPU_HASH`` the ``bpf_map_update_elem()`` and ``bpf_map_lookup_elem()`` helpers automatically access the hash slot for the current CPU. -.. c:function:: +bpf_map_lookup_percpu_elem() +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_percpu_elem(struct bpf_map *map, const void *key, u32 cpu) The ``bpf_map_lookup_percpu_elem()`` helper can be used to lookup the @@ -89,7 +108,11 @@ See ``tools/testing/selftests/bpf/progs/test_spin_lock.c``. Userspace --------- -.. c:function:: +bpf_map_get_next_key() +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + int bpf_map_get_next_key(int fd, const void *cur_key, void *next_key) In userspace, it is possible to iterate through the keys of a hash using diff --git a/Documentation/bpf/map_lpm_trie.rst b/Documentation/bpf/map_lpm_trie.rst index 31be1aa7ba2c..74d64a30f500 100644 --- a/Documentation/bpf/map_lpm_trie.rst +++ b/Documentation/bpf/map_lpm_trie.rst @@ -35,7 +35,11 @@ Usage Kernel BPF ---------- -.. c:function:: +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) The longest prefix entry for a given data value can be found using the @@ -48,7 +52,11 @@ performing longest prefix lookups. For example, when searching for the longest prefix match for an IPv4 address, ``prefixlen`` should be set to ``32``. -.. c:function:: +bpf_map_update_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags) Prefix entries can be added or updated using the ``bpf_map_update_elem()`` @@ -61,7 +69,11 @@ case of failure. The flags parameter must be one of BPF_ANY, BPF_NOEXIST or BPF_EXIST, but the value is ignored, giving BPF_ANY semantics. -.. c:function:: +bpf_map_delete_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_delete_elem(struct bpf_map *map, const void *key) Prefix entries can be deleted using the ``bpf_map_delete_elem()`` @@ -74,7 +86,11 @@ Userspace Access from userspace uses libbpf APIs with the same names as above, with the map identified by ``fd``. -.. c:function:: +bpf_map_get_next_key() +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + int bpf_map_get_next_key (int fd, const void *cur_key, void *next_key) A userspace program can iterate through the entries in an LPM trie using diff --git a/Documentation/bpf/map_of_maps.rst b/Documentation/bpf/map_of_maps.rst index 07212b9227a9..7b5617c2d017 100644 --- a/Documentation/bpf/map_of_maps.rst +++ b/Documentation/bpf/map_of_maps.rst @@ -45,7 +45,11 @@ Usage Kernel BPF Helper ----------------- -.. c:function:: +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) Inner maps can be retrieved using the ``bpf_map_lookup_elem()`` helper. This diff --git a/Documentation/bpf/map_queue_stack.rst b/Documentation/bpf/map_queue_stack.rst index f20e31a647b9..8d14ed49d6e1 100644 --- a/Documentation/bpf/map_queue_stack.rst +++ b/Documentation/bpf/map_queue_stack.rst @@ -28,7 +28,11 @@ Usage Kernel BPF ---------- -.. c:function:: +bpf_map_push_elem() +~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_push_elem(struct bpf_map *map, const void *value, u64 flags) An element ``value`` can be added to a queue or stack using the @@ -38,14 +42,22 @@ when the queue or stack is full, the oldest element will be removed to make room for ``value`` to be added. Returns ``0`` on success, or negative error in case of failure. -.. c:function:: +bpf_map_peek_elem() +~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_peek_elem(struct bpf_map *map, void *value) This helper fetches an element ``value`` from a queue or stack without removing it. Returns ``0`` on success, or negative error in case of failure. -.. c:function:: +bpf_map_pop_elem() +~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + long bpf_map_pop_elem(struct bpf_map *map, void *value) This helper removes an element into ``value`` from a queue or @@ -55,7 +67,11 @@ stack. Returns ``0`` on success, or negative error in case of failure. Userspace --------- -.. c:function:: +bpf_map_update_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + int bpf_map_update_elem (int fd, const void *key, const void *value, __u64 flags) A userspace program can push ``value`` onto a queue or stack using libbpf's @@ -64,7 +80,11 @@ A userspace program can push ``value`` onto a queue or stack using libbpf's same semantics as the ``bpf_map_push_elem`` kernel helper. Returns ``0`` on success, or negative error in case of failure. -.. c:function:: +bpf_map_lookup_elem() +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + int bpf_map_lookup_elem (int fd, const void *key, void *value) A userspace program can peek at the ``value`` at the head of a queue or stack @@ -72,7 +92,11 @@ using the libbpf ``bpf_map_lookup_elem`` function. The ``key`` parameter must be set to ``NULL``. Returns ``0`` on success, or negative error in case of failure. -.. c:function:: +bpf_map_lookup_and_delete_elem() +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + int bpf_map_lookup_and_delete_elem (int fd, const void *key, void *value) A userspace program can pop a ``value`` from the head of a queue or stack using diff --git a/Documentation/bpf/map_xskmap.rst b/Documentation/bpf/map_xskmap.rst new file mode 100644 index 000000000000..7093b8208451 --- /dev/null +++ b/Documentation/bpf/map_xskmap.rst @@ -0,0 +1,192 @@ +.. SPDX-License-Identifier: GPL-2.0-only +.. Copyright (C) 2022 Red Hat, Inc. + +=================== +BPF_MAP_TYPE_XSKMAP +=================== + +.. note:: + - ``BPF_MAP_TYPE_XSKMAP`` was introduced in kernel version 4.18 + +The ``BPF_MAP_TYPE_XSKMAP`` is used as a backend map for XDP BPF helper +call ``bpf_redirect_map()`` and ``XDP_REDIRECT`` action, like 'devmap' and 'cpumap'. +This map type redirects raw XDP frames to `AF_XDP`_ sockets (XSKs), a new type of +address family in the kernel that allows redirection of frames from a driver to +user space without having to traverse the full network stack. An AF_XDP socket +binds to a single netdev queue. A mapping of XSKs to queues is shown below: + +.. code-block:: none + + +---------------------------------------------------+ + | xsk A | xsk B | xsk C |<---+ User space + =========================================================|========== + | Queue 0 | Queue 1 | Queue 2 | | Kernel + +---------------------------------------------------+ | + | Netdev eth0 | | + +---------------------------------------------------+ | + | +=============+ | | + | | key | xsk | | | + | +---------+ +=============+ | | + | | | | 0 | xsk A | | | + | | | +-------------+ | | + | | | | 1 | xsk B | | | + | | BPF |-- redirect -->+-------------+-------------+ + | | prog | | 2 | xsk C | | + | | | +-------------+ | + | | | | + | | | | + | +---------+ | + | | + +---------------------------------------------------+ + +.. note:: + An AF_XDP socket that is bound to a certain <netdev/queue_id> will *only* + accept XDP frames from that <netdev/queue_id>. If an XDP program tries to redirect + from a <netdev/queue_id> other than what the socket is bound to, the frame will + not be received on the socket. + +Typically an XSKMAP is created per netdev. This map contains an array of XSK File +Descriptors (FDs). The number of array elements is typically set or adjusted using +the ``max_entries`` map parameter. For AF_XDP ``max_entries`` is equal to the number +of queues supported by the netdev. + +.. note:: + Both the map key and map value size must be 4 bytes. + +Usage +===== + +Kernel BPF +---------- +bpf_redirect_map() +^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + long bpf_redirect_map(struct bpf_map *map, u32 key, u64 flags) + +Redirect the packet to the endpoint referenced by ``map`` at index ``key``. +For ``BPF_MAP_TYPE_XSKMAP`` this map contains references to XSK FDs +for sockets attached to a netdev's queues. + +.. note:: + If the map is empty at an index, the packet is dropped. This means that it is + necessary to have an XDP program loaded with at least one XSK in the + XSKMAP to be able to get any traffic to user space through the socket. + +bpf_map_lookup_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) + +XSK entry references of type ``struct xdp_sock *`` can be retrieved using the +``bpf_map_lookup_elem()`` helper. + +User space +---------- +.. note:: + XSK entries can only be updated/deleted from user space and not from + a BPF program. Trying to call these functions from a kernel BPF program will + result in the program failing to load and a verifier warning. + +bpf_map_update_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags) + +XSK entries can be added or updated using the ``bpf_map_update_elem()`` +helper. The ``key`` parameter is equal to the queue_id of the queue the XSK +is attaching to. And the ``value`` parameter is the FD value of that socket. + +Under the hood, the XSKMAP update function uses the XSK FD value to retrieve the +associated ``struct xdp_sock`` instance. + +The flags argument can be one of the following: + +- BPF_ANY: Create a new element or update an existing element. +- BPF_NOEXIST: Create a new element only if it did not exist. +- BPF_EXIST: Update an existing element. + +bpf_map_lookup_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int bpf_map_lookup_elem(int fd, const void *key, void *value) + +Returns ``struct xdp_sock *`` or negative error in case of failure. + +bpf_map_delete_elem() +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int bpf_map_delete_elem(int fd, const void *key) + +XSK entries can be deleted using the ``bpf_map_delete_elem()`` +helper. This helper will return 0 on success, or negative error in case of +failure. + +.. note:: + When `libxdp`_ deletes an XSK it also removes the associated socket + entry from the XSKMAP. + +Examples +======== +Kernel +------ + +The following code snippet shows how to declare a ``BPF_MAP_TYPE_XSKMAP`` called +``xsks_map`` and how to redirect packets to an XSK. + +.. code-block:: c + + struct { + __uint(type, BPF_MAP_TYPE_XSKMAP); + __type(key, __u32); + __type(value, __u32); + __uint(max_entries, 64); + } xsks_map SEC(".maps"); + + + SEC("xdp") + int xsk_redir_prog(struct xdp_md *ctx) + { + __u32 index = ctx->rx_queue_index; + + if (bpf_map_lookup_elem(&xsks_map, &index)) + return bpf_redirect_map(&xsks_map, index, 0); + return XDP_PASS; + } + +User space +---------- + +The following code snippet shows how to update an XSKMAP with an XSK entry. + +.. code-block:: c + + int update_xsks_map(struct bpf_map *xsks_map, int queue_id, int xsk_fd) + { + int ret; + + ret = bpf_map_update_elem(bpf_map__fd(xsks_map), &queue_id, &xsk_fd, 0); + if (ret < 0) + fprintf(stderr, "Failed to update xsks_map: %s\n", strerror(errno)); + + return ret; + } + +For an example on how create AF_XDP sockets, please see the AF_XDP-example and +AF_XDP-forwarding programs in the `bpf-examples`_ directory in the `libxdp`_ repository. +For a detailed explaination of the AF_XDP interface please see: + +- `libxdp-readme`_. +- `AF_XDP`_ kernel documentation. + +.. note:: + The most comprehensive resource for using XSKMAPs and AF_XDP is `libxdp`_. + +.. _libxdp: https://github.com/xdp-project/xdp-tools/tree/master/lib/libxdp +.. _AF_XDP: https://www.kernel.org/doc/html/latest/networking/af_xdp.html +.. _bpf-examples: https://github.com/xdp-project/bpf-examples +.. _libxdp-readme: https://github.com/xdp-project/xdp-tools/tree/master/lib/libxdp#using-af_xdp-sockets diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst index 620eb667ac7a..c99000ab6d9b 100644 --- a/Documentation/bpf/programs.rst +++ b/Documentation/bpf/programs.rst @@ -7,3 +7,6 @@ Program Types :glob: prog_* + +For a list of all program types, see :ref:`program_types_and_elf` in +the :ref:`libbpf` documentation. diff --git a/Documentation/bpf/redirect.rst b/Documentation/bpf/redirect.rst new file mode 100644 index 000000000000..2fa2b0b05004 --- /dev/null +++ b/Documentation/bpf/redirect.rst @@ -0,0 +1,81 @@ +.. SPDX-License-Identifier: GPL-2.0-only +.. Copyright (C) 2022 Red Hat, Inc. + +======== +Redirect +======== +XDP_REDIRECT +############ +Supported maps +-------------- + +XDP_REDIRECT works with the following map types: + +- ``BPF_MAP_TYPE_DEVMAP`` +- ``BPF_MAP_TYPE_DEVMAP_HASH`` +- ``BPF_MAP_TYPE_CPUMAP`` +- ``BPF_MAP_TYPE_XSKMAP`` + +For more information on these maps, please see the specific map documentation. + +Process +------- + +.. kernel-doc:: net/core/filter.c + :doc: xdp redirect + +.. note:: + Not all drivers support transmitting frames after a redirect, and for + those that do, not all of them support non-linear frames. Non-linear xdp + bufs/frames are bufs/frames that contain more than one fragment. + +Debugging packet drops +---------------------- +Silent packet drops for XDP_REDIRECT can be debugged using: + +- bpf_trace +- perf_record + +bpf_trace +^^^^^^^^^ +The following bpftrace command can be used to capture and count all XDP tracepoints: + +.. code-block:: none + + sudo bpftrace -e 'tracepoint:xdp:* { @cnt[probe] = count(); }' + Attaching 12 probes... + ^C + + @cnt[tracepoint:xdp:mem_connect]: 18 + @cnt[tracepoint:xdp:mem_disconnect]: 18 + @cnt[tracepoint:xdp:xdp_exception]: 19605 + @cnt[tracepoint:xdp:xdp_devmap_xmit]: 1393604 + @cnt[tracepoint:xdp:xdp_redirect]: 22292200 + +.. note:: + The various xdp tracepoints can be found in ``source/include/trace/events/xdp.h`` + +The following bpftrace command can be used to extract the ``ERRNO`` being returned as +part of the err parameter: + +.. code-block:: none + + sudo bpftrace -e \ + 'tracepoint:xdp:xdp_redirect*_err {@redir_errno[-args->err] = count();} + tracepoint:xdp:xdp_devmap_xmit {@devmap_errno[-args->err] = count();}' + +perf record +^^^^^^^^^^^ +The perf tool also supports recording tracepoints: + +.. code-block:: none + + perf record -a -e xdp:xdp_redirect_err \ + -e xdp:xdp_redirect_map_err \ + -e xdp:xdp_exception \ + -e xdp:xdp_devmap_xmit + +References +=========== + +- https://github.com/xdp-project/xdp-tutorial/tree/master/tracing02-xdp-monitor |