summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* bpf: Make struct cgroup btf id globalYonghong Song2022-10-262-1/+2
| | | | | | | | | | Make struct cgroup btf id global so later patch can reuse the same btf id. Acked-by: David Vernet <void@manifault.com> Signed-off-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/20221026042840.672602-1-yhs@fb.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* Merge branch 'bpf: Avoid unnecessary deadlock detection and failure in task ↵Alexei Starovoitov2022-10-2612-84/+431
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | storage' Martin KaFai Lau says: ==================== From: Martin KaFai Lau <martin.lau@kernel.org> The commit bc235cdb423a ("bpf: Prevent deadlock from recursive bpf_task_storage_[get|delete]") added deadlock detection to avoid a tracing program from recurring on the bpf_task_storage_{get,delete}() helpers. These helpers acquire a spin lock and it will lead to deadlock. It is unnecessary for the bpf_lsm and bpf_iter programs which do not recur. The situation is the same as the existing bpf_pid_task_storage_{lookup,delete}_elem() which are used in the syscall and they also do not have deadlock detection. This set is to add new bpf_task_storage_{get,delete}() helper proto without the deadlock detection. The set also removes the prog->active check from the bpf_lsm and bpf_iter program. Please see the individual patch for details. ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Tracing prog can still do lookup under busy lockMartin KaFai Lau2022-10-262-5/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch modifies the task_ls_recursion test to check that the first bpf_task_storage_get(&map_a, ...) in BPF_PROG(on_update) can still do the lockless lookup even it cannot acquire the percpu busy lock. If the lookup succeeds, it will increment the value by 1 and the value in the task storage map_a will become 200+1=201. After that, BPF_PROG(on_update) tries to delete from map_a and should get -EBUSY because it cannot acquire the percpu busy lock after finding the data. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-10-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Ensure no task storage failure for bpf_lsm.s prog due to ↵Martin KaFai Lau2022-10-262-0/+145
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deadlock detection This patch adds a test to check for deadlock failure in bpf_task_storage_{get,delete} when called by a sleepable bpf_lsm prog. It also checks if the prog_info.recursion_misses is non zero. The test starts with 32 threads and they are affinitized to one cpu. In my qemu setup, with CONFIG_PREEMPT=y, I can reproduce it within one second if it is run without the previous patches of this set. Here is the test error message before adding the no deadlock detection version of the bpf_task_storage_{get,delete}: test_nodeadlock:FAIL:bpf_task_storage_get busy unexpected bpf_task_storage_get busy: actual 2 != expected 0 test_nodeadlock:FAIL:bpf_task_storage_delete busy unexpected bpf_task_storage_delete busy: actual 2 != expected 0 Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-9-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Add new bpf_task_storage_delete proto with no deadlock detectionMartin KaFai Lau2022-10-263-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bpf_lsm and bpf_iter do not recur that will cause a deadlock. The situation is similar to the bpf_pid_task_storage_delete_elem() which is called from the syscall map_delete_elem. It does not need deadlock detection. Otherwise, it will cause unnecessary failure when calling the bpf_task_storage_delete() helper. This patch adds bpf_task_storage_delete proto that does not do deadlock detection. It will be used by bpf_lsm and bpf_iter program. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-8-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: bpf_task_storage_delete_recur does lookup first before the deadlock checkMartin KaFai Lau2022-10-261-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | Similar to the earlier change in bpf_task_storage_get_recur. This patch changes bpf_task_storage_delete_recur such that it does the lookup first. It only returns -EBUSY if it needs to take the spinlock to do the deletion when potential deadlock is detected. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-7-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Add new bpf_task_storage_get proto with no deadlock detectionMartin KaFai Lau2022-10-263-1/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bpf_lsm and bpf_iter do not recur that will cause a deadlock. The situation is similar to the bpf_pid_task_storage_lookup_elem() which is called from the syscall map_lookup_elem. It does not need deadlock detection. Otherwise, it will cause unnecessary failure when calling the bpf_task_storage_get() helper. This patch adds bpf_task_storage_get proto that does not do deadlock detection. It will be used by bpf_lsm and bpf_iter programs. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-6-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Avoid taking spinlock in bpf_task_storage_get if potential deadlock is ↵Martin KaFai Lau2022-10-262-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | detected bpf_task_storage_get() does a lookup and optionally inserts new data if BPF_LOCAL_STORAGE_GET_F_CREATE is present. During lookup, it will cache the lookup result and caching requires to acquire a spinlock. When potential deadlock is detected (by the bpf_task_storage_busy pcpu-counter added in commit bc235cdb423a ("bpf: Prevent deadlock from recursive bpf_task_storage_[get|delete]")), the current behavior is returning NULL immediately to avoid deadlock. It is too pessimistic. This patch will go ahead to do a lookup (which is a lockless operation) but it will avoid caching it in order to avoid acquiring the spinlock. When lookup fails to find the data and BPF_LOCAL_STORAGE_GET_F_CREATE is set, an insertion is needed and this requires acquiring a spinlock. This patch will still return NULL when a potential deadlock is detected. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-5-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Refactor the core bpf_task_storage_get logic into a new functionMartin KaFai Lau2022-10-261-18/+26
| | | | | | | | | | | | | | | | | | | | | | This patch creates a new function __bpf_task_storage_get() and moves the core logic of the existing bpf_task_storage_get() into this new function. This new function will be shared by another new helper proto in the latter patch. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-4-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Append _recur naming to the bpf_task_storage helper protoMartin KaFai Lau2022-10-263-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the "_recur" naming to the bpf_task_storage_{get,delete} proto. In a latter patch, they will only be used by the tracing programs that requires a deadlock detection because a tracing prog may use bpf_task_storage_{get,delete} recursively and cause a deadlock. Another following patch will add a different helper proto for the non tracing programs because they do not need the deadlock prevention. This patch does this rename to prepare for this future proto additions. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-3-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Remove prog->active check for bpf_lsm and bpf_iterMartin KaFai Lau2022-10-266-54/+98
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The commit 64696c40d03c ("bpf: Add __bpf_prog_{enter,exit}_struct_ops for struct_ops trampoline") removed prog->active check for struct_ops prog. The bpf_lsm and bpf_iter is also using trampoline. Like struct_ops, the bpf_lsm and bpf_iter have fixed hooks for the prog to attach. The kernel does not call the same hook in a recursive way. This patch also removes the prog->active check for bpf_lsm and bpf_iter. A later patch has a test to reproduce the recursion issue for a sleepable bpf_lsm program. This patch appends the '_recur' naming to the existing enter and exit functions that track the prog->active counter. New __bpf_prog_{enter,exit}[_sleepable] function are added to skip the prog->active tracking. The '_struct_ops' version is also removed. It also moves the decision on picking the enter and exit function to the new bpf_trampoline_{enter,exit}(). It returns the '_recur' ones for all tracing progs to use. For bpf_lsm, bpf_iter, struct_ops (no prog->active tracking after 64696c40d03c), and bpf_lsm_cgroup (no prog->active tracking after 69fd337a975c7), it will return the functions that don't track the prog->active. Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20221025184524.3526117-2-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* libbpf: Btf dedup identical struct test needs check for nested structs/arraysAlan Maguire2022-10-261-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When examining module BTF, it is common to see core kernel structures such as sk_buff, net_device duplicated in the module. After adding debug messaging to BTF it turned out that much of the problem was down to the identical struct test failing during deduplication; sometimes the compiler adds identical structs. However it turns out sometimes that type ids of identical struct members can also differ, even when the containing structs are still identical. To take an example, for struct sk_buff, debug messaging revealed that the identical struct matching was failing for the anon struct "headers"; specifically for the first field: __u8 __pkt_type_offset[0]; /* 128 0 */ Looking at the code in BTF deduplication, we have code that guards against the possibility of identical struct definitions, down to type ids, and identical array definitions. However in this case we have a struct which is being defined twice but does not have identical type ids since each duplicate struct has separate type ids for the above array member. A similar problem (though not observed) could occur for struct-in-struct. The solution is to make the "identical struct" test check members not just for matching ids, but to also check if they in turn are identical structs or arrays. The results of doing this are quite dramatic (for some modules at least); I see the number of type ids drop from around 10000 to just over 1000 in one module for example. For testing use latest pahole or apply [1], otherwise dedups can fail for the reasons described there. Also fix return type of btf_dedup_identical_arrays() as suggested by Andrii to match boolean return type used elsewhere. Fixes: efdd3eb8015e ("libbpf: Accommodate DWARF/compiler bug with duplicated structs") Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/1666622309-22289-1-git-send-email-alan.maguire@oracle.com [1] https://lore.kernel.org/bpf/1666364523-9648-1-git-send-email-alan.maguire
* Merge branch 'bpf: Fixes for kprobe multi on kernel modules'Alexei Starovoitov2022-10-2511-17/+306
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Jiri Olsa says: ==================== hi, Martynas reported kprobe _multi link does not resolve symbols from kernel modules, which attach by address works. In addition while fixing that I realized we do not take module reference if the module has kprobe_multi link on top of it and can be removed. There's mo crash related to this, it will silently disappear from ftrace tables, while kprobe_multi link stays up with no data. This patchset has fixes for both issues. v3 changes: - reorder fields in struct bpf_kprobe_multi_link [Andrii] - added ack [Andrii] v2 changes: - added acks (Song) - added comment to kallsyms_callback (Song) - change module_callback realloc logic (Andrii) - get rid of macros in tests (Andrii) thanks, jirka ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Add kprobe_multi kmod attach api testsJiri Olsa2022-10-252-0/+139
| | | | | | | | | | | | | | | | | | | | | | | | | | Adding kprobe_multi kmod attach api tests that attach bpf_testmod functions via bpf_program__attach_kprobe_multi_opts. Running it as serial test, because we don't want other tests to reload bpf_testmod while it's running. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-9-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Add kprobe_multi check to module attach testJiri Olsa2022-10-252-0/+13
| | | | | | | | | | | | | | | | | | | | Adding test that makes sure the kernel module won't be removed if there's kprobe multi link defined on top of it. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-8-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Add bpf_testmod_fentry_* functionsJiri Olsa2022-10-251-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding 3 bpf_testmod_fentry_* functions to have a way to test kprobe multi link on kernel module. They follow bpf_fentry_test* functions prototypes/code. Adding equivalent functions to all bpf_fentry_test* does not seems necessary at the moment, could be added later. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-7-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * selftests/bpf: Add load_kallsyms_refresh functionJiri Olsa2022-10-252-7/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding load_kallsyms_refresh function to re-read symbols from /proc/kallsyms file. This will be needed to get proper functions addresses from bpf_testmod.ko module, which is loaded/unloaded several times during the tests run, so symbols might be already old when we need to use them. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-6-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Take module reference on kprobe_multi linkJiri Olsa2022-10-251-0/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we allow to create kprobe multi link on function from kernel module, but we don't take the module reference to ensure it's not unloaded while we are tracing it. The multi kprobe link is based on fprobe/ftrace layer which takes different approach and releases ftrace hooks when module is unloaded even if there's tracer registered on top of it. Adding code that gathers all the related modules for the link and takes their references before it's attached. All kernel module references are released after link is unregistered. Note that we do it the same way already for trampoline probes (but for single address). Acked-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-5-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpf: Rename __bpf_kprobe_multi_cookie_cmp to bpf_kprobe_multi_addrs_cmpJiri Olsa2022-10-251-3/+3
| | | | | | | | | | | | | | | | | | | | Renaming __bpf_kprobe_multi_cookie_cmp to bpf_kprobe_multi_addrs_cmp, because it's more suitable to current and upcoming code. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-4-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * ftrace: Add support to resolve module symbols in ftrace_lookup_symbolsJiri Olsa2022-10-251-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently ftrace_lookup_symbols iterates only over core symbols, adding module_kallsyms_on_each_symbol call to check on modules symbols as well. Also removing 'args.found == args.cnt' condition, because it's already checked in kallsyms_callback function. Also removing 'err < 0' check, because both *kallsyms_on_each_symbol functions do not return error. Reported-by: Martynas Pumputis <m@lambda.lt> Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-3-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * kallsyms: Make module_kallsyms_on_each_symbol generally availableJiri Olsa2022-10-252-2/+9
|/ | | | | | | | | | | | | | | Making module_kallsyms_on_each_symbol generally available, so it can be used outside CONFIG_LIVEPATCH option in following changes. Rather than adding another ifdef option let's make the function generally available (when CONFIG_KALLSYMS and CONFIG_MODULES options are defined). Cc: Christoph Hellwig <hch@lst.de> Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20221025134148.3300700-2-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* Merge branch 'bpftool: Add LLVM as default library for disassembling JIT-ed ↵Alexei Starovoitov2022-10-2512-106/+320
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | programs' Quentin Monnet says: ==================== To disassemble instructions for JIT-ed programs, bpftool has relied on the libbfd library. This has been problematic in the past: libbfd's interface is not meant to be stable and has changed several times, hence the detection of the two related features from the Makefile (disassembler-four-args and disassembler-init-styled). When it comes to shipping bpftool, this has also caused issues with several distribution maintainers unwilling to support the feature (for example, Debian's page for binutils-dev, libbfd's package, says: "Note that building Debian packages which depend on the shared libbfd is Not Allowed."). This patchset adds support for LLVM as the primary library for disassembling instructions for JIT-ed programs. We keep libbfd as a fallback. One reason for this is that currently it works well, we have all we need in terms of features detection in the Makefile, so it provides a fallback for disassembling JIT-ed programs if libbfd is installed but LLVM is not. The other reason is that libbfd supports nfp instruction for Netronome's SmartNICs and can be used to disassemble offloaded programs, something that LLVM cannot do (Niklas confirmed that the feature is still in use). However, if libbfd's interface breaks again in the future, we might reconsider keeping support for it. v4: - Rebase to address a conflict with commit 2c76238eaddd ("bpftool: Add "bootstrap" feature to version output"). v3: - Extend commit description (patch 6) with notes on llvm-dev and LLVM's disassembler stability. v2: - Pass callback when creating the LLVM disassembler, so that the branch targets are printed as addresses (instead of byte offsets). - Add last commit to "support" other arch with LLVM, although we don't know any supported triple yet. - Use $(LLVM_CONFIG) instead of llvm-config in Makefile. - Pass components to llvm-config --libs to limit the number of libraries to pass on the command line, in Makefile. - Rebase split of FEATURE_TESTS and FEATURE_DISPLAY in Makefile. ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Add llvm feature to "bpftool version"Quentin Monnet2022-10-252-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Similarly to "libbfd", add a "llvm" feature to the output of command "bpftool version" to indicate that LLVM is used for disassembling JIT-ed programs. This feature is mutually exclusive (from Makefile definitions) with "libbfd". Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/20221025150329.97371-9-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Support setting alternative arch for JIT disasm with LLVMQuentin Monnet2022-10-254-18/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For offloaded BPF programs, instead of failing to create the LLVM disassembler without even looking for a triple at all, do run the function that attempts to retrieve a valid architecture name for the device. It will still fail for the LLVM disassembler, because currently we have no valid triple to return (NFP disassembly is not supported by LLVM). But failing in that function is more logical than to assume in jit_disasm.c that passing an "arch" name is simply not supported. Suggested-by: Song Liu <song@kernel.org> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/r/20221025150329.97371-8-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Add LLVM as default library for disassembling JIT-ed programsQuentin Monnet2022-10-253-23/+141
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To disassemble instructions for JIT-ed programs, bpftool has relied on the libbfd library. This has been problematic in the past: libbfd's interface is not meant to be stable and has changed several times. For building bpftool, we have to detect how the libbfd version on the system behaves, which is why we have to handle features disassembler-four-args and disassembler-init-styled in the Makefile. When it comes to shipping bpftool, this has also caused issues with several distribution maintainers unwilling to support the feature (see for example Debian's page for binutils-dev, which ships libbfd: "Note that building Debian packages which depend on the shared libbfd is Not Allowed." [0]). For these reasons, we add support for LLVM as an alternative to libbfd for disassembling instructions of JIT-ed programs. Thanks to the preparation work in the previous commits, it's easy to add the library by passing the relevant compilation options in the Makefile, and by adding the functions for setting up the LLVM disassembler in file jit_disasm.c. The LLVM disassembler requires the LLVM development package (usually llvm-dev or llvm-devel). The expectation is that the interface for this disassembler will be more stable. There is a note in LLVM's Developer Policy [1] stating that the stability for the C API is "best effort" and not guaranteed, but at least there is some effort to keep compatibility when possible (which hasn't really been the case for libbfd so far). Furthermore, the Debian page for the related LLVM package does not caution against linking to the lib, as binutils-dev page does. Naturally, the display of disassembled instructions comes with a few minor differences. Here is a sample output with libbfd (already supported before this patch): # bpftool prog dump jited id 56 bpf_prog_6deef7357e7b4530: 0: nopl 0x0(%rax,%rax,1) 5: xchg %ax,%ax 7: push %rbp 8: mov %rsp,%rbp b: push %rbx c: push %r13 e: push %r14 10: mov %rdi,%rbx 13: movzwq 0xb4(%rbx),%r13 1b: xor %r14d,%r14d 1e: or $0x2,%r14d 22: mov $0x1,%eax 27: cmp $0x2,%r14 2b: jne 0x000000000000002f 2d: xor %eax,%eax 2f: pop %r14 31: pop %r13 33: pop %rbx 34: leave 35: ret LLVM supports several variants that we could set when initialising the disassembler, for example with: LLVMSetDisasmOptions(*ctx, LLVMDisassembler_Option_AsmPrinterVariant); but the default printer is used for now. Here is the output with LLVM: # bpftool prog dump jited id 56 bpf_prog_6deef7357e7b4530: 0: nopl (%rax,%rax) 5: nop 7: pushq %rbp 8: movq %rsp, %rbp b: pushq %rbx c: pushq %r13 e: pushq %r14 10: movq %rdi, %rbx 13: movzwq 180(%rbx), %r13 1b: xorl %r14d, %r14d 1e: orl $2, %r14d 22: movl $1, %eax 27: cmpq $2, %r14 2b: jne 0x2f 2d: xorl %eax, %eax 2f: popq %r14 31: popq %r13 33: popq %rbx 34: leave 35: retq The LLVM disassembler comes as the default choice, with libbfd as a fall-back. Of course, we could replace libbfd entirely and avoid supporting two different libraries. One reason for keeping libbfd is that, right now, it works well, we have all we need in terms of features detection in the Makefile, so it provides a fallback for disassembling JIT-ed programs if libbfd is installed but LLVM is not. The other motivation is that libbfd supports nfp instruction for Netronome's SmartNICs and can be used to disassemble offloaded programs, something that LLVM cannot do. If libbfd's interface breaks again in the future, we might reconsider keeping support for it. [0] https://packages.debian.org/buster/binutils-dev [1] https://llvm.org/docs/DeveloperPolicy.html#c-api-changes Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/20221025150329.97371-7-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Refactor disassembler for JIT-ed programsQuentin Monnet2022-10-251-45/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | Refactor disasm_print_insn() to extract the code specific to libbfd and move it to dedicated functions. There is no functional change. This is in preparation for supporting an alternative library for disassembling the instructions. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20221025150329.97371-6-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Group libbfd defs in Makefile, only pass them if we use libbfdQuentin Monnet2022-10-251-12/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bpftool uses libbfd for disassembling JIT-ed programs. But the feature is optional, and the tool can be compiled without libbfd support. The Makefile sets the relevant variables accordingly. It also sets variables related to libbfd's interface, given that it has changed over time. Group all those libbfd-related definitions so that it's easier to understand what we are testing for, and only use variables related to libbfd's interface if we need libbfd in the first place. In addition to make the Makefile clearer, grouping the definitions related to disassembling JIT-ed programs will help support alternatives to libbfd. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20221025150329.97371-5-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Split FEATURE_TESTS/FEATURE_DISPLAY definitions in MakefileQuentin Monnet2022-10-251-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | Make FEATURE_TESTS and FEATURE_DISPLAY easier to read and less likely to be subject to conflicts on updates by having one feature per line. Suggested-by: Andres Freund <andres@anarazel.de> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20221025150329.97371-4-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Remove asserts from JIT disassemblerQuentin Monnet2022-10-254-35/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The JIT disassembler in bpftool is the only components (with the JSON writer) using asserts to check the return values of functions. But it does not do so in a consistent way, and diasm_print_insn() returns no value, although sometimes the operation failed. Remove the asserts, and instead check the return values, print messages on errors, and propagate the error to the caller from prog.c. Remove the inclusion of assert.h from jit_disasm.c, and also from map.c where it is unused. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20221025150329.97371-3-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Define _GNU_SOURCE only onceQuentin Monnet2022-10-257-0/+14
|/ | | | | | | | | | | | | | | | _GNU_SOURCE is defined in several source files for bpftool, but only one of them takes the precaution of checking whether the value is already defined. Add #ifndef for other occurrences too. This is in preparation for the support of disassembling JIT-ed programs with LLVM, with $(llvm-config --cflags) passing -D_GNU_SOURCE as a compilation argument. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Tested-by: Niklas Söderlund <niklas.soderlund@corigine.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20221025150329.97371-2-quentin@isovalent.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: Add write to hashmap to array_map iter testDave Marchevsky2022-10-222-7/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Modify iter prog in existing bpf_iter_bpf_array_map.c, which currently dumps arraymap key/val, to also do a write of (val, key) into a newly-added hashmap. Confirm that the write succeeds as expected by modifying the userspace runner program. Before a change added in an earlier commit - considering PTR_TO_BUF reg a valid input to helpers which expect MAP_{KEY,VAL} - the verifier would've rejected this prog change due to type mismatch. Since using current iter's key/val to access a separate map is a reasonable usecase, let's add support for it. Note that the test prog cannot directly write (val, key) into hashmap via bpf_map_update_elem when both come from iter context because key is marked MEM_RDONLY. This is due to bpf_map_update_elem - and other basic map helpers - taking ARG_PTR_TO_MAP_{KEY,VALUE} w/o MEM_RDONLY type flag. bpf_map_{lookup,update,delete}_elem don't modify their input key/val so it should be possible to tag their args READONLY, but due to the ubiquitous use of these helpers and verifier checks for type == MAP_VALUE, such a change is nontrivial and seems better to address in a followup series. Also fixup some 'goto's in test runner's map checking loop. Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20221020160721.4030492-4-davemarchevsky@fb.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: Add test verifying bpf_ringbuf_reserve retval use in map opsDave Marchevsky2022-10-223-4/+140
| | | | | | | | | | | | | | | | | | | Add a test_ringbuf_map_key test prog, borrowing heavily from extant test_ringbuf.c. The program tries to use the result of bpf_ringbuf_reserve as map_key, which was not possible before previouis commits in this series. The test runner added to prog_tests/ringbuf.c verifies that the program loads and does basic sanity checks to confirm that it runs as expected. Also, refactor test_ringbuf such that runners for existing test_ringbuf and newly-added test_ringbuf_map_key are subtests of 'ringbuf' top-level test. Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20221020160721.4030492-3-davemarchevsky@fb.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* bpf: Consider all mem_types compatible for map_{key,value} argsDave Marchevsky2022-10-221-13/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After the previous patch, which added PTR_TO_MEM | MEM_ALLOC type map_key_value_types, the only difference between map_key_value_types and mem_types sets is PTR_TO_BUF and PTR_TO_MEM, which are in the latter set but not the former. Helpers which expect ARG_PTR_TO_MAP_KEY or ARG_PTR_TO_MAP_VALUE already effectively expect a valid blob of arbitrary memory that isn't necessarily explicitly associated with a map. When validating a PTR_TO_MAP_{KEY,VALUE} arg, the verifier expects meta->map_ptr to have already been set, either by an earlier ARG_CONST_MAP_PTR arg, or custom logic like that in process_timer_func or process_kptr_func. So let's get rid of map_key_value_types and just use mem_types for those args. This has the effect of adding PTR_TO_BUF and PTR_TO_MEM to the set of compatible types for ARG_PTR_TO_MAP_KEY and ARG_PTR_TO_MAP_VALUE. PTR_TO_BUF is used by various bpf_iter implementations to represent a chunk of valid r/w memory in ctx args for iter prog. PTR_TO_MEM is used by networking, tracing, and ringbuf helpers to represent a chunk of valid memory. The PTR_TO_MEM | MEM_ALLOC type added in previous commit is specific to ringbuf helpers. Presence or absence of MEM_ALLOC doesn't change the validity of using PTR_TO_MEM as a map_{key,val} input. Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20221020160721.4030492-2-davemarchevsky@fb.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* bpf: Allow ringbuf memory to be used as map keyDave Marchevsky2022-10-221-0/+1
| | | | | | | | | | | | | | | | | | | | | This patch adds support for the following pattern: struct some_data *data = bpf_ringbuf_reserve(&ringbuf, sizeof(struct some_data, 0)); if (!data) return; bpf_map_lookup_elem(&another_map, &data->some_field); bpf_ringbuf_submit(data); Currently the verifier does not consider bpf_ringbuf_reserve's PTR_TO_MEM | MEM_ALLOC ret type a valid key input to bpf_map_lookup_elem. Since PTR_TO_MEM is by definition a valid region of memory, it is safe to use it as a key for lookups. Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20221020160721.4030492-1-davemarchevsky@fb.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* Merge branch 'Add support for aarch64 to selftests/bpf/vmtest.sh'Andrii Nakryiko2022-10-224-3/+268
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Manu Bretelle says: ==================== This patchset adds initial support for running BPF's vmtest on aarch64 architecture. It includes a `config.aarch64` heavily based on `config.s390x` Makes vmtest.sh handle aarch64 and set QEMU variables to values that works on that arch. Finally, it provides a DENYLIST.aarch64 that takes care of currently broken tests on aarch64 so the vmtest run passes. This was tested by running: LLVM_STRIP=llvm-strip-16 CLANG=clang-16 \ tools/testing/selftests/bpf/vmtest.sh -- \ ./test_progs -d \ \"$(cat tools/testing/selftests/bpf/DENYLIST{,.aarch64} \ | cut -d'#' -f1 \ | sed -e 's/^[[:space:]]*//' \ -e 's/[[:space:]]*$//' \ | tr -s '\n' ','\ )\" on an aarch64 host. ==================== Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
| * selftests/bpf: Initial DENYLIST for aarch64Manu Bretelle2022-10-221-0/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Those tests are currently failing on aarch64, ignore them until they are individually addressed. Using this deny list, vmtest.sh ran successfully using LLVM_STRIP=llvm-strip-16 CLANG=clang-16 \ tools/testing/selftests/bpf/vmtest.sh -- \ ./test_progs -d \ \"$(cat tools/testing/selftests/bpf/DENYLIST{,.aarch64} \ | cut -d'#' -f1 \ | sed -e 's/^[[:space:]]*//' \ -e 's/[[:space:]]*$//' \ | tr -s '\n' ','\ )\" Signed-off-by: Manu Bretelle <chantr4@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20221021210701.728135-5-chantr4@gmail.com
| * selftests/bpf: Update vmtests.sh to support aarch64Manu Bretelle2022-10-221-0/+6
| | | | | | | | | | | | | | | | | | Add handling of aarch64 when setting QEMU options and provide the right path to aarch64 kernel image. Signed-off-by: Manu Bretelle <chantr4@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20221021210701.728135-4-chantr4@gmail.com
| * selftests/bpf: Add config.aarch64Manu Bretelle2022-10-221-0/+181
| | | | | | | | | | | | | | | | | | | | config.aarch64, similarly to config.{s390x,x86_64} is a config enabling building a kernel on aarch64 to be used in bpf's selftests/kernel-patches CI. Signed-off-by: Manu Bretelle <chantr4@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20221021210701.728135-3-chantr4@gmail.com
| * selftests/bpf: Remove entries from config.s390x already present in configManu Bretelle2022-10-221-3/+0
|/ | | | | | | | | | | | | | | | | | | | | | | | | | `config.s390x` had entries already present in `config`. When generating the config used by vmtest, we concatenate the `config` file with the `config.{arch}` one, making those entries duplicated. This patch removes that duplication. Before: $ comm -1 -2 <(sort tools/testing/selftests/bpf/config.s390x) <(sort tools/testing/selftests/bpf/config) CONFIG_MODULE_SIG=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y $ Ater: $ comm -1 -2 <(sort tools/testing/selftests/bpf/config.s390x) <(sort tools/testing/selftests/bpf/config) $ Signed-off-by: Manu Bretelle <chantr4@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20221021210701.728135-2-chantr4@gmail.com
* bpftool: Add "bootstrap" feature to version outputQuentin Monnet2022-10-211-32/+49
| | | | | | | | | | | | | | | | | | | | Along with the version number, "bpftool version" displays a list of features that were selected at compilation time for bpftool. It would be useful to indicate in that list whether a binary is a bootstrap version of bpftool. Given that an increasing number of components rely on bootstrap versions for generating skeletons, this could help understand what a binary is capable of if it has been copied outside of the usual "bootstrap" directory. To detect a bootstrap version, we simply rely on the absence of implementation for the do_prog() function. To do this, we must move the (unchanged) list of commands before do_version(), which in turn requires renaming this "cmds" array to avoid shadowing it with the "cmds" argument in cmd_select(). Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20221020100332.69563-1-quentin@isovalent.com
* bpftool: Set binary name to "bpftool" in help and version outputQuentin Monnet2022-10-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Commands "bpftool help" or "bpftool version" use argv[0] to display the name of the binary. While it is a convenient way to retrieve the string, it does not always produce the most readable output. For example, because of the way bpftool is currently packaged on Ubuntu (using a wrapper script), the command displays the absolute path for the binary: $ bpftool version | head -n 1 /usr/lib/linux-tools/5.15.0-50-generic/bpftool v5.15.60 More generally, there is no apparent reason for keeping the whole path and exact binary name in this output. If the user wants to understand what binary is being called, there are other ways to do so. This commit replaces argv[0] with "bpftool", to simply reflect what the tool is called. This is aligned on what "ip" or "tc" do, for example. As an additional benefit, this seems to help with integration with Meson for packaging [0]. [0] https://github.com/NixOS/nixpkgs/pull/195934 Suggested-by: Vladimír Čunát <vladimir.cunat@nic.cz> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20221020100300.69328-1-quentin@isovalent.com
* libbpf: Avoid allocating reg_name with sscanf in parse_usdt_arg()Xu Kuohai2022-10-211-10/+6
| | | | | | | | | | | | | The reg_name in parse_usdt_arg() is used to hold register name, which is short enough to be held in a 16-byte array, so we could define reg_name as char reg_name[16] to avoid dynamically allocating reg_name with sscanf. Suggested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com> Signed-off-by: Xu Kuohai <xukuohai@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/bpf/20221018145538.2046842-1-xukuohai@huaweicloud.com
* selftests/bpf: fix task_local_storage/exit_creds rcu usageDelyan Kratunov2022-10-212-3/+18
| | | | | | | | | | | | | | | | | | | | | | BPF CI has revealed flakiness in the task_local_storage/exit_creds test. The failure point in CI [1] is that null_ptr_count is equal to 0, which indicates that the program hasn't run yet. This points to the kern_sync_rcu (sys_membarrier -> synchronize_rcu underneath) not waiting sufficiently. Indeed, synchronize_rcu only waits for read-side sections that started before the call. If the program execution starts *during* the synchronize_rcu invocation (due to, say, preemption), the test won't wait long enough. As a speculative fix, make the synchornize_rcu calls in a loop until an explicit run counter has gone up. [1]: https://github.com/kernel-patches/bpf/actions/runs/3268263235/jobs/5374940791 Signed-off-by: Delyan Kratunov <delyank@meta.com> Link: https://lore.kernel.org/r/156d4ef82275a074e8da8f4cffbd01b0c1466493.camel@meta.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* Merge branch 'bpftool: Add autoattach for bpf prog load|loadall'Alexei Starovoitov2022-10-213-4/+88
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Wang Yufen says: ==================== This patchset add "autoattach" optional for "bpftool prog load(_all)" to support one-step load-attach-pin_link. v8 -> v9: fix link leak, and change pathname_concat(specify not just buffer pointer, but also it's size) v7 -> v8: for the programs not supporting autoattach, fall back to reguler pinning instead of skipping v6 -> v7: add info msg print and update doc for the skip program v5 -> v6: skip the programs not supporting auto-attach, and change optional name from "auto_attach" to "autoattach" v4 -> v5: some formatting nits of doc v3 -> v4: rename functions, update doc, bash and do_help() v2 -> v3: switch to extend prog load command instead of extend perf v2: https://patchwork.kernel.org/project/netdevbpf/patch/20220824033837.458197-1-weiyongjun1@huawei.com/ v1: https://patchwork.kernel.org/project/netdevbpf/patch/20220816151725.153343-1-weiyongjun1@huawei.com/ ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Update the bash completion(add autoattach to prog load)Wang Yufen2022-10-211-0/+1
| | | | | | | | | | | | | | | | | | Add autoattach optional to prog load|loadall for supporting one-step load-attach-pin_link. Signed-off-by: Wang Yufen <wangyufen@huawei.com> Link: https://lore.kernel.org/r/1665736275-28143-4-git-send-email-wangyufen@huawei.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Update doc (add autoattach to prog load)Wang Yufen2022-10-211-2/+13
| | | | | | | | | | | | | | | | | | Add autoattach optional to prog load|loadall for supporting one-step load-attach-pin_link. Signed-off-by: Wang Yufen <wangyufen@huawei.com> Link: https://lore.kernel.org/r/1665736275-28143-3-git-send-email-wangyufen@huawei.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * bpftool: Add autoattach for bpf prog load|loadallWang Yufen2022-10-211-2/+74
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add autoattach optional to support one-step load-attach-pin_link. For example, $ bpftool prog loadall test.o /sys/fs/bpf/test autoattach $ bpftool link 26: tracing name test1 tag f0da7d0058c00236 gpl loaded_at 2022-09-09T21:39:49+0800 uid 0 xlated 88B jited 55B memlock 4096B map_ids 3 btf_id 55 28: kprobe name test3 tag 002ef1bef0723833 gpl loaded_at 2022-09-09T21:39:49+0800 uid 0 xlated 88B jited 56B memlock 4096B map_ids 3 btf_id 55 57: tracepoint name oncpu tag 7aa55dfbdcb78941 gpl loaded_at 2022-09-09T21:41:32+0800 uid 0 xlated 456B jited 265B memlock 4096B map_ids 17,13,14,15 btf_id 82 $ bpftool link 1: tracing prog 26 prog_type tracing attach_type trace_fentry 3: perf_event prog 28 10: perf_event prog 57 The autoattach optional can support tracepoints, k(ret)probes, u(ret)probes. Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Wang Yufen <wangyufen@huawei.com> Link: https://lore.kernel.org/r/1665736275-28143-2-git-send-email-wangyufen@huawei.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: fix missing BPF object filesWang Yufen2022-10-219-29/+40
| | | | | | | | | | | | | | After commit afef88e65554 ("selftests/bpf: Store BPF object files with .bpf.o extension"), we should use *.bpf.o instead of *.o. In addition, use the BPF_FILE variable to save the BPF object file name, which can be better identified and modified. Fixes: afef88e65554 ("selftests/bpf: Store BPF object files with .bpf.o extension") Signed-off-by: Wang Yufen <wangyufen@huawei.com> Acked-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/1666235134-562-1-git-send-email-wangyufen@huawei.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* bpf, docs: Reformat BPF maps page to be more readableDonald Hunter2022-10-211-36/+65
| | | | | | | | | | Add a more complete introduction, with links to man pages. Move toctree of map types above usage notes. Format usage notes to improve readability. Signed-off-by: Donald Hunter <donald.hunter@gmail.com> Link: https://lore.kernel.org/r/20221012152715.25073-1-donald.hunter@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* Merge branch 'bpf,x64: Use BMI2 for shifts'Alexei Starovoitov2022-10-202-12/+118
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Jie Meng says: ==================== With baseline x64 instruction set, shift count can only be an immediate or in %cl. The implicit dependency on %cl makes it necessary to shuffle registers around and/or add push/pop operations. BMI2 provides shift instructions that can use any general register as the shift count, saving us instructions and a few bytes in most cases. Suboptimal codegen when %ecx is source and/or destination is also addressed and unnecessary instructions are removed. test_progs: Summary: 267/1340 PASSED, 25 SKIPPED, 0 FAILED test_progs-no_alu32: Summary: 267/1333 PASSED, 26 SKIPPED, 0 FAILED test_verifier: Summary: 1367 PASSED, 636 SKIPPED, 0 FAILED (same result with or without BMI2) test_maps: OK, 0 SKIPPED lib/test_bpf: test_bpf: Summary: 1026 PASSED, 0 FAILED, [1014/1014 JIT'ed] test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED --- v4 -> v5: - More comments regarding instruction encoding v3 -> v4: - Fixed a regression when BMI2 isn't available ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
generated by cgit v1.2.3 (git 2.39.1) at 2025-02-05 05:03:05 +0100