summaryrefslogtreecommitdiffstats
path: root/tools/lib/bpf (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge tag 'perf-tools-for-v6.12-1-2024-09-19' of ↵Linus Torvalds2024-09-222-3/+11
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools Pull perf tools updates from Arnaldo Carvalho de Melo: - Use BPF + BTF to collect and pretty print syscall and tracepoint arguments in 'perf trace', done as an GSoC activity - Data-type profiling improvements: - Cache debuginfo to speed up data type resolution - Add the 'typecln' sort order, to show which cacheline in a target is hot or cold. The following shows members in the cfs_rq's first cache line: $ perf report -s type,typecln,typeoff -H ... - 2.67% struct cfs_rq + 1.23% struct cfs_rq: cache-line 2 + 0.57% struct cfs_rq: cache-line 4 + 0.46% struct cfs_rq: cache-line 6 - 0.41% struct cfs_rq: cache-line 0 0.39% struct cfs_rq +0x14 (h_nr_running) 0.02% struct cfs_rq +0x38 (tasks_timeline.rb_leftmost) - When a typedef resolves to a unnamed struct, use the typedef name - When a struct has just one basic type field (int, etc), resolve the type sort order to the name of the struct, not the type of the field - Support type folding/unfolding in the data-type annotation TUI - Fix bitfields offsets and sizes - Initial support for PowerPC, using libcapstone and the usual objdump disassembly parsing routines - Add support for disassembling and addr2line using the LLVM libraries, speeding up those operations - Support --addr2line option in 'perf script' as with other tools - Intel branch counters (LBR event logging) support, only available in recent Intel processors, for instance, the new "brcntr" field can be asked from 'perf script' to print the information collected from this feature: $ perf script -F +brstackinsn,+brcntr # Branch counter abbr list: # branch-instructions:ppp = A # branch-misses = B # '-' No event occurs # '+' Event occurrences may be lost due to branch counter saturated tchain_edit 332203 3366329.405674: 53030 branch-instructions:ppp: 401781 f3+0x2c (home/sdp/test/tchain_edit) f3+31: 0000000000401774 insn: eb 04 br_cntr: AA # PRED 5 cycles [5] 000000000040177a insn: 81 7d fc 0f 27 00 00 0000000000401781 insn: 7e e3 br_cntr: A # PRED 1 cycles [6] 2.00 IPC 0000000000401766 insn: 8b 45 fc 0000000000401769 insn: 83 e0 01 000000000040176c insn: 85 c0 000000000040176e insn: 74 06 br_cntr: A # PRED 1 cycles [7] 4.00 IPC 0000000000401776 insn: 83 45 fc 01 000000000040177a insn: 81 7d fc 0f 27 00 00 0000000000401781 insn: 7e e3 br_cntr: A # PRED 7 cycles [14] 0.43 IPC - Support Timed PEBS (Precise Event-Based Sampling), a recent hardware feature in Intel processors - Add 'perf ftrace profile' subcommand, using ftrace's function-graph tracer so that users can see the total, average, max execution time as well as the number of invocations easily, for instance: $ sudo perf ftrace profile -G __x64_sys_perf_event_open -- \ perf stat -e cycles -C1 true 2> /dev/null | head # Total (us) Avg (us) Max (us) Count Function 65.611 65.611 65.611 1 __x64_sys_perf_event_open 30.527 30.527 30.527 1 anon_inode_getfile 30.260 30.260 30.260 1 __anon_inode_getfile 29.700 29.700 29.700 1 alloc_file_pseudo 17.578 17.578 17.578 1 d_alloc_pseudo 17.382 17.382 17.382 1 __d_alloc 16.738 16.738 16.738 1 kmem_cache_alloc_lru 15.686 15.686 15.686 1 perf_event_alloc 14.012 7.006 11.264 2 obj_cgroup_charge - 'perf sched timehist' improvements, including the addition of priority showing/filtering command line options - Varios improvements to the 'perf probe', including 'perf test' regression testings - Introduce the 'perf check', initially to check if some feature is in place, using it in 'perf test' - Various fixes for 32-bit systems - Address more leak sanitizer failures - Fix memory leaks (LBR, disasm lock ops, etc) - More reference counting fixes (branch_info, etc) - Constify 'struct perf_tool' parameters to improve code generation and reduce the chances of having its internals changed, which isn't expected - More constifications in various other places - Add more build tests, including for JEVENTS - Add more 'perf test' entries ('perf record LBR', pipe/inject, --setup-filter, 'perf ftrace', 'cgroup sampling', etc) - Inject build ids for all entries in a call chain in 'perf inject', not just for the main sample - Improve the BPF based sample filter, allowing root to setup filters in bpffs that then can be used by non-root users - Allow filtering by cgroups with the BPF based sample filter - Allow a more compact way for 'perf mem report' using the -T/--type-profile and also provide a --sort option similar to the one in 'perf report', 'perf top', to setup the sort order manually - Fix --group behavior in 'perf annotate' when leader has no samples, where it was not showing anything even when other events in the group had samples - Fix spinlock and rwlock accounting in 'perf lock contention' - Fix libsubcmd fixdep Makefile dependencies - Improve 'perf ftrace' error message when ftrace isn't available - Update various Intel JSON vendor event files - ARM64 CoreSight hardware tracing infrastructure improvements, mostly not visible to users - Update power10 JSON events * tag 'perf-tools-for-v6.12-1-2024-09-19' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools: (310 commits) perf trace: Mark the 'head' arg in the set_robust_list syscall as coming from user space perf trace: Mark the 'rseq' arg in the rseq syscall as coming from user space perf env: Find correct branch counter info on hybrid perf evlist: Print hint for group tools: Drop nonsensical -O6 perf pmu: To info add event_type_desc perf evsel: Add accessor for tool_event perf pmus: Fake PMU clean up perf list: Avoid potential out of bounds memory read perf help: Fix a typo ("bellow") perf ftrace: Detect whether ftrace is enabled on system perf test shell probe_vfs_getname: Remove extraneous '=' from probe line number regex perf build: Require at least clang 16.0.6 to build BPF skeletons perf trace: If a syscall arg is marked as 'const', assume it is coming _from_ userspace perf parse-events: Remove duplicated include in parse-events.c perf callchain: Allow symbols to be optional when resolving a callchain perf inject: Lazy build-id mmap2 event insertion perf inject: Add new mmap2-buildid-all option perf inject: Fix build ID injection perf annotate-data: Add pr_debug_scope() ...
| * tools build: Correct bpf fixdep dependenciesBrian Norris2024-08-052-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The dependencies in tools/lib/bpf/Makefile are incorrect. Before we recurse to build $(BPF_IN_STATIC), we need to build its 'fixdep' executable. I can't use the usual shortcut from Makefile.include: <target>: <sources> fixdep because its 'fixdep' target relies on $(OUTPUT), and $(OUTPUT) differs in the parent 'make' versus the child 'make' -- so I imitate it via open-coding. I tweak a few $(MAKE) invocations while I'm at it, because 1. I'm adding a new recursive make; and 2. these recursive 'make's print spurious lines about files that are "up to date" (which isn't normally a feature in Kbuild subtargets) or "jobserver not available" (see [1]) I also need to tweak the assignment of the OUTPUT variable, so that relative path builds work. For example, for 'make tools/lib/bpf', OUTPUT is unset, and is usually treated as "cwd" -- but recursive make will change cwd and so OUTPUT has a new meaning. For consistency, I ensure OUTPUT is always an absolute path. And $(Q) gets a backup definition in tools/build/Makefile.include, because Makefile.include is sometimes included without tools/build/Makefile, so the "quiet command" stuff doesn't actually work consistently without it. After this change, top-level builds result in an empty grep result from: $ grep 'cannot find fixdep' $(find tools/ -name '*.cmd') [1] https://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html If we're not using $(MAKE) directly, then we need to use more '+'. Signed-off-by: Brian Norris <briannorris@chromium.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Cc: Ian Rogers <irogers@google.com> Cc: Josh Poimboeuf <jpoimboe@kernel.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Richter <tmricht@linux.ibm.com> Link: https://lore.kernel.org/r/20240715203325.3832977-4-briannorris@chromium.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
* | libbpf: Add bpf_object__token_fd accessorIhor Solodrai2024-09-133-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | Add a LIBBPF_API function to retrieve the token_fd from a bpf_object. Without this accessor, if user needs a token FD they have to get it manually via bpf_token_create, even though a token might have been already created by bpf_object__load. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Ihor Solodrai <ihor.solodrai@pm.me> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240913001858.3345583-1-ihor.solodrai@pm.me
* | libbpf: Fix uretprobe.multi.s programs auto attachmentJiri Olsa2024-09-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As reported by Andrii we don't currently recognize uretprobe.multi.s programs as return probes due to using (wrong) strcmp function. Using str_has_pfx() instead to match uretprobe.multi prefix. Tests are passing, because the return program was executed as entry program and all counts were incremented properly. Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240910125336.3056271-1-jolsa@kernel.org
* | libbpf: Fix some typos in commentsYusheng Zheng2024-09-108-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix some spelling errors in the code comments of libbpf: betwen -> between paremeters -> parameters knowning -> knowing definiton -> definition compatiblity -> compatibility overriden -> overridden occured -> occurred proccess -> process managment -> management nessary -> necessary Signed-off-by: Yusheng Zheng <yunwei356@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240909225952.30324-1-yunwei356@gmail.com
* | libbpf: Fixed getting wrong return address on arm64 architectureShuyi Cheng2024-09-101-1/+1
| | | | | | | | | | | | | | | | | | | | ARM64 has a separate lr register to store the return address, so here you only need to read the lr register to get the return address, no need to dereference it again. Signed-off-by: Shuyi Cheng <chengshuyi@linux.alibaba.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/1725787433-77262-1-git-send-email-chengshuyi@linux.alibaba.com
* | libbpf: Workaround (another) -Wmaybe-uninitialized false positiveSam James2024-09-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We get this with GCC 15 -O3 (at least): ``` libbpf.c: In function ‘bpf_map__init_kern_struct_ops’: libbpf.c:1109:18: error: ‘mod_btf’ may be used uninitialized [-Werror=maybe-uninitialized] 1109 | kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux; | ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libbpf.c:1094:28: note: ‘mod_btf’ was declared here 1094 | struct module_btf *mod_btf; | ^~~~~~~ In function ‘find_struct_ops_kern_types’, inlined from ‘bpf_map__init_kern_struct_ops’ at libbpf.c:1102:8: libbpf.c:982:21: error: ‘btf’ may be used uninitialized [-Werror=maybe-uninitialized] 982 | kern_type = btf__type_by_id(btf, kern_type_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libbpf.c: In function ‘bpf_map__init_kern_struct_ops’: libbpf.c:967:21: note: ‘btf’ was declared here 967 | struct btf *btf; | ^~~ ``` This is similar to the other libbpf fix from a few weeks ago for the same modelling-errno issue (fab45b962749184e1a1a57c7c583782b78fad539). Signed-off-by: Sam James <sam@gentoo.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://bugs.gentoo.org/939106 Link: https://lore.kernel.org/bpf/f6962729197ae7cdf4f6d1512625bd92f2322d31.1725630494.git.sam@gentoo.org
* | libbpf: fix some typos in libbpfLin Yikai2024-09-056-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hi, fix some spelling errors in libbpf, the details are as follows: -in the code comments: termintaing->terminating architecutre->architecture requring->requiring recored->recoded sanitise->sanities allowd->allowed abover->above see bpf_udst_arg()->see bpf_usdt_arg() Signed-off-by: Lin Yikai <yikai.lin@vivo.com> Link: https://lore.kernel.org/r/20240905110354.3274546-3-yikai.lin@vivo.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* | libbpf: Fix accessing first syscall argument on RV64Pu Lehui2024-09-051-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | On RV64, as Ilya mentioned before [0], the first syscall parameter should be accessed through orig_a0 (see arch/riscv64/include/asm/syscall.h), otherwise it will cause selftests like bpf_syscall_macro, vmlinux, test_lsm, etc. to fail on RV64. Let's fix it by using the struct pt_regs style CO-RE direct access. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220209021745.2215452-1-iii@linux.ibm.com [0] Link: https://lore.kernel.org/bpf/20240831041934.1629216-5-pulehui@huaweicloud.com
* | libbpf: Access first syscall argument with CO-RE direct read on arm64Pu Lehui2024-09-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently PT_REGS_PARM1 SYSCALL(x) is consistent with PT_REGS_PARM1_CORE SYSCALL(x), which will introduce the overhead of BPF_CORE_READ(), taking into account the read pt_regs comes directly from the context, let's use CO-RE direct read to access the first system call argument. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Xu Kuohai <xukuohai@huawei.com> Link: https://lore.kernel.org/bpf/20240831041934.1629216-3-pulehui@huaweicloud.com
* | libbpf: Access first syscall argument with CO-RE direct read on s390Pu Lehui2024-09-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Currently PT_REGS_PARM1 SYSCALL(x) is consistent with PT_REGS_PARM1_CORE SYSCALL(x), which will introduce the overhead of BPF_CORE_READ(), taking into account the read pt_regs comes directly from the context, let's use CO-RE direct read to access the first system call argument. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240831041934.1629216-2-pulehui@huaweicloud.com
* | libbpf: Ensure new BTF objects inherit input endiannessTony Ambardar2024-08-301-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New split BTF needs to preserve base's endianness. Similarly, when creating a distilled BTF, we need to preserve original endianness. Fix by updating libbpf's btf__distill_base() and btf_new_empty() to retain the byte order of any source BTF objects when creating new ones. Fixes: ba451366bf44 ("libbpf: Implement basic split BTF support") Fixes: 58e185a0dc35 ("libbpf: Add btf__distill_base() creating split BTF with distilled base BTF") Reported-by: Song Liu <song@kernel.org> Reported-by: Eduard Zingerman <eddyz87@gmail.com> Suggested-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Tony Ambardar <tony.ambardar@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/6358db36c5f68b07873a0a5be2d062b1af5ea5f8.camel@gmail.com/ Link: https://lore.kernel.org/bpf/20240830095150.278881-1-tony.ambardar@gmail.com
* | libbpf: Fix bpf_object__open_skeleton()'s mishandling of optionsAndrii Nakryiko2024-08-291-33/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We do an ugly copying of options in bpf_object__open_skeleton() just to be able to set object name from skeleton's recorded name (while still allowing user to override it through opts->object_name). This is not just ugly, but it also is broken due to memcpy() that doesn't take into account potential skel_opts' and user-provided opts' sizes differences due to backward and forward compatibility. This leads to copying over extra bytes and then failing to validate options properly. It could, technically, lead also to SIGSEGV, if we are unlucky. So just get rid of that memory copy completely and instead pass default object name into bpf_object_open() directly, simplifying all this significantly. The rule now is that obj_name should be non-NULL for bpf_object_open() when called with in-memory buffer, so validate that explicitly as well. We adopt bpf_object__open_mem() to this as well and generate default name (based on buffer memory address and size) outside of bpf_object_open(). Fixes: d66562fba1ce ("libbpf: Add BPF object skeleton support") Reported-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Daniel Müller <deso@posteo.net> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240827203721.1145494-1-andrii@kernel.org
* | libbpf: Workaround -Wmaybe-uninitialized false positiveSam James2024-08-161-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In `elf_close`, we get this with GCC 15 -O3 (at least): ``` In function ‘elf_close’, inlined from ‘elf_close’ at elf.c:53:6, inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2: elf.c:57:9: warning: ‘elf_fd.elf’ may be used uninitialized [-Wmaybe-uninitialized] 57 | elf_end(elf_fd->elf); | ^~~~~~~~~~~~~~~~~~~~ elf.c: In function ‘elf_find_func_offset_from_file’: elf.c:377:23: note: ‘elf_fd.elf’ was declared here 377 | struct elf_fd elf_fd; | ^~~~~~ In function ‘elf_close’, inlined from ‘elf_close’ at elf.c:53:6, inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2: elf.c:58:9: warning: ‘elf_fd.fd’ may be used uninitialized [-Wmaybe-uninitialized] 58 | close(elf_fd->fd); | ^~~~~~~~~~~~~~~~~ elf.c: In function ‘elf_find_func_offset_from_file’: elf.c:377:23: note: ‘elf_fd.fd’ was declared here 377 | struct elf_fd elf_fd; | ^~~~~~ ``` In reality, our use is fine, it's just that GCC doesn't model errno here (see linked GCC bug). Suppress -Wmaybe-uninitialized accordingly by initializing elf_fd.fd to -1 and elf_fd.elf to NULL. I've done this in two other functions as well given it could easily occur there too (same access/use pattern). Signed-off-by: Sam James <sam@gentoo.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://gcc.gnu.org/PR114952 Link: https://lore.kernel.org/bpf/14ec488a1cac02794c2fa2b83ae0cef1bce2cb36.1723578546.git.sam@gentoo.org
* | libbpf: Fix license for btf_relocate.cAlan Maguire2024-08-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | License should be // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) ...as with other libbpf files. Fixes: 19e00c897d50 ("libbpf: Split BTF relocation") Reported-by: Neill Kapron <nkapron@google.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/bpf/20240810093504.2111134-1-alan.maguire@oracle.com
* | libbpf: Don't take direct pointers into BTF data from st_opsDavid Vernet2024-07-301-10/+13
|/ | | | | | | | | | | | | | | | | | | | | | In struct bpf_struct_ops, we have take a pointer to a BTF type name, and a struct btf_type. This was presumably done for convenience, but can actually result in subtle and confusing bugs given that BTF data can be invalidated before a program is loaded. For example, in sched_ext, we may sometimes resize a data section after a skeleton has been opened, but before the struct_ops scheduler map has been loaded. This may cause the BTF data to be realloc'd, which can then cause a UAF when loading the program because the struct_ops map has pointers directly into the BTF data. We're already storing the BTF type_id in struct bpf_struct_ops. Because type_id is stable, we can therefore just update the places where we were looking at those pointers to instead do the lookups we need from the type_id. Fixes: 590a00888250 ("bpf: libbpf: Add STRUCT_OPS support") Signed-off-by: David Vernet <void@manifault.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240724171459.281234-1-void@manifault.com
* libbpf: Fix no-args func prototype BTF dumping syntaxAndrii Nakryiko2024-07-171-3/+5
| | | | | | | | | | | | | | | | | | | | | | For all these years libbpf's BTF dumper has been emitting not strictly valid syntax for function prototypes that have no input arguments. Instead of `int (*blah)()` we should emit `int (*blah)(void)`. This is not normally a problem, but it manifests when we get kfuncs in vmlinux.h that have no input arguments. Due to compiler internal specifics, we get no BTF information for such kfuncs, if they are not declared with proper `(void)`. The fix is trivial. We also need to adjust a few ancient tests that happily assumed `()` is correct. Fixes: 351131b51c7a ("libbpf: add btf_dump API for BTF-to-C conversion") Reported-by: Tejun Heo <tj@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Stanislav Fomichev <sdf@fomichev.me> Link: https://lore.kernel.org/bpf/20240712224442.282823-1-andrii@kernel.org
* libbpf: improve old BPF skeleton handling for map auto-attachAndrii Nakryiko2024-07-101-12/+14
| | | | | | | | | | | | | | | | | Improve how we handle old BPF skeletons when it comes to BPF map auto-attachment. Emit one warn-level message per each struct_ops map that could have been auto-attached, if user provided recent enough BPF skeleton version. Don't spam log if there are no relevant struct_ops maps, though. This should help users realize that they probably need to regenerate BPF skeleton header with more recent bpftool/libbpf-cargo (or whatever other means of BPF skeleton generation). Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20240708204540.4188946-4-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* libbpf: fix BPF skeleton forward/backward compat handlingAndrii Nakryiko2024-07-101-20/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BPF skeleton was designed from day one to be extensible. Generated BPF skeleton code specifies actual sizes of map/prog/variable skeletons for that reason and libbpf is supposed to work with newer/older versions correctly. Unfortunately, it was missed that we implicitly embed hard-coded most up-to-date (according to libbpf's version of libbpf.h header used to compile BPF skeleton header) sizes of those structs, which can differ from the actual sizes at runtime when libbpf is used as a shared library. We have a few places were we just index array of maps/progs/vars, which implicitly uses these potentially invalid sizes of structs. This patch aims to fix this problem going forward. Once this lands, we'll backport these changes in Github repo to create patched releases for older libbpfs. Acked-by: Eduard Zingerman <eddyz87@gmail.com> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Fixes: d66562fba1ce ("libbpf: Add BPF object skeleton support") Fixes: 430025e5dca5 ("libbpf: Add subskeleton scaffolding") Fixes: 08ac454e258e ("libbpf: Auto-attach struct_ops BPF maps in BPF skeleton") Co-developed-by: Mykyta Yatsenko <yatsenko@meta.com> Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240708204540.4188946-3-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* libbpf: Add NULL checks to bpf_object__{prev_map,next_map}Andreas Ziegler2024-07-081-2/+2
| | | | | | | | | | | | | | | | | | | | | | | In the current state, an erroneous call to bpf_object__find_map_by_name(NULL, ...) leads to a segmentation fault through the following call chain: bpf_object__find_map_by_name(obj = NULL, ...) -> bpf_object__for_each_map(pos, obj = NULL) -> bpf_object__next_map((obj = NULL), NULL) -> return (obj = NULL)->maps While calling bpf_object__find_map_by_name with obj = NULL is obviously incorrect, this should not lead to a segmentation fault but rather be handled gracefully. As __bpf_map__iter already handles this situation correctly, we can delegate the check for the regular case there and only add a check in case the prev or next parameter is NULL. Signed-off-by: Andreas Ziegler <ziegler.andreas@siemens.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20240703083436.505124-1-ziegler.andreas@siemens.com
* libbpf: Fix error handling in btf__distill_base()Alan Maguire2024-07-011-1/+1
| | | | | | | | | | | Coverity points out that after calling btf__new_empty_split() the wrong value is checked for error. Fixes: 58e185a0dc35 ("libbpf: Add btf__distill_base() creating split BTF with distilled base BTF") Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20240629100058.2866763-1-alan.maguire@oracle.com
* libbpf: Fix clang compilation error in btf_relocate.cAlan Maguire2024-06-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When building with clang for ARCH=i386, the following errors are observed: CC kernel/bpf/btf_relocate.o ./tools/lib/bpf/btf_relocate.c:206:23: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion] 206 | info[id].needs_size = true; | ^ ~ ./tools/lib/bpf/btf_relocate.c:256:25: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion] 256 | base_info.needs_size = true; | ^ ~ 2 errors generated. The problem is we use 1-bit, 31-bit bitfields in a signed int. Changing to bool needs_size: 1; unsigned int size:31; ...resolves the error and pahole reports that 4 bytes are used for the underlying representation: $ pahole btf_name_info tools/lib/bpf/btf_relocate.o struct btf_name_info { const char * name; /* 0 8 */ unsigned int needs_size:1; /* 8: 0 4 */ unsigned int size:31; /* 8: 1 4 */ __u32 id; /* 12 4 */ /* size: 16, cachelines: 1, members: 4 */ /* last cacheline: 16 bytes */ }; Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240624192903.854261-1-alan.maguire@oracle.com
* libbpf: Skip base btf sanity checksAntoine Tenart2024-06-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | When upgrading to libbpf 1.3 we noticed a big performance hit while loading programs using CORE on non base-BTF symbols. This was tracked down to the new BTF sanity check logic. The issue is the base BTF definitions are checked first for the base BTF and then again for every module BTF. Loading 5 dummy programs (using libbpf-rs) that are using CORE on a non-base BTF symbol on my system: - Before this fix: 3s. - With this fix: 0.1s. Fix this by only checking the types starting at the BTF start id. This should ensure the base BTF is still checked as expected but only once (btf->start_id == 1 when creating the base BTF), and then only additional types are checked for each module BTF. Fixes: 3903802bb99a ("libbpf: Add basic BTF sanity validation") Signed-off-by: Antoine Tenart <atenart@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Link: https://lore.kernel.org/bpf/20240624090908.171231-1-atenart@kernel.org
* libbpf,bpf: Share BTF relocate-related code with kernelAlan Maguire2024-06-212-0/+31
| | | | | | | | | | | | | | | | | | | | | Share relocation implementation with the kernel. As part of this, we also need the type/string iteration functions so also share btf_iter.c file. Relocation code in kernel and userspace is identical save for the impementation of the reparenting of split BTF to the relocated base BTF and retrieval of the BTF header from "struct btf"; these small functions need separate user-space and kernel implementations for the separate "struct btf"s they operate upon. One other wrinkle on the kernel side is we have to map .BTF.ids in modules as they were generated with the type ids used at BTF encoding time. btf_relocate() optionally returns an array mapping from old BTF ids to relocated ids, so we use that to fix up these references where needed for kfuncs. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240620091733.1967885-5-alan.maguire@oracle.com
* libbpf: Split field iter code into its own file kernelAlan Maguire2024-06-213-163/+170
| | | | | | | | | This will allow it to be shared with the kernel. No functional change. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240620091733.1967885-4-alan.maguire@oracle.com
* libbpf: BTF relocation followup fixing naming, loop logicAlan Maguire2024-06-211-41/+31
| | | | | | | | | | | | | Use less verbose names in BTF relocation code and fix off-by-one error and typo in btf_relocate.c. Simplify loop over matching distilled types, moving from assigning a _next value in loop body to moving match check conditions into the guard. Suggested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240620091733.1967885-2-alan.maguire@oracle.com
* libbpf: Checking the btf_type kind when fixing variable offsetsDonglin Peng2024-06-211-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I encountered an issue when building the test_progs from the repository [1]: $ pwd /work/Qemu/x86_64/linux-6.10-rc2/tools/testing/selftests/bpf/ $ make test_progs V=1 [...] ./tools/sbin/bpftool gen object ./ip_check_defrag.bpf.linked2.o ./ip_check_defrag.bpf.linked1.o libbpf: failed to find symbol for variable 'bpf_dynptr_slice' in section '.ksyms' Error: failed to link './ip_check_defrag.bpf.linked1.o': No such file or directory (2) [...] Upon investigation, I discovered that the btf_types referenced in the '.ksyms' section had a kind of BTF_KIND_FUNC instead of BTF_KIND_VAR: $ bpftool btf dump file ./ip_check_defrag.bpf.linked1.o [...] [2] DATASEC '.ksyms' size=0 vlen=2 type_id=16 offset=0 size=0 (FUNC 'bpf_dynptr_from_skb') type_id=17 offset=0 size=0 (FUNC 'bpf_dynptr_slice') [...] [16] FUNC 'bpf_dynptr_from_skb' type_id=82 linkage=extern [17] FUNC 'bpf_dynptr_slice' type_id=85 linkage=extern [...] For a detailed analysis, please refer to [2]. We can add a kind checking to fix the issue. [1] https://github.com/eddyz87/bpf/tree/binsort-btf-dedup [2] https://lore.kernel.org/all/0c0ef20c-c05e-4db9-bad7-2cbc0d6dfae7@oracle.com/ Fixes: 8fd27bf69b86 ("libbpf: Add BPF static linker BTF and BTF.ext support") Signed-off-by: Donglin Peng <dolinux.peng@gmail.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240619122355.426405-1-dolinux.peng@gmail.com
* libbpf: Make btf_parse_elf process .BTF.base transparentlyEduard Zingerman2024-06-172-54/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | Update btf_parse_elf() to check if .BTF.base section is present. The logic is as follows: if .BTF.base section exists: distilled_base := btf_new(.BTF.base) if distilled_base: btf := btf_new(.BTF, .base_btf=distilled_base) if base_btf: btf_relocate(btf, base_btf) else: btf := btf_new(.BTF) return btf In other words: - if .BTF.base section exists, load BTF from it and use it as a base for .BTF load; - if base_btf is specified and .BTF.base section exist, relocate newly loaded .BTF against base_btf. Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240613095014.357981-6-alan.maguire@oracle.com
* libbpf: Split BTF relocationAlan Maguire2024-06-176-1/+542
| | | | | | | | | | | | | | | | | | | | | | | | | Map distilled base BTF type ids referenced in split BTF and their references to the base BTF passed in, and if the mapping succeeds, reparent the split BTF to the base BTF. Relocation is done by first verifying that distilled base BTF only consists of named INT, FLOAT, ENUM, FWD, STRUCT and UNION kinds; then we sort these to speed lookups. Once sorted, the base BTF is iterated, and for each relevant kind we check for an equivalent in distilled base BTF. When found, the mapping from distilled -> base BTF id and string offset is recorded. In establishing mappings, we need to ensure we check STRUCT/UNION size when the STRUCT/UNION is embedded in a split BTF STRUCT/UNION, and when duplicate names exist for the same STRUCT/UNION. Otherwise size is ignored in matching STRUCT/UNIONs. Once all mappings are established, we can update type ids and string offsets in split BTF and reparent it to the new base. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240613095014.357981-4-alan.maguire@oracle.com
* libbpf: Add btf__distill_base() creating split BTF with distilled base BTFAlan Maguire2024-06-173-6/+335
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To support more robust split BTF, adding supplemental context for the base BTF type ids that split BTF refers to is required. Without such references, a simple shuffling of base BTF type ids (without any other significant change) invalidates the split BTF. Here the attempt is made to store additional context to make split BTF more robust. This context comes in the form of distilled base BTF providing minimal information (name and - in some cases - size) for base INTs, FLOATs, STRUCTs, UNIONs, ENUMs and ENUM64s along with modified split BTF that points at that base and contains any additional types needed (such as TYPEDEF, PTR and anonymous STRUCT/UNION declarations). This information constitutes the minimal BTF representation needed to disambiguate or remove split BTF references to base BTF. The rules are as follows: - INT, FLOAT, FWD are recorded in full. - if a named base BTF STRUCT or UNION is referred to from split BTF, it will be encoded as a zero-member sized STRUCT/UNION (preserving size for later relocation checks). Only base BTF STRUCT/UNIONs that are either embedded in split BTF STRUCT/UNIONs or that have multiple STRUCT/UNION instances of the same name will _need_ size checks at relocation time, but as it is possible a different set of types will be duplicates in the later to-be-resolved base BTF, we preserve size information for all named STRUCT/UNIONs. - if an ENUM[64] is named, a ENUM forward representation (an ENUM with no values) of the same size is used. - in all other cases, the type is added to the new split BTF. Avoiding struct/union/enum/enum64 expansion is important to keep the distilled base BTF representation to a minimum size. When successful, new representations of the distilled base BTF and new split BTF that refers to it are returned. Both need to be freed by the caller. So to take a simple example, with split BTF with a type referring to "struct sk_buff", we will generate distilled base BTF with a 0-member STRUCT sk_buff of the appropriate size, and the split BTF will refer to it instead. Tools like pahole can utilize such split BTF to populate the .BTF section (split BTF) and an additional .BTF.base section. Then when the split BTF is loaded, the distilled base BTF can be used to relocate split BTF to reference the current (and possibly changed) base BTF. So for example if "struct sk_buff" was id 502 when the split BTF was originally generated, we can use the distilled base BTF to see that id 502 refers to a "struct sk_buff" and replace instances of id 502 with the current (relocated) base BTF sk_buff type id. Distilled base BTF is small; when building a kernel with all modules using distilled base BTF as a test, overall module size grew by only 5.3Mb total across ~2700 modules. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20240613095014.357981-2-alan.maguire@oracle.com
* Merge tag 'for-netdev' of ↵Jakub Kicinski2024-06-116-168/+368
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next Daniel Borkmann says: ==================== pull-request: bpf-next 2024-06-06 We've added 54 non-merge commits during the last 10 day(s) which contain a total of 50 files changed, 1887 insertions(+), 527 deletions(-). The main changes are: 1) Add a user space notification mechanism via epoll when a struct_ops object is getting detached/unregistered, from Kui-Feng Lee. 2) Big batch of BPF selftest refactoring for sockmap and BPF congctl tests, from Geliang Tang. 3) Add BTF field (type and string fields, right now) iterator support to libbpf instead of using existing callback-based approaches, from Andrii Nakryiko. 4) Extend BPF selftests for the latter with a new btf_field_iter selftest, from Alan Maguire. 5) Add new kfuncs for a generic, open-coded bits iterator, from Yafang Shao. 6) Fix BPF selftests' kallsyms_find() helper under kernels configured with CONFIG_LTO_CLANG_THIN, from Yonghong Song. 7) Remove a bunch of unused structs in BPF selftests, from David Alan Gilbert. 8) Convert test_sockmap section names into names understood by libbpf so it can deduce program type and attach type, from Jakub Sitnicki. 9) Extend libbpf with the ability to configure log verbosity via LIBBPF_LOG_LEVEL environment variable, from Mykyta Yatsenko. 10) Fix BPF selftests with regards to bpf_cookie and find_vma flakiness in nested VMs, from Song Liu. 11) Extend riscv32/64 JITs to introduce shift/add helpers to generate Zba optimization, from Xiao Wang. 12) Enable BPF programs to declare arrays and struct fields with kptr, bpf_rb_root, and bpf_list_head, from Kui-Feng Lee. * tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (54 commits) selftests/bpf: Drop useless arguments of do_test in bpf_tcp_ca selftests/bpf: Use start_test in test_dctcp in bpf_tcp_ca selftests/bpf: Use start_test in test_dctcp_fallback in bpf_tcp_ca selftests/bpf: Add start_test helper in bpf_tcp_ca selftests/bpf: Use connect_to_fd_opts in do_test in bpf_tcp_ca libbpf: Auto-attach struct_ops BPF maps in BPF skeleton selftests/bpf: Add btf_field_iter selftests selftests/bpf: Fix send_signal test with nested CONFIG_PARAVIRT libbpf: Remove callback-based type/string BTF field visitor helpers bpftool: Use BTF field iterator in btfgen libbpf: Make use of BTF field iterator in BTF handling code libbpf: Make use of BTF field iterator in BPF linker code libbpf: Add BTF field iterator selftests/bpf: Ignore .llvm.<hash> suffix in kallsyms_find() selftests/bpf: Fix bpf_cookie and find_vma in nested VM selftests/bpf: Test global bpf_list_head arrays. selftests/bpf: Test global bpf_rb_root arrays and fields in nested struct types. selftests/bpf: Test kptr arrays and kptrs in nested struct fields. bpf: limit the number of levels of a nested struct type. bpf: look into the types of the fields of a struct type recursively. ... ==================== Link: https://lore.kernel.org/r/20240606223146.23020-1-daniel@iogearbox.net Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * libbpf: Auto-attach struct_ops BPF maps in BPF skeletonMykyta Yatsenko2024-06-063-3/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Similarly to `bpf_program`, support `bpf_map` automatic attachment in `bpf_object__attach_skeleton`. Currently only struct_ops maps could be attached. On bpftool side, code-generate links in skeleton struct for struct_ops maps. Similarly to `bpf_program_skeleton`, set links in `bpf_map_skeleton`. On libbpf side, extend `bpf_map` with new `autoattach` field to support enabling or disabling autoattach functionality, introducing getter/setter for this field. `bpf_object__(attach|detach)_skeleton` is extended with attaching/detaching struct_ops maps logic. Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240605175135.117127-1-yatsenko@meta.com
| * libbpf: Remove callback-based type/string BTF field visitor helpersAndrii Nakryiko2024-06-052-132/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Now that all libbpf/bpftool code switched to btf_field_iter, remove btf_type_visit_type_ids() and btf_type_visit_str_offs() callback-based helpers as not needed anymore. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240605001629.4061937-6-andrii@kernel.org
| * libbpf: Make use of BTF field iterator in BTF handling codeAndrii Nakryiko2024-06-051-22/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | Use new BTF field iterator logic to replace all the callback-based visitor calls. There is still a .BTF.ext callback-based visitor APIs that should be converted, which will happens as a follow up. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240605001629.4061937-4-andrii@kernel.org
| * libbpf: Make use of BTF field iterator in BPF linker codeAndrii Nakryiko2024-06-053-26/+40
| | | | | | | | | | | | | | | | | | | | | | | | Switch all BPF linker code dealing with iterating BTF type ID and string offset fields to new btf_field_iter facilities. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240605001629.4061937-3-andrii@kernel.org
| * libbpf: Add BTF field iteratorAndrii Nakryiko2024-06-052-0/+186
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement iterator-based type ID and string offset BTF field iterator. This is used extensively in BTF-handling code and BPF linker code for various sanity checks, rewriting IDs/offsets, etc. Currently this is implemented as visitor pattern calling custom callbacks, which makes the logic (especially in simple cases) unnecessarily obscure and harder to follow. Having equivalent functionality using iterator pattern makes for simpler to understand and maintain code. As we add more code for BTF processing logic in libbpf, it's best to switch to iterator pattern before adding more callback-based code. The idea for iterator-based implementation is to record offsets of necessary fields within fixed btf_type parts (which should be iterated just once), and, for kinds that have multiple members (based on vlen field), record where in each member necessary fields are located. Generic iteration code then just keeps track of last offset that was returned and handles N members correctly. Return type is just u32 pointer, where NULL is returned when all relevant fields were already iterated. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240605001629.4061937-2-andrii@kernel.org
| * libbpf: keep FD_CLOEXEC flag when dup()'ing FDAndrii Nakryiko2024-06-011-7/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure to preserve and/or enforce FD_CLOEXEC flag on duped FDs. Use dup3() with O_CLOEXEC flag for that. Without this fix libbpf effectively clears FD_CLOEXEC flag on each of BPF map/prog FD, which is definitely not the right or expected behavior. Reported-by: Lennart Poettering <lennart@poettering.net> Fixes: bc308d011ab8 ("libbpf: call dup2() syscall directly") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20240529223239.504241-1-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| * libbpf: Configure log verbosity with env variableMykyta Yatsenko2024-05-292-2/+28
| | | | | | | | | | | | | | | | | | | | Configure logging verbosity by setting LIBBPF_LOG_LEVEL environment variable, which is applied only to default logger. Once user set their custom logging callback, it is up to them to handle filtering. Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240524131840.114289-1-yatsenko@meta.com
* | libbpf: don't close(-1) in multi-uprobe feature detectorAndrii Nakryiko2024-05-311-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | Guard close(link_fd) with extra link_fd >= 0 check to prevent close(-1). Detected by Coverity static analysis. Fixes: 04d939a2ab22 ("libbpf: detect broken PID filtering logic for multi-uprobe") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20240529231212.768828-1-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* | libbpf: detect broken PID filtering logic for multi-uprobeAndrii Nakryiko2024-05-251-1/+30
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Libbpf is automatically (and transparently to user) detecting multi-uprobe support in the kernel, and, if supported, uses multi-uprobes to improve USDT attachment speed. USDTs can be attached system-wide or for the specific process by PID. In the latter case, we rely on correct kernel logic of not triggering USDT for unrelated processes. As such, on older kernels that do support multi-uprobes, but still have broken PID filtering logic, we need to fall back to singular uprobes. Unfortunately, whether user is using PID filtering or not is known at the attachment time, which happens after relevant BPF programs were loaded into the kernel. Also unfortunately, we need to make a call whether to use multi-uprobes or singular uprobe for SEC("usdt") programs during BPF object load time, at which point we have no information about possible PID filtering. The distinction between single and multi-uprobes is small, but important for the kernel. Multi-uprobes get BPF_TRACE_UPROBE_MULTI attach type, and kernel internally substitiute different implementation of some of BPF helpers (e.g., bpf_get_attach_cookie()) depending on whether uprobe is multi or singular. So, multi-uprobes and singular uprobes cannot be intermixed. All the above implies that we have to make an early and conservative call about the use of multi-uprobes. And so this patch modifies libbpf's existing feature detector for multi-uprobe support to also check correct PID filtering. If PID filtering is not yet fixed, we fall back to singular uprobes for USDTs. This extension to feature detection is simple thanks to kernel's -EINVAL addition for pid < 0. Acked-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240521163401.3005045-4-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* libbpf: fix feature detectors when using token_fdAndrii Nakryiko2024-05-152-2/+2
| | | | | | | | | | | | | | | | Adjust `union bpf_attr` size passed to kernel in two feature-detecting functions to take into account prog_token_fd field. Libbpf is avoiding memset()'ing entire `union bpf_attr` by only using minimal set of bpf_attr's fields. Two places have been missed when wiring BPF token support in libbpf's feature detection logic. Fix them trivially. Fixes: f3dcee938f48 ("libbpf: Wire up token_fd into feature probing logic") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240513180804.403775-1-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* bpf: Avoid uninitialized value in BPF_CORE_READ_BITFIELDJose E. Marchesi2024-05-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [Changes from V1: - Use a default branch in the switch statement to initialize `val'.] GCC warns that `val' may be used uninitialized in the BPF_CRE_READ_BITFIELD macro, defined in bpf_core_read.h as: [...] unsigned long long val; \ [...] \ switch (__CORE_RELO(s, field, BYTE_SIZE)) { \ case 1: val = *(const unsigned char *)p; break; \ case 2: val = *(const unsigned short *)p; break; \ case 4: val = *(const unsigned int *)p; break; \ case 8: val = *(const unsigned long long *)p; break; \ } \ [...] val; \ } \ This patch adds a default entry in the switch statement that sets `val' to zero in order to avoid the warning, and random values to be used in case __builtin_preserve_field_info returns unexpected values for BPF_FIELD_BYTE_SIZE. Tested in bpf-next master. No regressions. Signed-off-by: Jose E. Marchesi <jose.marchesi@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240508101313.16662-1-jose.marchesi@oracle.com
* libbpf: improve early detection of doomed-to-fail BPF program loadingAndrii Nakryiko2024-05-081-1/+14
| | | | | | | | | | | | | | | | | | | | Extend libbpf's pre-load checks for BPF programs, detecting more typical conditions that are destinated to cause BPF program failure. This is an opportunity to provide more helpful and actionable error message to users, instead of potentially very confusing BPF verifier log and/or error. In this case, we detect struct_ops BPF program that was not referenced anywhere, but still attempted to be loaded (according to libbpf logic). Suggest that the program might need to be used in some struct_ops variable. User will get a message of the following kind: libbpf: prog 'test_1_forgotten': SEC("struct_ops") program isn't referenced anywhere, did you forget to use it? Suggested-by: Tejun Heo <tj@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240507001335.1445325-6-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* libbpf: fix libbpf_strerror_r() handling unknown errorsAndrii Nakryiko2024-05-081-2/+14
| | | | | | | | | | | | | | | | strerror_r(), used from libbpf-specific libbpf_strerror_r() wrapper is documented to return error in two different ways, depending on glibc version. Take that into account when handling strerror_r()'s own errors, which happens when we pass some non-standard (internal) kernel error to it. Before this patch we'd have "ERROR: strerror_r(524)=22", which is quite confusing. Now for the same situation we'll see a bit less visually scary "unknown error (-524)". At least we won't confuse user with irrelevant EINVAL (22). Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240507001335.1445325-5-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* libbpf: handle yet another corner case of nulling out struct_ops programAndrii Nakryiko2024-05-081-1/+9
| | | | | | | | | | | | | | | | There is yet another corner case where user can set STRUCT_OPS program reference in STRUCT_OPS map to NULL, but libbpf will fail to disable autoload for such BPF program. This time it's the case of "new" kernel which has type information about callback field, but user explicitly nulled-out program reference from user-space after opening BPF object. Fix, hopefully, the last remaining unhandled case. Fixes: 0737df6de946 ("libbpf: better fix for handling nulled-out struct_ops program") Fixes: f973fccd43d3 ("libbpf: handle nulled-out program in struct_ops correctly") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240507001335.1445325-3-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* libbpf: remove unnecessary struct_ops prog validity checkAndrii Nakryiko2024-05-081-10/+3
| | | | | | | | | | | | | | | libbpf ensures that BPF program references set in map->st_ops->progs[i] during open phase are always valid STRUCT_OPS programs. This is done in bpf_object__collect_st_ops_relos(). So there is no need to double-check that in bpf_map__init_kern_struct_ops(). Simplify the code by removing unnecessary check. Also, we avoid using local prog variable to keep code similar to the upcoming fix, which adds similar logic in another part of bpf_map__init_kern_struct_ops(). Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240507001335.1445325-2-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* libbpf: Avoid casts from pointers to enums in bpf_tracing.hJose E. Marchesi2024-05-032-47/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [Differences from V1: - Do not introduce a global typedef, as this is a public header. - Keep the void* casts in BPF_KPROBE_READ_RET_IP and BPF_KRETPROBE_READ_RET_IP, as these are necessary for converting to a const void* argument of bpf_probe_read_kernel.] The BPF_PROG, BPF_KPROBE and BPF_KSYSCALL macros defined in tools/lib/bpf/bpf_tracing.h use a clever hack in order to provide a convenient way to define entry points for BPF programs as if they were normal C functions that get typed actual arguments, instead of as elements in a single "context" array argument. For example, PPF_PROGS allows writing: SEC("struct_ops/cwnd_event") void BPF_PROG(cwnd_event, struct sock *sk, enum tcp_ca_event event) { bbr_cwnd_event(sk, event); dctcp_cwnd_event(sk, event); cubictcp_cwnd_event(sk, event); } That expands into a pair of functions: void ____cwnd_event (unsigned long long *ctx, struct sock *sk, enum tcp_ca_event event) { bbr_cwnd_event(sk, event); dctcp_cwnd_event(sk, event); cubictcp_cwnd_event(sk, event); } void cwnd_event (unsigned long long *ctx) { _Pragma("GCC diagnostic push") _Pragma("GCC diagnostic ignored \"-Wint-conversion\"") return ____cwnd_event(ctx, (void*)ctx[0], (void*)ctx[1]); _Pragma("GCC diagnostic pop") } Note how the 64-bit unsigned integers in the incoming CTX get casted to a void pointer, and then implicitly converted to whatever type of the actual argument in the wrapped function. In this case: Arg1: unsigned long long -> void * -> struct sock * Arg2: unsigned long long -> void * -> enum tcp_ca_event The behavior of GCC and clang when facing such conversions differ: pointer -> pointer Allowed by the C standard. GCC: no warning nor error. clang: no warning nor error. pointer -> integer type [C standard says the result of this conversion is implementation defined, and it may lead to unaligned pointer etc.] GCC: error: integer from pointer without a cast [-Wint-conversion] clang: error: incompatible pointer to integer conversion [-Wint-conversion] pointer -> enumerated type GCC: error: incompatible types in assigment (*) clang: error: incompatible pointer to integer conversion [-Wint-conversion] These macros work because converting pointers to pointers is allowed, and converting pointers to integers also works provided a suitable integer type even if it is implementation defined, much like casting a pointer to uintptr_t is guaranteed to work by the C standard. The conversion errors emitted by both compilers by default are silenced by the pragmas. However, the GCC error marked with (*) above when assigning a pointer to an enumerated value is not associated with the -Wint-conversion warning, and it is not possible to turn it off. This is preventing building the BPF kernel selftests with GCC. This patch fixes this by avoiding intermediate casts to void*, replaced with casts to `unsigned long long', which is an integer type capable of safely store a BPF pointer, much like the standard uintptr_t. Testing performed in bpf-next master: - vmtest.sh -- ./test_verifier - vmtest.sh -- ./test_progs - make M=samples/bpf No regressions. Signed-off-by: Jose E. Marchesi <jose.marchesi@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240502170925.3194-1-jose.marchesi@oracle.com
* libbpf: Fix bpf_ksym_exists() in GCCJose E. Marchesi2024-05-031-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The macro bpf_ksym_exists is defined in bpf_helpers.h as: #define bpf_ksym_exists(sym) ({ \ _Static_assert(!__builtin_constant_p(!!sym), #sym " should be marked as __weak"); \ !!sym; \ }) The purpose of the macro is to determine whether a given symbol has been defined, given the address of the object associated with the symbol. It also has a compile-time check to make sure the object whose address is passed to the macro has been declared as weak, which makes the check on `sym' meaningful. As it happens, the check for weak doesn't work in GCC in all cases, because __builtin_constant_p not always folds at parse time when optimizing. This is because optimizations that happen later in the compilation process, like inlining, may make a previously non-constant expression a constant. This results in errors like the following when building the selftests with GCC: bpf_helpers.h:190:24: error: expression in static assertion is not constant 190 | _Static_assert(!__builtin_constant_p(!!sym), #sym " should be marked as __weak"); \ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ Fortunately recent versions of GCC support a __builtin_has_attribute that can be used to directly check for the __weak__ attribute. This patch changes bpf_helpers.h to use that builtin when building with a recent enough GCC, and to omit the check if GCC is too old to support the builtin. The macro used for GCC becomes: #define bpf_ksym_exists(sym) ({ \ _Static_assert(__builtin_has_attribute (*sym, __weak__), #sym " should be marked as __weak"); \ !!sym; \ }) Note that since bpf_ksym_exists is designed to get the address of the object associated with symbol SYM, we pass *sym to __builtin_has_attribute instead of sym. When an expression is passed to __builtin_has_attribute then it is the type of the passed expression that is checked for the specified attribute. The expression itself is not evaluated. This accommodates well with the existing usages of the macro: - For function objects: struct task_struct *bpf_task_acquire(struct task_struct *p) __ksym __weak; [...] bpf_ksym_exists(bpf_task_acquire) - For variable objects: extern const struct rq runqueues __ksym __weak; /* typed */ [...] bpf_ksym_exists(&runqueues) Note also that BPF support was added in GCC 10 and support for __builtin_has_attribute in GCC 9. Locally tested in bpf-next master branch. No regressions. Signed-of-by: Jose E. Marchesi <jose.marchesi@oracle.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/bpf/20240428112559.10518-1-jose.marchesi@oracle.com
* libbpf: fix ring_buffer__consume_n() return result logicAndrii Nakryiko2024-05-031-1/+1
| | | | | | | | | | | | Add INT_MAX check to ring_buffer__consume_n(). We do the similar check to handle int return result of all these ring buffer APIs in other APIs and ring_buffer__consume_n() is missing one. This patch fixes this omission. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20240430201952.888293-2-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* libbpf: fix potential overflow in ring__consume_n()Andrii Nakryiko2024-05-031-1/+1
| | | | | | | | | | | | | ringbuf_process_ring() return int64_t, while ring__consume_n() assigns it to int. It's highly unlikely, but possible for ringbuf_process_ring() to return value larger than INT_MAX, so use int64_t. ring__consume_n() does check INT_MAX before returning int result to the user. Fixes: 4d22ea94ea33 ("libbpf: Add ring__consume_n / ring_buffer__consume_n") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20240430201952.888293-1-andrii@kernel.org Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-02 16:46:15 +0100