summaryrefslogtreecommitdiffstats
path: root/kernel/gen_kheaders.sh (follow)
Commit message (Collapse)AuthorAgeFilesLines
* kheaders: use `command -v` to test for existence of `cpio`Miguel Ojeda2024-05-291-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Commit 13e1df09284d ("kheaders: explicitly validate existence of cpio command") added an explicit check for `cpio` using `type`. However, `type` in `dash` (which is used in some popular distributions and base images as the shell script runner) prints the missing message to standard output, and thus no error is printed: $ bash -c 'type missing >/dev/null' bash: line 1: type: missing: not found $ dash -c 'type missing >/dev/null' $ For instance, this issue may be seen by loongarch builders, given its defconfig enables CONFIG_IKHEADERS since commit 9cc1df421f00 ("LoongArch: Update Loongson-3 default config file"). Therefore, use `command -v` instead to have consistent behavior, and take the chance to provide a more explicit error. Fixes: 13e1df09284d ("kheaders: explicitly validate existence of cpio command") Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kheaders: explicitly define file modes for archived headersMatthias Maennich2024-05-291-1/+1
| | | | | | | | | | | | | | | Build environments might be running with different umask settings resulting in indeterministic file modes for the files contained in kheaders.tar.xz. The file itself is served with 444, i.e. world readable. Archive the files explicitly with 744,a+X to improve reproducibility across build environments. --mode=0444 is not suitable as directories need to be executable. Also, 444 makes it hard to delete all the readonly files after extraction. Cc: stable@vger.kernel.org Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* Revert "kheaders: substituting --sort in archive creation"Masahiro Yamada2023-05-281-6/+3
| | | | | | | | | | | | | This reverts commit 700dea5a0bea9f64eba89fae7cb2540326fdfdc1. The reason for that commit was --sort=ORDER introduced in tar 1.28 (2014). More than 3 years have passed since then. Requiring GNU tar 1.28 should be fine now because we require GCC 5.1 (2015). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
* kheaders: use standard naming for the temporary directoryThomas Weißschuh2023-01-221-1/+1
| | | | | | | | | | | | | | If the kheaders archive generation is interrupted then this directory may be left on disk and not ignored by git. By using the standard naming schema for temporary files and directories the default .gitignore and "make clean" rules will apply. Suggested-by: Nicolas Schier <nicolas@fjasle.eu> Suggested-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> Tested-by: Nicolas Schier <nicolas@fjasle.eu> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kheaders: explicitly validate existence of cpio commandThomas Weißschuh2023-01-181-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the cpio command is not available the error emitted by gen_kheaders.so is not clear as all output of the call to cpio is discarded: GNU make 4.4: GEN kernel/kheaders_data.tar.xz find: 'standard output': Broken pipe find: write error make[2]: *** [kernel/Makefile:157: kernel/kheaders_data.tar.xz] Error 127 make[1]: *** [scripts/Makefile.build:504: kernel] Error 2 GNU make < 4.4: GEN kernel/kheaders_data.tar.xz make[2]: *** [kernel/Makefile:157: kernel/kheaders_data.tar.xz] Error 127 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [scripts/Makefile.build:504: kernel] Error 2 Add an explicit check that will trigger a clear message about the issue: CHK kernel/kheaders_data.tar.xz ./kernel/gen_kheaders.sh: line 17: type: cpio: not found The other commands executed by gen_kheaders.sh are part of a standard installation, so they are not checked. Reported-by: Amy Parker <apark0006@student.cerritos.edu> Link: https://lore.kernel.org/lkml/CAPOgqxFva=tOuh1UitCSN38+28q3BNXKq19rEsVNPRzRqKqZ+g@mail.gmail.com/ Signed-off-by: Thomas Weißschuh <linux@weissschuh.net> Reviewed-by: Nicolas Schier <n.schier@avm.de> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kbuild: build init/built-in.a just onceMasahiro Yamada2022-09-281-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Kbuild builds init/built-in.a twice; first during the ordinary directory descending, second from scripts/link-vmlinux.sh. We do this because UTS_VERSION contains the build version and the timestamp. We cannot update it during the normal directory traversal since we do not yet know if we need to update vmlinux. UTS_VERSION is temporarily calculated, but omitted from the update check. Otherwise, vmlinux would be rebuilt every time. When Kbuild results in running link-vmlinux.sh, it increments the version number in the .version file and takes the timestamp at that time to really fix UTS_VERSION. However, updating the same file twice is a footgun. To avoid nasty timestamp issues, all build artifacts that depend on init/built-in.a are atomically generated in link-vmlinux.sh, where some of them do not need rebuilding. To fix this issue, this commit changes as follows: [1] Split UTS_VERSION out to include/generated/utsversion.h from include/generated/compile.h include/generated/utsversion.h is generated just before the vmlinux link. It is generated under include/generated/ because some decompressors (s390, x86) use UTS_VERSION. [2] Split init_uts_ns and linux_banner out to init/version-timestamp.c from init/version.c init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary directory descending, they are compiled with __weak and used to determine if vmlinux needs relinking. Just before the vmlinux link, they are compiled without __weak to embed the real version and timestamp. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kheaders: Have cpio unconditionally replace filesDaniel Mentz2022-05-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | For out-of-tree builds, this script invokes cpio twice to copy header files from the srctree and subsequently from the objtree. According to a comment in the script, there might be situations in which certain files already exist in the destination directory when header files are copied from the objtree: "The second CPIO can complain if files already exist which can happen with out of tree builds having stale headers in srctree. Just silence CPIO for now." GNU cpio might simply print a warning like "newer or same age version exists", but toybox cpio exits with a non-zero exit code unless the command line option "-u" is specified. To improve compatibility with toybox cpio, add the command line option "-u" to unconditionally replace existing files in the destination directory. Signed-off-by: Daniel Mentz <danielmentz@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kbuild: clean up ${quiet} checks in shell scriptsMasahiro Yamada2021-05-261-3/+1
| | | | | | | | | | | | | | | | There were efforts to make 'make -s' really silent when it is a warning-free build. The conventional way was to let a shell script check ${quiet}, and if it is 'silent_', suppress the stdout by itself. With the previous commit, the 'cmd' takes care of it now. The 'cmd' is also invoked from if_changed, if_changed_dep, and if_changed_rule. You can omit ${quiet} checks in shell scripts when they are invoked from the 'cmd' macro. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kbuild: redo fake deps at include/config/*.hAlexey Dobriyan2021-04-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make include/config/foo/bar.h fake deps files generation simpler. * delete .h suffix those aren't header files, shorten filenames, * delete tolower() Linux filesystems can deal with both upper and lowercase filenames very well, * put everything in 1 directory Presumably 'mkdir -p' split is from dark times when filesystems handled huge directories badly, disks were round adding to seek times. x86_64 allmodconfig lists 12364 files in include/config. ../obj/include/config/ ├── 104_QUAD_8 ├── 60XX_WDT ├── 64BIT ... ├── ZSWAP_DEFAULT_ON ├── ZSWAP_ZPOOL_DEFAULT └── ZSWAP_ZPOOL_DEFAULT_ZBUD 0 directories, 12364 files Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kbuild: add variables for compression toolsDenis Efremov2020-06-061-1/+1
| | | | | | | | | | | | | | | | | | | | Allow user to use alternative implementations of compression tools, such as pigz, pbzip2, pxz. For example, multi-threaded tools to speed up the build: $ make GZIP=pigz BZIP2=pbzip2 Variables _GZIP, _BZIP2, _LZOP are used internally because original env vars are reserved by the tools. The use of GZIP in gzip tool is obsolete since 2015. However, alternative implementations (e.g., pigz) still rely on it. BZIP2, BZIP, LZOP vars are not obsolescent. The credit goes to @grsecurity. As a sidenote, for multi-threaded lzma, xz compression one can use: $ export XZ_OPT="--threads=0" Signed-off-by: Denis Efremov <efremov@linux.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* kheaders: explain why include/config/autoconf.h is excluded from md5sumMasahiro Yamada2019-11-111-2/+9
| | | | | | | | This comment block explains why include/generated/compile.h is omitted, but nothing about include/generated/autoconf.h, which might be more difficult to understand. Add more comments. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: remove the last bashism to allow sh to run itMasahiro Yamada2019-11-111-6/+7
| | | | | | | | | | | | 'pushd' ... 'popd' is the last bash-specific code in this script. One way to avoid it is to run the code in a sub-shell. With that addressed, you can run this script with sh. I replaced $(BASH) with $(CONFIG_SHELL), and I changed the hashbang to #!/bin/sh. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: optimize header copy for in-tree buildsMasahiro Yamada2019-11-111-7/+9
| | | | | | | | | | | This script copies headers by the cpio command twice; first from srctree, and then from objtree. However, when we building in-tree, we know the srctree and the objtree are the same. That is, all the headers copied by the first cpio are overwritten by the second one. Skip the first cpio when we are building in-tree. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: optimize md5sum calculation for in-tree buildsMasahiro Yamada2019-11-111-16/+16
| | | | | | | | | | | | | | | | | | | | This script computes md5sum of headers in srctree and in objtree. However, when we are building in-tree, we know the srctree and the objtree are the same. That is, we end up with the same computation twice. In fact, the first two lines of kernel/kheaders.md5 are always the same for in-tree builds. Unify the two md5sum calculations. For in-tree builds ($building_out_of_srctree is empty), we check only two directories, "include", and "arch/$SRCARCH/include". For out-of-tree builds ($building_out_of_srctree is 1), we check 4 directories, "$srctree/include", "$srctree/arch/$SRCARCH/include", "include", and "arch/$SRCARCH/include" since we know they are all different. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: remove unneeded 'cat' command piped to 'head' / 'tail'Masahiro Yamada2019-11-111-4/+4
| | | | | | | | | | | | | | | | | | | | The 'head' and 'tail' commands can take a file path directly. So, you do not need to run 'cat'. cat kernel/kheaders.md5 | head -1 ... is equivalent to: head -1 kernel/kheaders.md5 and the latter saves forking one process. While I was here, I replaced 'head -1' with 'head -n 1'. I also replaced '==' with '=' since we do not have a good reason to use the bashism. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: substituting --sort in archive creationDmitry Goldin2019-10-171-4/+7
| | | | | | | | | | | | | | | | | The option --sort=ORDER was only introduced in tar 1.28 (2014), which is rather new and might not be available in some setups. This patch tries to replicate the previous behaviour as closely as possible to fix the kheaders build for older environments. It does not produce identical archives compared to the previous version due to minor sorting differences but produces reproducible results itself in my tests. Reported-by: Andreas Schwab <schwab@suse.de> Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch> Tested-by: Andreas Schwab <schwab@suse.de> Tested-by: Quentin Perret <qperret@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: make headers archive reproducibleDmitry Goldin2019-10-051-1/+4
| | | | | | | | | | | | | | | | | | | | | In commit 43d8ce9d65a5 ("Provide in-kernel headers to make extending kernel easier") a new mechanism was introduced, for kernels >=5.2, which embeds the kernel headers in the kernel image or a module and exposes them in procfs for use by userland tools. The archive containing the header files has nondeterminism caused by header files metadata. This patch normalizes the metadata and utilizes KBUILD_BUILD_TIMESTAMP if provided and otherwise falls back to the default behaviour. In commit f7b101d33046 ("kheaders: Move from proc to sysfs") it was modified to use sysfs and the script for generation of the archive was renamed to what is being patched. Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
* kheaders: include only headers into kheaders_data.tar.xzMasahiro Yamada2019-07-091-31/+16
| | | | | | | | | | | | | | | | | | | | | | | | Currently, kheaders_data.tar.xz contains some build scripts as well as headers. None of them is needed in the header archive. For ARCH=x86, this commit excludes the following from the archive: arch/x86/include/asm/Kbuild arch/x86/include/uapi/asm/Kbuild include/asm-generic/Kbuild include/config/auto.conf include/config/kernel.release include/config/tristate.conf include/uapi/asm-generic/Kbuild include/uapi/Kbuild kernel/gen_kheaders.sh This change is actually motivated for the planned header compile-testing because it will generate more build artifacts, which should not be included in the archive. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
* kheaders: remove meaningless -R option of 'ls'Masahiro Yamada2019-07-091-4/+4
| | | | | | | | | | | | | | | The -R option of 'ls' is supposed to be used for directories. -R, --recursive list subdirectories recursively Since 'find ... -type f' only matches to regular files, we do not expect directories passed to the 'ls' command here. Giving -R is harmless at least, but unneeded. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
* kheaders: Do not regenerate archive if config is not changedJoel Fernandes (Google)2019-05-241-4/+11
| | | | | | | | | | | | | | | Linus reported an issue that doing an allmodconfig was causing the kheaders archive to be regenerated even though the config is the same. This patch fixes the issue by ignoring the config-related header files for "knowing when to regenerate based on timestamps". Instead, if the CONFIG_X_Y option really changes, then we there are the include/config/X/Y.h which will already tells us "if a config really changed". So we don't really need these files for regeneration detection anyway, and ignoring them fixes Linus's issue. Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* kheaders: Move from proc to sysfsJoel Fernandes (Google)2019-05-241-0/+89
The kheaders archive consisting of the kernel headers used for compiling bpf programs is in /proc. However there is concern that moving it here will make it permanent. Let us move it to /sys/kernel as discussed [1]. [1] https://lore.kernel.org/patchwork/patch/1067310/#1265969 Suggested-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>