summaryrefslogtreecommitdiffstats
path: root/src/ukify (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
* | test_ukify: print message when skipping whole test fileZbigniew Jędrzejewski-Szmek2023-05-091-2/+4
|/
* test_ukify: add test for combining config and cmdlineZbigniew Jędrzejewski-Szmek2023-05-051-0/+80
|
* test_ukify: rework how --flakes argument is appendedZbigniew Jędrzejewski-Szmek2023-05-053-6/+16
| | | | | | | | | | | | | | | | | | | | The usual approach is to put 'addopts = --flakes' in setup.cfg. Unfortunately this fails badly when pytest-flakes is not installed: ERROR: usage: test_ukify.py [options] [file_or_dir] [file_or_dir] [...] test_ukify.py: error: unrecognized arguments: --flakes pytest-flakes is not packaged everywhere, and this test is not very important, so let's just do it only if pytest-flakes is available. We now detect if pytest-flakes is available and only add '--flakes' conditionally. This unfortunately means that when invoked via 'pytest' or directly as 'src/ukify/test/test_ukify.py', '--flakes' will not be appended automatically. But I don't see a nice way to achieve previous automatic behaviour. (I first considered making 'setup.cfg' templated. But then it is created in the build directory, but we would need it in the source directory for pytest to load it automatically. So to load the file, we'd need to give an argument to pytest anyway, so we don't gain anything with this more complex approach.)
* test_ukify: propagate failureZbigniew Jędrzejewski-Szmek2023-05-051-1/+1
| | | | | Oops. This explains why the tests were "passing" in CI even though a direct pytest invocation would fail.
* ukify: appease mypyZbigniew Jędrzejewski-Szmek2023-05-051-21/+42
| | | | | | | Note to self: PEP 585 introduced using collection types as types, and is available since 3.9. PEP 604 allows writing unions with "|", but is only available since 3.10, so not yet here because we maintain compat with 3.9.
* test_ukify: add tests for the new functionalityZbigniew Jędrzejewski-Szmek2023-05-051-0/+88
|
* ukify: PeError → PEErrorZbigniew Jędrzejewski-Szmek2023-05-051-13/+13
| | | | | We don't lowercase acronyms in systemd usually. Remove unnused f'' prefix to avoid a pylint warning.
* ukify: rework option parsing to support a config fileZbigniew Jędrzejewski-Szmek2023-05-051-153/+417
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some ways this is similar to mkosi: we have a argparse.ArgumentParser() with a bunch of options, and a configparser.ConfigParser() with an overlapping set of options. Many options are settable in both places, but not all. In mkosi, we define this in three places (a dataclass, and a function for argparse, and a function for configparser). Here, we have one huge list of ConfigItem instances. Each instance specifies the full metadata for both parsers. Argparse generates a --help string for all the options, and we also append a config file sample to --help based on the ConfigItem data: $ python src/ukify/ukify.py --help|tail -n 25 config file: [UKI] Linux = LINUX Initrd = INITRD… Cmdline = TEXT|@PATH OSRelease = TEXT|@PATH DeviceTree = PATH Splash = BMP PCRPKey = KEY Uname = VERSION EFIArch = ia32|x64|arm|aa64|riscv64 Stub = STUB PCRBanks = BANK… SigningEngine = ENGINE SecureBootPrivateKey = SB_KEY SecureBootCertificate = SB_CERT SignKernel = SIGN_KERNEL [PCRSignature:NAME] PCRPrivateKey = PATH PCRPublicKey = PATH Phases = PHASE-PATH… While writing this I needed to check the argument parsing, so I added a --summary switch. It just pretty-prints the resulting option dictionary: $ python src/ukify/ukify.py /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd --pcr-private-key=PRIV.key --pcr-public-key=PUB.key --config=man/ukify-example.conf --summary Host arch 'x86_64', EFI arch 'x64' {'_groups': [0, 'initrd', 'system'], 'cmdline': 'A1 B2 C3', 'config': 'man/ukify-example.conf', 'devicetree': None, 'efi_arch': 'x64', 'initrd': [PosixPath('initrd1'), PosixPath('initrd2'), PosixPath('initrd3'), PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd')], 'linux': PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux'), 'measure': None, 'os_release': PosixPath('/etc/os-release'), 'output': 'linux.efi', 'pcr_banks': ['sha1', 'sha384'], 'pcr_private_keys': [PosixPath('PRIV.key'), PosixPath('pcr-private-initrd-key.pem'), PosixPath('pcr-private-system-key.pem')], 'pcr_public_keys': [PosixPath('PUB.key'), PosixPath('pcr-public-initrd-key.pem'), PosixPath('pcr-public-system-key.pem')], 'pcrpkey': None, 'phase_path_groups': [None, ['enter-initrd'], ['enter-initrd:leave-initrd', 'enter-initrd:leave-initrd:sysinit', 'enter-initrd:leave-initrd:sysinit:ready']], 'sb_cert': PosixPath('mkosi.secure-boot.crt'), 'sb_key': PosixPath('mkosi.secure-boot.key'), 'sections': [], 'sign_kernel': None, 'signing_engine': None, 'splash': None, 'stub': PosixPath('/usr/lib/systemd/boot/efi/linuxx64.efi.stub'), 'summary': True, 'tools': None, 'uname': None} With --summary, existence of input paths is not checked. I think we'll want to show them, instead of throwing an error, but in red, similarly to 'bootctl list'. This also fixes tests which were failing with e.g. E FileNotFoundError: [Errno 2] No such file or directory: '/ARG1' =========================== short test summary info ============================ FAILED ../src/ukify/test/test_ukify.py::test_parse_args_minimal - FileNotFoun... FAILED ../src/ukify/test/test_ukify.py::test_parse_args_many - FileNotFoundEr... FAILED ../src/ukify/test/test_ukify.py::test_parse_sections - FileNotFoundErr... =================== 3 failed, 10 passed, 3 skipped in 1.51s ====================
* test_ukify: fix two failing testsZbigniew Jędrzejewski-Szmek2023-05-031-4/+1
| | | | | | | | | | | Fixup for 22ad038ac6e4fe5e4a68555f0e70bd0a16fb5616 and 3fc5eed47091363247012454df458e1a3303bf12. It seems that the tests are not executed properly in CI. Nevertheless, test-ukify appears in logs: rpm-build:fedora-rawhide-x86_64: 409/1191 systemd / test-ukify OK 0.16s This is strange.
* test_ukify: fix loop iterationZbigniew Jędrzejewski-Szmek2023-05-031-1/+1
| | | | We'd try to access 'linux' or 'initrd' after failing to set it.
* ukify: add missing headerZbigniew Jędrzejewski-Szmek2023-05-031-0/+15
| | | | This file is installed, so it should have the long header.
* ukify: use UPPERCASE for parameter namesZbigniew Jędrzejewski-Szmek2023-05-031-2/+4
| | | | | | | | | | | | | | | We generally nowadays use UPPERCASE for parameters in variuos help text. Let's be consistent here too, and also drop duplicated 'usage:': $ ukify -h usage: ukify [options…] LINUX INITRD… ukify -h | --help Build and sign Unified Kernel Images positional arguments: LINUX vmlinuz file [.linux section] INITRD… initrd files [.initrd section] ...
* ukify: allow building PE addonLuca Boccassi2023-04-262-6/+32
| | | | | Make the kernel optional too, so that we can easily build and sign a PE addon, that can be used to carry extra command line options.
* ukify: Strip symbol/string table for old stubsJan Janssen2023-04-011-0/+13
|
* ukify: Add workarounds for older stubsDaan De Meyer2023-03-281-4/+24
| | | | | | | | | Older stubs are either not stripped, causing their total size to be unaligned because of an unaligned symbol table at the end, or stripped, causing the raw data pointers and sizes to be unaligned because strip does not follow the PE spec correctly when stripping. Let's add workarounds for both issues, so that we can use ukify with older stubs as well.
* Revert "ukify: Weaken file alignment assertions"Daan De Meyer2023-03-281-7/+1
| | | | This reverts commit 23428bb19e49cf510c65e2896f1a7e4b12ca1dbc.
* ukify: Weaken file alignment assertionsDaan De Meyer2023-03-271-1/+7
| | | | | | Older versions of the stub are not aligned to the PE file alignment size. If we remove the assertions, the UKI still boots without issues, so let's drop the assertions and print a message about it instead.
* ukify: Add riscv32 and loongarch supportJan Janssen2023-03-241-0/+3
|
* ukify: Use pefile to add sections to EFI stubJan Janssen2023-03-241-50/+60
|
* ukify: allow uncompressed kernel images for UNAME detection on aarch64 and ↵Tobias Powalowski2023-03-221-0/+4
| | | | | | | | | | | | riscv64 (#26929) Uncompressed aarch64 and riscv64 kernels have a different startpoint than x86. Example output from ukify: aarch64: NotImplementedError: unknown file format (starts with b'MZ@\xfa') riscv64: NotImplementedError: unknown file format (starts with b'MZo\x10') Add check for (b'MZ') to catch both in one call. Fix: https://github.com/systemd/systemd/issues/26923
* tree-wide: Use correct SPDX license identifierJan Janssen2023-02-242-2/+2
|
* ukify: Set fast_load option when parsing PE filesDaan De Meyer2023-02-201-2/+2
| | | | | | Let's skip parsing of some irrelevant information that we don't use to speed up building UKIs with large initrds from +-15s to less than 1s.
* ukify: fix padding lengthYu Watanabe2023-02-151-1/+2
|
* treewide: fix a few typos in NEWS, docs and commentsDmitry V. Levin2023-02-151-1/+1
|
* tree-wide: fix typo and comment style updateYu Watanabe2023-02-151-1/+1
|
* ukify: add explanatory message when import failsZbigniew Jędrzejewski-Szmek2023-02-081-6/+13
|
* ukify: python 3.9 compat followupDaan De Meyer2023-01-271-1/+7
|
* ukify: Downgrade required python version to 3.9Daan De Meyer2023-01-201-21/+18
|
* ukify: Fix version stringDaan De Meyer2023-01-151-1/+1
| | | | Let's make sure we mimick the version of our other CLI tooling.
* ukify: Fix tools detection if --tools was not passedJan Janssen2023-01-111-1/+5
| | | | | | In 789a642738d28cf2a8ad3f65df9c0c136e83af09 llvm-objcopy was given higher priority over objcopy, but this would only work if --tools was also passed.
* Merge pull request #25912 from DaanDeMeyer/ukifyDaan De Meyer2023-01-021-2/+5
|\ | | | | ukify fixes
| * ukify: Require specifying --tools for each tools directoryDaan De Meyer2023-01-021-1/+1
| | | | | | | | | | | | | | Instead of consuming N arguments as tools directories, let's always only consume one argument per specification of --tools. This avoids issues where the linux image and initrd are interpreted as tools directories.
| * ukify: Handle directories in path_is_readable()Daan De Meyer2023-01-021-1/+4
| |
* | ukify: Fix section offset calculationJan Janssen2023-01-021-3/+3
|/ | | | | | | | | | | | | objcopy seems to expect that the offset passed to --change-section-vma is absolute instead of relative to ImageBase. If this is not accounted for an invalid image is created that cannot be loaded: 0 .osrel 0000016b 0000000200016000 0000000200016000 00000400 2**2 … 6 .text 0000d242 0000000140001000 0000000140001000 00c6e800 2**4 This isn't an issue with gnu-efi based PE images, but natively created ones will have a non-zero ImageBase.
* Merge pull request #25814 from DaanDeMeyer/ukifyLuca Boccassi2022-12-221-14/+31
|\ | | | | ukify: Prefer using llvm-objcopy instead of objcopy
| * ukify: Validate that there are no overlapping sectionsDaan De Meyer2022-12-221-0/+14
| | | | | | | | | | Let's make sure that after calling objcopy we have no overlapping sections in the UKI
| * ukify: Prefer using llvm-objcopy instead of objcopyDaan De Meyer2022-12-221-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | llvm-objcopy works on stubs built for foreign architectures whereas objcopy doesn't so let's prefer using llvm-objcopy instead of objcopy. llvm-objcopy automatically sets the virtual address and doesn't provide an option to set it manually so we only add --change-section-vma when using objcopy The default section flags differ between llvm-objcopy and objcopy so we add a default for the section flags so we make sure all sections are read-only data unless specified otherwise.
| * ukify: Allow passing multiple directories to --toolsDaan De Meyer2022-12-221-4/+6
| |
* | ukify: check early if inputs exist and are readableZbigniew Jędrzejewski-Szmek2022-12-201-6/+25
| | | | | | | | | | | | | | | | | | | | | | | | It's much nicer for the user if we fail early instead of doing partial processing if we cannot read some input. We can't do those checks immediately from argparse.Parser.parse_args(), because we want to fully process the commandline first. In particular, even with invalid args, if --help is specified somewhere, we want to handle that. Thus, we need to delay the checks after argparse.Parser.parse_args() returns. Ukify didn't have type annotations on functions, but it probably should. Jörg's suggested correction included them and we might just as well start here.
* | ukify: catch error when loading foreign pe fileZbigniew Jędrzejewski-Szmek2022-12-201-1/+4
|/ | | | | | | | | | | The autodetection code is supposed to throw ValueError when it cannot figure out the version so that we fall back to the next method. With the patch: Kernel version not specified, starting autodetection 😖. Real-Mode Kernel Header magic not found + readelf --notes vmlinuz/arm64/vmlinuz-6.0.9-300.fc37.aarch64 readelf: vmlinuz/arm64/vmlinuz-6.0.9-300.fc37.aarch64: Error: Not an ELF file - it has the wrong magic bytes at the start Found uname version: 6.0.9-300.fc37.aarch64
* ukify: allow multiple initrdsZbigniew Jędrzejewski-Szmek2022-12-072-11/+38
| | | | | | | | | If given, multiple initrds are concatenated into a temporary file which then becomes the .initrd section. It is also possible to give no initrd. After all, some machines boot without an initrd, and it should be possible to use the stub without requiring an initrd. (The stub might not like this, but this is something to fix there.)
* ci: install pefileZbigniew Jędrzejewski-Szmek2022-12-072-1/+8
|
* ukify: try to find the uname string in the linux image if not specifiedZbigniew Jędrzejewski-Szmek2022-12-072-0/+131
| | | | | | | | | | | | | | | The approach is based on mkinicpio's autodetection. This is hacky as hell. Some cases are actually fairly nice: ppc64el images have a note that contains 'uname -r'. (The note is not uniquely labeled at all, and only contains the release part instead of the full version-hostname-release string, and we don't actually care about ppc, and it's very hard to read the note from Python, but in general that'd be the approach I'd like.) I opted to simply read and decompress the full linux binary in some cases. Python doesn't make it easy to do streaming decompression with regexp matching, and it doesn't seem to matter much: the image decompresses in a fraction of a second.
* tests: add pytest tests for ukifyZbigniew Jędrzejewski-Szmek2022-12-079-0/+510
| | | | | | | | | | | | | | | | | | | Some gymnastics were needed to import ukify as a module. Before the file was templated, this was trivial: insert the directory in sys.path, call import. But it's a real pain to import the unsuffixed file after processing. Instead, the untemplated file is imported, which works well enough for tests and is very simple. The tests can be called via pytest: PATH=build/:$PATH pytest -v src/ukify/test/test_ukify.py or directly: PATH=build/:$PATH src/ukify/test/test_ukify.py or via the meson test machinery output: meson test -C build test-ukify -v or without verbose output: meson test -C build test-ukify Zekret files are obfuscated using base64.
* meson,ukify: hook up ukify, add --version optionZbigniew Jędrzejewski-Szmek2022-12-071-0/+6
| | | | | | | | | | | | | | | | | | | The option is added because we have a similar one for kernel-install. This program requires python, and some people might want to skip it because of this. The tool is installed in /usr/lib/systemd for now, since the interface might change. A template file is used, but there is no .in suffix. The problem is that we'll later want to import the file as a module for tests, but recent Python versions make it annoyingly hard to import a module from a file without a .py suffix. imp.load_sources() works, but it is deprecated and throws warnings. importlib.machinery.SourceFileLoader().load_module() works, but is also deprecated. And the documented replacements are a maze of twisted little callbacks that result in an empty module. So let's take the easy way out, and skip the suffix which makes it easy to import the template as a module after adding the directory to sys.path.
* ukify: add helper to create UKIsZbigniew Jędrzejewski-Szmek2022-12-071-0/+576
Features: - adds sections .linux, .initrd, .uname, .osrel, .pcrpkey, .pcrsig, .cmdline, .splash - multiple initrds can be concatenated - section flags are set properly (READONLY, DATA or CODE) - uses systemd-measure to precalculate pcr measurements and create a signed json policy - the inner linux image will be signed automatically with sbsign if unsigned - uses sbsign to sign the output image - offsets are calculated so that sections are placed adjacent, with .linux last - custom sections are possible - multiple pcr signing keys can be specified and different boot phase paths can be signed with different keys - most things can be overriden (path to tools, stub file, signing keys, pcr banks, boot phase paths, whether to sign things) - superficial verification of slash bmp is done - kernel uname "scraping" from the kernel if not specified (in a later patch) TODO: - change systemd-measure to not require a functional TPM2. W/o this, we'd need to support all banks in the build machine, which is hard to guarantee. - load signing keys from /etc/kernel/ - supress exceptions, so if something external fails, the user will not see a traceback - conversion to BMP from other formats $ sudo /usr/lib/systemd/ukify \ --tools=build/ \ --measure \ /lib/modules/6.0.5-300.fc37.x86_64/vmlinuz \ /boot/08a5690a2eed47cf92ac0a5d2e3cf6b0/6.0.5-300.fc37.x86_64/initrd \ --secureboot-private-key=server.key --secureboot-certificate=server.crt \ --pcr-private-key=tpm2-pcr-private.pem --pcr-public-key=tpm2-pcr-public.pem \ --cmdline='rw quiet' \ --section test:TESTTESTTEST \ --section test2:TESTTESTTEST2 \ --pcr-banks=sha1 \ --uname="$(uname -rv)" Host arch 'x86_64', efi arch 'x64' + sbverify --list /lib/modules/6.0.5-300.fc37.x86_64/vmlinuz + build/systemd-measure calculate --linux=/lib/modules/6.0.5-300.fc37.x86_64/vmlinuz --osrel=/etc/os-release --cmdline=/tmp/tmpcmdline_5aufjir --pcrpkey=tpm2-pcr-public.pem --initrd=/boot/08a5690a2eed47cf92ac0a5d2e3cf6b0/6.0.5-300.fc37.x86_64/initrd --bank=sha1 11:sha1=03df5e5243bc002b959d52359fe04e266d0b5ebf 11:sha1=54949b82bae32e80343ff0f01eeeeb75f4c07d3f 11:sha1=0fc62be88aa9c5ad7282aa8adb504f451bcec9df 11:sha1=b71155e7fcd467f7c1696f675e37887032e2eafa + build/systemd-measure sign --linux=/lib/modules/6.0.5-300.fc37.x86_64/vmlinuz --osrel=/etc/os-release --cmdline=/tmp/tmpcmdline_5aufjir --pcrpkey=tpm2-pcr-public.pem --initrd=/boot/08a5690a2eed47cf92ac0a5d2e3cf6b0/6.0.5-300.fc37.x86_64/initrd --bank=sha1 --private-key=tpm2-pcr-private.pem --public-key=tpm2-pcr-public.pem + objcopy /usr/lib/systemd/boot/efi/linuxx64.efi.stub --add-section .osrel=/etc/os-release --change-section-vma .osrel=0x22000 --add-section .cmdline=/tmp/tmpcmdline_5aufjir --change-section-vma .cmdline=0x23000 --add-section .pcrpkey=tpm2-pcr-public.pem --change-section-vma .pcrpkey=0x24000 --add-section .initrd=/boot/08a5690a2eed47cf92ac0a5d2e3cf6b0/6.0.5-300.fc37.x86_64/initrd --change-section-vma .initrd=0x25000 --add-section .uname=/tmp/tmpuname0v3uzh5r --change-section-vma .uname=0x4009000 --add-section .test=/tmp/tmptestuxve59c8 --change-section-vma .test=0x400a000 --add-section .test2=/tmp/tmptest2_i143p9i --change-section-vma .test2=0x400b000 --add-section .pcrsig=/tmp/tmppcrsigdtcqxz_w --change-section-vma .pcrsig=0x400c000 --add-section .linux=/lib/modules/6.0.5-300.fc37.x86_64/vmlinuz --change-section-vma .linux=0x400d000 /tmp/uki4vsbf7y8 + sbsign --key server.key --cert server.crt /tmp/uki4vsbf7y8 --output vmlinuz.efi warning: data remaining[79849520 vs 79866644]: gaps between PE/COFF sections? warning: data remaining[79849520 vs 79866648]: gaps between PE/COFF sections? Signing Unsigned original image Wrote signed vmlinuz.efi
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-07 20:18:36 +0100