summaryrefslogtreecommitdiffstats
path: root/lib/xz (follow)
Commit message (Collapse)AuthorAgeFilesLines
* lib/xz, lib/decompress_unxz.c: Fix spelling in commentsLasse Collin2021-10-191-1/+1
| | | | | | | | | uncompressible -> incompressible non-splitted -> non-split Link: https://lore.kernel.org/r/20211010213145.17462-6-xiang@kernel.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
* lib/xz: Add MicroLZMA decoderLasse Collin2021-10-194-3/+178
| | | | | | | | | | | | | | | | | | | | | | | | MicroLZMA is a yet another header format variant where the first byte of a raw LZMA stream (without the end of stream marker) has been replaced with a bitwise-negation of the lc/lp/pb properties byte. MicroLZMA was created to be used in EROFS but can be used by other things too where wasting minimal amount of space for headers is important. This is implemented using most of the LZMA2 code as is so the amount of new code is small. The API has a few extra features compared to the XZ decoder. On the other hand, the API lacks XZ_BUF_ERROR support which is important to take into account when using this API. MicroLZMA doesn't support BCJ filters. In theory they could be added later as there are many unused/reserved values for the first byte of the compressed stream but in practice it is somewhat unlikely to happen due to a few implementation reasons. Link: https://lore.kernel.org/r/20211010213145.17462-5-xiang@kernel.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
* lib/xz: Move s->lzma.len = 0 initialization to lzma_reset()Lasse Collin2021-10-191-2/+1
| | | | | | | | | | It's a more logical place even if the resetting needs to be done only once per LZMA2 stream (if lzma_reset() called in the middle of an LZMA2 stream, .len will already be 0). Link: https://lore.kernel.org/r/20211010213145.17462-4-xiang@kernel.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
* lib/xz: Validate the value before assigning it to an enum variableLasse Collin2021-10-191-3/+3
| | | | | | | | | | | | | | | | | This might matter, for example, if the underlying type of enum xz_check was a signed char. In such a case the validation wouldn't have caught an unsupported header. I don't know if this problem can occur in the kernel on any arch but it's still good to fix it because some people might copy the XZ code to their own projects from Linux instead of the upstream XZ Embedded repository. This change may increase the code size by a few bytes. An alternative would have been to use an unsigned int instead of enum xz_check but using an enumeration looks cleaner. Link: https://lore.kernel.org/r/20211010213145.17462-3-xiang@kernel.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
* lib/xz: Avoid overlapping memcpy() with invalid input with in-place ↵Lasse Collin2021-10-191-2/+19
| | | | | | | | | | | | | | | | | | decompression With valid files, the safety margin described in lib/decompress_unxz.c ensures that these buffers cannot overlap. But if the uncompressed size of the input is larger than the caller thought, which is possible when the input file is invalid/corrupt, the buffers can overlap. Obviously the result will then be garbage (and usually the decoder will return an error too) but no other harm will happen when such an over-run occurs. This change only affects uncompressed LZMA2 chunks and so this should have no effect on performance. Link: https://lore.kernel.org/r/20211010213145.17462-2-xiang@kernel.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
* lib/decompressors: fix spelling mistakesZhen Lei2021-07-012-5/+5
| | | | | | | | | | | | | | | | | | | | Fix some spelling mistakes in comments: sentinal ==> sentinel compresed ==> compressed dependeny ==> dependency immediatelly ==> immediately dervied ==> derived splitted ==> split nore ==> not independed ==> independent asumed ==> assumed Link: https://lkml.kernel.org/r/20210604085656.12257-1-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> Cc: Jiri Kosina <jkosina@suse.cz> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* Revert "lib: Revert use of fallthrough pseudo-keyword in lib/"Nick Desaulniers2020-11-182-10/+10
| | | | | | | | | | | | | | | | This reverts commit 6a9dc5fd6170 ("lib: Revert use of fallthrough pseudo-keyword in lib/") Now that we can build arch/powerpc/boot/ free of -Wimplicit-fallthrough, re-enable these fixes for lib/. Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Gustavo A. R. Silva <gustavoars@kernel.org> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Link: https://github.com/ClangBuiltLinux/linux/issues/236 Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
* lib: Revert use of fallthrough pseudo-keyword in lib/Gustavo A. R. Silva2020-08-242-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following build error for powerpc64 was reported by Nathan Chancellor: "$ scripts/config --file arch/powerpc/configs/powernv_defconfig -e KERNEL_XZ $ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc64le-linux- distclean powernv_defconfig zImage ... In file included from arch/powerpc/boot/../../../lib/decompress_unxz.c:234, from arch/powerpc/boot/decompress.c:38: arch/powerpc/boot/../../../lib/xz/xz_dec_stream.c: In function 'dec_main': arch/powerpc/boot/../../../lib/xz/xz_dec_stream.c:586:4: error: 'fallthrough' undeclared (first use in this function) 586 | fallthrough; | ^~~~~~~~~~~ This will end up affecting distribution configurations such as Debian and OpenSUSE according to my testing. I am not sure what the solution is, the PowerPC wrapper does not set -D__KERNEL__ so I am not sure that compiler_attributes.h can be safely included." In order to avoid these sort of problems, it seems that the best solution is to use /* fall through */ comments instead of the fallthrough pseudo-keyword macro in lib/, for now. Reported-by: Nathan Chancellor <natechancellor@gmail.com> Fixes: df561f6688fe ("treewide: Use fallthrough pseudo-keyword") Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Reviewed-and-tested-by: Nathan Chancellor <natechancellor@gmail.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* treewide: Use fallthrough pseudo-keywordGustavo A. R. Silva2020-08-242-10/+10
| | | | | | | | | | Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
* lib/: replace HTTP links with HTTPS onesAlexander A. Klimov2020-08-125-5/+5
| | | | | | | | | | | | Rationale: Reduces attack surface on kernel devs opening the links for MITM as HTTPS traffic is much harder to manipulate. Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Coly Li <colyli@suse.de> [crc64.c] Link: http://lkml.kernel.org/r/20200726112154.16510-1-grandmaster@al2klimov.de Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* docs: move remaining stuff under Documentation/*.txt to Documentation/stagingMauro Carvalho Chehab2020-06-191-1/+1
| | | | | | | | | | | | | | There are several files that I was unable to find a proper place for them, and 3 ones that are still in plain old text format. Let's place those stuff behind the carpet, as we'd like to keep the root directory clean. We can later discuss and move those into better places. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/11bd0d75e65a874f7c276a0aeab0fe13f3376f5f.1592203650.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net>
* lib/xz: fix XZ_DYNALLOC to avoid useless memory reallocationsLasse Collin2019-11-161-0/+1
| | | | | | | | | | | | | | s->dict.allocated was initialized to 0 but never set after a successful allocation, thus the code always thought that the dictionary buffer has to be reallocated. Link: http://lkml.kernel.org/r/20191104185107.3b6330df@tukaani.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Reported-by: Yu Sun <yusun2@cisco.com> Acked-by: Daniel Walker <danielwa@cisco.com> Cc: "Yixia Si (yisi)" <yisi@cisco.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* treewide: Add SPDX license identifier - Makefile/KconfigThomas Gleixner2019-05-212-0/+2
| | | | | | | | | | | | | | Add SPDX license identifiers to all Make/Kconfig files which: - Have no license information of any form These files fall under the project license, GPL v2 only. The resulting SPDX license identifier is: GPL-2.0-only Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* lib/xz: Put CRC32_POLY_LE in xz_private.hJoel Stanley2018-10-022-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes a regression introduced by faa16bc404d72a5 ("lib: Use existing define with polynomial"). The cleanup added a dependency on include/linux, which broke the PowerPC boot wrapper/decompresser when KERNEL_XZ is enabled: BOOTCC arch/powerpc/boot/decompress.o In file included from arch/powerpc/boot/../../../lib/decompress_unxz.c:233, from arch/powerpc/boot/decompress.c:42: arch/powerpc/boot/../../../lib/xz/xz_crc32.c:18:10: fatal error: linux/crc32poly.h: No such file or directory #include <linux/crc32poly.h> ^~~~~~~~~~~~~~~~~~~ The powerpc decompresser is a hairy corner of the kernel. Even while building a 64-bit kernel it needs to build a 32-bit binary and therefore avoid including files from include/linux. This allows users of the xz library to avoid including headers from 'include/linux/' while still achieving the cleanup of the magic number. Fixes: faa16bc404d72a5 ("lib: Use existing define with polynomial") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: kbuild test robot <lkp@intel.com> Suggested-by: Christophe LEROY <christophe.leroy@c-s.fr> Signed-off-by: Joel Stanley <joel@jms.id.au> Tested-by: Meelis Roos <mroos@linux.ee> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* lib: Use existing define with polynomialKrzysztof Kozlowski2018-07-271-1/+2
| | | | | | | Do not define again the polynomial but use header with existing define. Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* lib/xz: Add fall-through comments to a switch statementLasse Collin2017-10-121-0/+16
| | | | | | | | | It's good style. I was also told that GCC 7 is more strict and might give a warning when such comments are missing. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Suggested-by: Andrei Borzenkov <arvidjaar@gmail.com> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
* lib/xz: enable all filters by default in KconfigLasse Collin2014-06-051-12/+12
| | | | | | | | | | | | | | | | This restores the old behavior that existed before 2013-02-22, when changes were made by 64dbfb444c150 ("decompressors: drop dependency on CONFIG_EXPERT") and 5dc49c75a2 ("decompressors: make the default XZ_DEC_* config match the selected architecture"). Disabling the filters only makes sense on embedded systems. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Acked-by: Kyle McMartin <kyle@infradead.org> Cc: Florian Fainelli <f.fainelli@gmail.com> Cc: Phillip Lougher <phillip@lougher.demon.co.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* lib/xz: add comments for the intentionally missing break statementsLasse Collin2014-06-051-0/+4
| | | | | | Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* decompressors: fix typo "POWERPC"Paul Bolle2013-03-131-1/+1
| | | | | | | | | | | | | | | | | | Commit 5dc49c75a26b ("decompressors: make the default XZ_DEC_* config match the selected architecture") added default y if POWERPC to lib/xz/Kconfig. But there is no Kconfig symbol POWERPC. The most general Kconfig symbol for the powerpc architecture is PPC. So let's use that. Signed-off-by: Paul Bolle <pebolle@tiscali.nl> Cc: Florian Fainelli <florian@openwrt.org> Cc: Lasse Collin <lasse.collin@tukaani.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* decompressors: make the default XZ_DEC_* config match the selected architectureFlorian Fainelli2013-02-221-6/+6
| | | | | | | | | | | | | Change the defautl XZ_DEC_* config symbol to match the configured architecture. It is perfectly legitimate to support multiple XZ BCJ filters for different architectures (e.g.: to mount foreign squashfs/xz compressed filesystems), it is however more natural not to select them all by default, but only the one matching the configured architecture. Signed-off-by: Florian Fainelli <florian@openwrt.org> Acked-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* decompressors: drop dependency on CONFIG_EXPERTFlorian Fainelli2013-02-221-6/+6
| | | | | | | | | | Remove the XZ_DEC_* depedencey on CONFIG_EXPERT as recommended by Lasse Colin. Signed-off-by: Florian Fainelli <florian@openwrt.org> Acked-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* decompressors: group XZ_DEC_* symbols under an if XZ_BCJ / endifFlorian Fainelli2013-02-221-6/+4
| | | | | | | | | | Group all architecture-specific BCJ filter configuration symbols under an if XZ_BCJ / endif statement. Signed-off-by: Florian Fainelli <florian@openwrt.org> Acked-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* XZ: Fix incorrect XZ_BUF_ERRORLasse Collin2011-09-211-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xz_dec_run() could incorrectly return XZ_BUF_ERROR if all of the following was true: - The caller knows how many bytes of output to expect and only provides that much output space. - When the last output bytes are decoded, the caller-provided input buffer ends right before the LZMA2 end of payload marker. So LZMA2 won't provide more output anymore, but it won't know it yet and thus won't return XZ_STREAM_END yet. - A BCJ filter is in use and it hasn't left any unfiltered bytes in the temp buffer. This can happen with any BCJ filter, but in practice it's more likely with filters other than the x86 BCJ. This fixes <https://bugzilla.redhat.com/show_bug.cgi?id=735408> where Squashfs thinks that a valid file system is corrupt. This also fixes a similar bug in single-call mode where the uncompressed size of a block using BCJ + LZMA2 was 0 bytes and caller provided no output space. Many empty .xz files don't contain any blocks and thus don't trigger this bug. This also tweaks a closely related detail: xz_dec_bcj_run() could call xz_dec_lzma2_run() to decode into temp buffer when it was known to be useless. This was harmless although it wasted a minuscule number of CPU cycles. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Cc: stable <stable@kernel.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* XZ: Fix missing <linux/kernel.h> includeLasse Collin2011-07-241-1/+1
| | | | | | | | | | | | | | <linux/kernel.h> is needed for min_t. The old version happened to work on x86 because <asm/unaligned.h> indirectly includes <linux/kernel.h>, but it didn't work on ARM. <linux/kernel.h> includes <asm/byteorder.h> so it's not necessary to include it explicitly anymore. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Cc: stable <stable@kernel.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* XZ decompressor: Fix decoding of empty LZMA2 streamsLasse Collin2011-05-021-3/+3
| | | | | | | | | | The old code considered valid empty LZMA2 streams to be corrupt. Note that a typical empty .xz file has no LZMA2 data at all, and thus most .xz files having no uncompressed data are handled correctly even without this fix. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* kconfig: rename CONFIG_EMBEDDED to CONFIG_EXPERTDavid Rientjes2011-01-211-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The meaning of CONFIG_EMBEDDED has long since been obsoleted; the option is used to configure any non-standard kernel with a much larger scope than only small devices. This patch renames the option to CONFIG_EXPERT in init/Kconfig and fixes references to the option throughout the kernel. A new CONFIG_EMBEDDED option is added that automatically selects CONFIG_EXPERT when enabled and can be used in the future to isolate options that should only be considered for embedded systems (RISC architectures, SLOB, etc). Calling the option "EXPERT" more accurately represents its intention: only expert users who understand the impact of the configuration changes they are making should enable it. Reviewed-by: Ingo Molnar <mingo@elte.hu> Acked-by: David Woodhouse <david.woodhouse@intel.com> Signed-off-by: David Rientjes <rientjes@google.com> Cc: Greg KH <gregkh@suse.de> Cc: "David S. Miller" <davem@davemloft.net> Cc: Jens Axboe <axboe@kernel.dk> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Robin Holt <holt@sgi.com> Cc: <linux-arch@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* decompressors: add XZ decompressor moduleLasse Collin2011-01-1311-0/+3344
In userspace, the .lzma format has become mostly a legacy file format that got superseded by the .xz format. Similarly, LZMA Utils was superseded by XZ Utils. These patches add support for XZ decompression into the kernel. Most of the code is as is from XZ Embedded <http://tukaani.org/xz/embedded.html>. It was written for the Linux kernel but is usable in other projects too. Advantages of XZ over the current LZMA code in the kernel: - Nice API that can be used by other kernel modules; it's not limited to kernel, initramfs, and initrd decompression. - Integrity check support (CRC32) - BCJ filters improve compression of executable code on certain architectures. These together with LZMA2 can produce a few percent smaller kernel or Squashfs images than plain LZMA without making the decompression slower. This patch: Add the main decompression code (xz_dec), testing module (xz_dec_test), wrapper script (xz_wrap.sh) for the xz command line tool, and documentation. The xz_dec module is enough to have a usable XZ decompressor e.g. for Squashfs. Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Alain Knaff <alain@knaff.lu> Cc: Albin Tonnerre <albin.tonnerre@free-electrons.com> Cc: Phillip Lougher <phillip@lougher.demon.co.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-21 14:47:59 +0100