Zstandard Changelog

What's new in Zstandard 1.5.6

Apr 1, 2024
  • This release highlights the deployment of Google Chrome 123, introducing zstd-encoding for Web traffic, introduced as a preferable option for compression of dynamic contents. With limited web server support for zstd-encoding due to its novelty, we are launching an updated Zstandard version to facilitate broader adoption.
  • New stable parameter ZSTD_c_targetCBlockSize:
  • Using zstd compression for large documents over the Internet, data is segmented into smaller blocks of up to 128 KB, for incremental updates. This is crucial for applications like Chrome that process parts of documents as they arrive. However, on slow or congested networks, there can be some brief unresponsiveness in the middle of a block transmission, delaying update. To mitigate such scenarios, libzstd introduces the new parameter ZSTD_c_targetCBlockSize, enabling the division of blocks into even smaller segments to enhance initial byte delivery speed. Activating this feature incurs a cost, both runtime (equivalent to -2% speed at level 8) and a slight compression efficiency decrease (ULL by @Cyan4973 in #3929
  • Fix building xxhash on AIX 5.1 by @likema in #3860
  • Fix building on HP-UX 11.11 PA-RISC by @likema in #3862
  • Fix AsyncIO reading seed queueing by @yoniko in #3940
  • Use ZSTD_LEGACY_SUPPORT=5 in "make test" by @embg in #3943
  • Pin sanitizer CI jobs to ubuntu-20.04 by @embg in #3945
  • chore: fix some typos by @acceptacross in #3949
  • new method to deal with offset==0 erroneous edge case by @Cyan4973 in #3937
  • add tests inspired from #2927 by @Cyan4973 in #3948
  • cmake refactor: move HP-UX specific logic into its own function by @Cyan4973 in #3946
  • Fix #3719 : mixing -c, -o and --rm by @Cyan4973 in #3942
  • minor: fix incorrect debug level by @Cyan4973 in #3936
  • add RISC-V emulation tests to Github CI by @Cyan4973 in #3934
  • prevent XXH64 from being autovectorized by XXH512 by default by @Cyan4973 in #3933
  • Stop Hardcoding the POSIX Version on BSDs by @felixhandte in #3952
  • Convert the CircleCI workflow to a GitHub Actions workflow by @jk0 in #3901
  • Add common compressed file types to --exclude-compressed by @daniellerozenblit in #3951
  • Export ZSTD_LEGACY_SUPPORT in tests/Makefile by @embg in #3955
  • Exercise ZSTD_findDecompressedSize() in the simple decompression fuzzer by @embg in #3959
  • Update ZSTD_RowFindBestMatch comment by @yoniko in #3947
  • Add the zeroSeq sample by @Cyan4973 in #3954
  • [cpu] Backport fix for rbx clobbering on Windows with Clang by @terrelln in #3957
  • Do not truncate file name in verbose mode by @Cyan4973 in #3956
  • updated documentation by @Cyan4973 in #3958
  • [asm][aarch64] Mark that BTI and PAC are supported by @terrelln in #3961
  • Use utimensat() on FreeBSD by @felixhandte in #3960
  • reduce the amount of #include in cover.h by @Cyan4973 in #3962
  • Remove Erroneous Exclusion of Hidden Files and Folders in --output-dir-mirror by @felixhandte in #3963
  • Promote ZSTD_c_targetCBlockSize Parameter to Stable API by @felixhandte in #3964
  • [cmake] Always create libzstd target by @terrelln in #3965
  • Remove incorrect docs regarding ZSTD_findFrameCompressedSize() by @embg in #3967
  • add line number to debug traces by @Cyan4973 in #3966
  • bump version number by @Cyan4973 in #3969
  • Export zstd's public headers via BUILD_INTERFACE by @terrelln in #3968
  • Fix bug with streaming decompression of magicless format by @embg in #3971
  • pzstd: use c++14 without conditions by @kanavin in #3682
  • Fix bugs in simple decompression fuzzer by @yoniko in #3978
  • Fuzzing and bugfixes for magicless-format decoding by @embg in #3976
  • Fix & fuzz ZSTD_generateSequences by @terrelln in #3981
  • Fail on errors when building fuzzers by @yoniko in #3979
  • [cmake] Emit warnings for contradictory build settings by @terrelln in #3975
  • Document the process for adding a new fuzzer by @embg in #3982
  • Fix -Werror=pointer-arith in fuzzers by @embg in #3983
  • Doc update by @Cyan4973 in #3977
  • v1.5.6 by @Cyan4973 in #3984
  • New Contributors:
  • @Kim-SSi made their first contribution in #3600
  • @mredig made their first contribution in #3614
  • @dunhor made their first contribution in #3636
  • @sighingnow made their first contribution in #3657
  • @nidhijaju made their first contribution in #3688
  • @gjasny made their first contribution in #3665
  • @Coder-256 made their first contribution in #3700
  • @LocutusOfBorg made their first contribution in #3695
  • @void0red made their first contribution in #3704
  • @nikohoffren made their first contribution in #3701
  • @alexsifivetw made their first contribution in #3712
  • @0o001 made their first contribution in #3728
  • @QBos07 made their first contribution in #3720
  • @JohanMabille made their first contribution in #3739
  • @klausholstjacobsen made their first contribution in #3745
  • @Saverio976 made their first contribution in #3795
  • @elasota made their first contribution in #3806
  • @jondo2010 made their first contribution in #3812
  • @gruenich made their first contribution in #3807
  • @michoecho made their first contribution in #3847
  • @KapJI made their first contribution in #3850
  • @jcelerier made their first contribution in #3760
  • @sandreenko made their first contribution in #3899
  • @teo-tsirpanis made their first contribution in #3811
  • @aimuz made their first contribution in #3840
  • @acceptacross made their first contribution in #3949
  • @jk0 made their first contribution in #3901

New in Zstandard 1.5.5 (Apr 6, 2023)

  • CHANGELOG:
  • fix: fix rare corruption bug affecting the high compression mode, reported by @danlark1 (#3517, @terrelln)
  • perf: improve mid-level compression speed (#3529, #3533, #3543, @yoniko and #3552, @terrelln)
  • lib: deprecated bufferless block-level API (#3534) by @terrelln
  • cli: mmap large dictionaries to save memory, by @daniellerozenblit
  • cli: improve speed of --patch-from mode (~+50%) (#3545) by @daniellerozenblit
  • cli: improve i/o speed (~+10%) when processing lots of small files (#3479) by @felixhandte
  • cli: zstd no longer crashes when requested to write into write-protected directory (#3541) by @felixhandte
  • cli: fix decompression into block device using -o (#3584, @Cyan4973) reported by @georgmu
  • build: fix zstd CLI compiled with lzma support but not zlib support (#3494) by @Hello71
  • build: fix cmake does no longer require 3.18 as minimum version (#3510) by @kou
  • build: fix MSVC+ClangCL linking issue (#3569) by @tru
  • build: fix zstd-dll, version of zstd CLI that links to the dynamic library (#3496) by @yoniko
  • build: fix MSVC warnings (#3495) by @embg
  • doc: updated zstd specification to clarify corner cases, by @Cyan4973
  • doc: document how to create fat binaries for macos (#3568) by @rickmark
  • misc: improve seekable format ingestion speed (~+100%) for very small chunk sizes (#3544) by @Cyan4973
  • misc: tests/fullbench can benchmark multiple files (#3516) by @dloidolt
  • FULL CHANGELIST (auto-generated):
  • Fix all MSVC warnings by @embg in #3495
  • Fix zstd-dll build missing dependencies by @yoniko in #3496
  • Bump github/codeql-action from 2.2.1 to 2.2.4 by @dependabot in #3503
  • Github Action to generate Win64 artifacts by @Cyan4973 in #3491
  • Use correct types in LZMA comp/decomp by @Hello71 in #3497
  • Make Github workflows permissions read-only by default by @yoniko in #3488
  • CI Workflow for external compressors dependencies by @yoniko in #3505
  • Fix cli-tests issues by @daniellerozenblit in #3509
  • Fix Permissions on Publish Release Artifacts Job by @felixhandte in #3511
  • Use f-variants of chmod() and chown() by @felixhandte in #3479
  • Don't require CMake 3.18 or later by @kou in #3510
  • meson: always build the zstd binary when tests are enabled by @eli-schwartz in #3490
  • [bug-fix] Fix rare corruption bug affecting the block splitter by @terrelln in #3517
  • Clarify zstd specification for Huffman blocks by @Cyan4973 in #3514
  • Fix typos found by codespell by @DimitriPapadopoulos in #3513
  • Bump github/codeql-action from 2.2.4 to 2.2.5 by @dependabot in #3518
  • fullbench with two files by @dloidolt in #3516
  • Add initialization of clevel to static cdict (#3525) by @yoniko in #3527
  • [linux-kernel] Fix assert definition by @terrelln in #3532
  • Add ZSTD_set{C,F,}Params() helper functions by @terrelln in #3530
  • Clarify dstCapacity requirements by @terrelln in #3...

New in Zstandard 1.5.4 (Feb 13, 2023)

  • Zstandard v1.5.4 is a pretty big release benefiting from one year of work, spread over > 650 commits. It offers significant performance improvements across multiple scenarios, as well as new features (detailed below). There is a crop of little bug fixes too, a few ones targeting the 32-bit mode are important enough to make this release a recommended upgrade.
  • Various Speed improvements:
  • This release has accumulated a number of scenario-specific improvements, that cumulatively benefit a good portion of installed base in one way or another.
  • Support of externally-defined sequence producers:
  • libzstd can now support external sequence producers via a new advanced registration function ZSTD_registerSequenceProducer() (#3333).
  • This API allows users to provide their own custom sequence producer which libzstd invokes to process each block. The produced list of sequences (literals and matches) is then post-processed by libzstd to produce valid compressed blocks.
  • This block-level offload API is a more granular complement of the existing frame-level offload API compressSequences() (introduced in v1.5.1). It offers an easier migration story for applications already integrated with libzstd: the user application continues to invoke the same compression functions ZSTD_compress2() or ZSTD_compressStream2() as usual, and transparently benefits from the specific properties of the external sequence producer. For example, the sequence producer could be tuned to take advantage of known characteristics of the input, to offer better speed / ratio.
  • One scenario that becomes possible is to combine this capability with hardware-accelerated matchfinders, such as the Intel® QuickAssist accelerator (Intel® QAT) provided in server CPUs such as the 4th Gen Intel® Xeon® Scalable processors (previously codenamed Sapphire Rapids). More details to be provided in future communications.
  • CHANGELOG:
  • perf: +20% faster huffman decompression for targets that can't compile x64 assembly (#3449, @terrelln)
  • perf: up to +10% faster streaming compression at levels 1-2 (#3114, @embg)
  • perf: +4-13% for levels 5-12 by optimizing function generation (#3295, @terrelln)
  • pref: +3-11% compression speed for arm target (#3199, #3164, #3145, #3141, #3138, @JunHe77 and #3139, #3160, @danlark1)
  • perf: +5-30% faster dictionary compression at levels 1-4 (#3086, #3114, #3152, @embg)
  • perf: +10-20% cold dict compression speed by prefetching CDict tables (#3177, @embg)
  • perf: +1% faster compression by removing a branch in ZSTD_fast_noDict (#3129, @felixhandte)
  • perf: Small compression ratio improvements in high compression mode (#2983, #3391, @Cyan4973 and #3285, #3302, @daniellerozenblit)
  • perf: small speed improvement by better detecting STATIC_BMI2 for clang (#3080, @TocarIP)
  • perf: Improved streaming performance when ZSTD_c_stableInBuffer is set (#2974, @Cyan4973)
  • cli: Asynchronous I/O for improved cli speed (#2975, #2985, #3021, #3022, @yoniko)
  • cli: Change zstdless behavior to align with zless (#2909, @binhdvo)
  • cli: Keep original file if -c or --stdout is given (#3052, @dirkmueller)
  • cli: Keep original files when result is concatenated into a single output with -o (#3450, @Cyan4973)
  • cli: Preserve Permissions and Ownership of regular files (#3432, @felixhandte)
  • cli: Print zlib/lz4/lzma library versions with -vv (#3030, @terrelln)
  • cli: Print checksum value for single frame files with -lv (#3332, @Cyan4973)
  • cli: Print dictID when present with -lv (#3184, @htnhan)
  • cli: when stderr is not the console, disable status updates, but preserve final summary (#3458, @Cyan4973)
  • cli: support --best and --no-name in gzip compatibility mode (#3059, @dirkmueller)
  • cli: support for posix high resolution timer clock_gettime(), for improved benchmark accuracy (#3423, @Cyan4973)
  • cli: improved help/usage (-h, -H) formatting (#3094, @dirkmueller and #3385, @jonpalmisc)
  • cli: Fix better handling of bogus numeric values (#3268, @ctkhanhly)
  • cli: Fix input consists of multiple files and stdin (#3222, @yoniko)
  • cli: Fix tiny files passthrough (#3215, @cgbur)
  • cli: Fix for -r on empty directory (#3027, @brailovich)
  • cli: Fix empty string as argument for --output-dir-* (#3220, @embg)
  • cli: Fix decompression memory usage reported by -vv --long (#3042, @u1f35c, and #3232, @zengyijing)
  • cli: Fix infinite loop when empty input is passed to trainer (#3081, @terrelln)
  • cli: Fix --adapt doesn't work when --no-progress is also set (#3354, @terrelln)
  • api: Support for External Sequence Producer (#3333, @embg)
  • api: Support for in-place decompression (#3432, @terrelln)
  • api: New ZSTD_CCtx_setCParams() function, set all parameters defined in a ZSTD_compressionParameters structure (#3403, @Cyan4973)
  • api: Streaming decompression detects incorrect header ID sooner (#3175, @Cyan4973)
  • api: Window size resizing optimization for edge case (#3345, @daniellerozenblit)
  • api: More accurate error codes for busy-loop scenarios (#3413, #3455, @Cyan4973)
  • api: Fix limit overflow in compressBound and decompressBound (#3362, #3373, Cyan4973) reported by @nigeltao
  • api: Deprecate several advanced experimental functions: streaming (#3408, @embg), copy (#3196, @mileshu)
  • bug: Fix corruption that rarely occurs in 32-bit mode with wlog=25 (#3361, @terrelln)
  • bug: Fix for block-splitter (#3033, @Cyan4973)
  • bug: Fixes for Sequence Compression API (#3023, #3040, @Cyan4973)
  • bug: Fix leaking thread handles on Windows (#3147, @animalize)
  • bug: Fix timing issues with cmake/meson builds (#3166, #3167, #3170, @Cyan4973)
  • build: Allow user to select legacy level for cmake (#3050, @shadchin)
  • build: Enable legacy support by default in cmake (#3079, @niamster)
  • build: Meson build script improvements (#3039, #3120, #3122, #3327, #3357, @eli-schwartz and #3276, @neheb)
  • build: Add aarch64 to supported architectures for zstd_trace (#3054, @ooosssososos)
  • build: support AIX architecture (#3219, @qiongsiwu)
  • build: Fix ZSTD_LIB_MINIFY build macro, which now reduces static library size by half (#3366, @terrelln)
  • build: Fix Windows issues with Multithreading translation layer (#3364, #3380, @yoniko) and ARM64 target (#3320, @cwoffenden)
  • build: Fix cmake script (#3382, #3392, @terrelln and #3252 @Tachi107 and #3167 @Cyan4973)
  • doc: Updated man page, providing more details for --train mode (#3112, @Cyan4973)
  • doc: Add decompressor errata document (#3092, @terrelln)
  • misc: Enable Intel CET (#2992, #2994, @hjl-tools)
  • misc: Fix contrib/ seekable format (#3058, @yhoogstrate and #3346, @daniellerozenblit)
  • misc: Improve speed of the one-file library generator (#3241, @wahern and #3005, @cwoffenden)
  • PR list (generated by Github)
  • x86-64: Enable Intel CET by @hjl-tools in #2992
  • Add GitHub Action Checking that Zstd Runs Successfully Under CET by @felixhandte in #3015
  • [opt] minor compression ratio improvement by @Cyan4973 in #2983
  • Simplify HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop by @WojciechMula in #3013
  • Async write for decompression by @yoniko in #2975
  • ZSTD CLI: Use buffered output by @yoniko in #2985
  • Use faster Python script to amalgamate by @cwoffenden in #3005
  • Change zstdless behavior to align with zless by @binhdvo in #2909
  • AsyncIO compression part 1 - refactor of existing asyncio code by @yoniko in #3021
  • Converge sumtype (offset | repcode) numeric representation towards offBase by @Cyan4973 in #2965
  • fix sequence compression API in Explicit Delimiter mode by @Cyan4973 in #3023
  • Lazy parameters adaptation (part 1 - ZSTD_c_stableInBuffer) by @Cyan4973 in #2974
  • Print zlib/lz4/lzma library versions in verbose version output by @terrelln in #3030
  • fix for -r on empty directory by @brailovich in #3027
  • Add new CLI testing platform by @terrelln in #3020
  • AsyncIO compression part 2 - added async read and asyncio to compression code by @yoniko in #3022
  • Macos playtest envvars fix by @yoniko in #3035
  • Fix required decompression memory usage reported by -vv + --long by @u1f35c in #3042
  • Select legacy level for cmake by @shadchin in #3050
  • [trace] Add aarch64 to supported architectures for zstd_trace by @ooosssososos in #3054
  • New features for largeNbDicts benchmark by @embg in #3063
  • Use helper function for bit manipulations. by @TocarIP in #3075
  • [programs] Fix infinite loop when empty input is passed to trainer by @terrelln in #3081
  • Enable STATIC_BMI2 for gcc/clang by @TocarIP in #3080
  • build:cmake: enable ZSTD legacy support by default by @niamster in #3079
  • Implement more gzip compatibility (#3037) by @dirkmueller in #3059
  • [doc] Add decompressor errata document by @terrelln in #3092
  • Handle newer less versions in zstdless testing by @dirkmueller in #3093
  • [contrib][linux] Fix a warning in zstd_reset_cstream() by @cyberknight777 in #3088
  • Software pipeline for ZSTD_compressBlock_fast_dictMatchState (+5-6% compression speed) by @embg in #3086
  • Keep original file if -c or --stdout is given by @dirkmueller in #3052
  • Split help in long and short version, cleanup formatting by @dirkmueller in #3094
  • updated man page, providing more details for --train mode by @Cyan4973 in #3112
  • Meson fixups for Windows by @eli-schwartz in #3039
  • meson: for internal linkage, link to both libzstd and a static copy of it by @eli-schwartz in #3122
  • Software pipeline for ZSTD_compressBlock_fast_extDict (+4-9% compression speed) by @embg in #3114
  • ZSTD_fast_noDict: Avoid Safety Check When Writing ip1 into Table by @felixhandte in #3129
  • Correct and clarify repcode offset history logic by @embg in #3127
  • [lazy] Optimize ZSTD_row_getMatchMask for levels 8-10 for ARM by @danlark1 in #3139
  • fix leaking thread handles on Windows by @animalize in #3147
  • Remove expensive assert in --rsyncable hot loop by @terrelln in #3154
  • Bugfix for huge dictionaries by @embg in #3157
  • common: apply two stage copy to aarch64 by @JunHe77 in #3145
  • dec: adjust seqSymbol load on aarch64 by @JunHe77 in #3141
  • Fix big endian ARM NEON path by @danlark1 in #3160
  • [contrib] largeNbDicts bugfix + improvements by @embg in #3161
  • display a warning message when using C90 clock_t by @Cyan4973 in #3166
  • remove explicit standard setting from cmake script by @Cyan4973 in #3167
  • removed gnu99 statement from meson recipe by @Cyan4973 in #3170
  • "Short cache" optimization for level 1-4 DMS (+5-30% compression speed) by @embg in #3152
  • Streaming decompression can detect incorrect header ID sooner by @Cyan4973 in #3175
  • Add prefetchCDictTables CCtxParam (+10-20% cold dict compression speed) by @embg in #3177
  • Fix ZSTD_BUILD_TESTS=ON with MSVC by @nocnokneo in #3180
  • zstd -lv to show dictID by @htnhan in #3184
  • Intial commit to address 3090. Added support to decompress empty block. by @udayanbapat in #3118
  • [largeNbDicts] Second try at fixing decompression segfault to always create compressInstructions by @zhuhan0 in #3209
  • Clarify benchmark chunking docstring by @embg in #3197
  • decomp: add prefetch for matched seq on aarch64 by @JunHe77 in #3164
  • lib: add hint to generate more pipeline friendly code by @JunHe77 in #3138
  • [AIX] Fix Compiler Flags and Bugs on AIX to Pass All Tests by @qiongsiwu in #3219
  • zlibWrapper: Update for zlib 1.2.12 by @orbea in #3217
  • Fix small file passthrough by @cgbur in #3215
  • Add warning when multi-thread decompression is requested by @tomcwang in #3208
  • stdin + multiple file fixes by @yoniko in #3222
  • [AIX] Fixing hash4Ptr for Big Endian Systems by @qiongsiwu in #3227
  • Disallow empty string as argument for --output-dir-flat and --output-dir-mirror by @embg in #3220
  • Deprecate ZSTD_getDecompressedSize() by @terrelln in #3225
  • [T124890272] Mark 2 Obsolete Functions(ZSTD_copy*Ctx) Deprecated in Zstd by @mileshu in #3196
  • fileio_types.h : avoid dependency on mem.h by @Cyan4973 in #3232
  • fixed: verbose output prints wrong value for wlog when doing --long by @zengyijing in #3226
  • Add explicit --pass-through flag and default to enabled for *cat by @terrelln in #3223
  • Document pass-through behavior by @cgbur in #3242
  • restore combine.sh bash performance while still sticking to POSIX by @wahern in #3241
  • Benchmark program for sequence compression API by @embg in #3257
  • streamline make clean list maintenance by adding a CLEAN variable by @Cyan4973 in #3256
  • drop -E flag in sed by @haampie in #3245
  • compress:check more bytes to reduce ZSTD_count call by @JunHe77 in #3199
  • build(cmake): improve pkg-config generation by @Tachi107 in #3252
  • Fix for zstd CLI accepts bogus values for numeric parameters by @ctkhanhly in #3268
  • ci: test pkg-config file by @Tachi107 in #3267
  • Move ZSTD_DEPRECATED before ZSTDLIB_API/ZSTDLIB_STATIC_API for clang by @MaskRay in #3273
  • Enable OpenSSF Scorecard Action by @felixhandte in #3277
  • fixed zstd-pgo target for GCC by @ilyakurdyukov in #3281
  • Cleaner threadPool initialization by @Cyan4973 in #3288
  • Make fuzzing work without ZSTD_MULTITHREAD by @danlark1 in #3291
  • Optimal huf depth by @daniellerozenblit in #3285
  • Make ZSTD_getDictID_fromDDict() Read DictID from DDict by @felixhandte in #3290
  • [contrib][linux-kernel] Generate SPDX license identifiers by @ojeda in #3294
  • [lazy] Use switch instead of indirect function calls, improving compression speed by @terrelln in #3295
  • [linux] Add zstd_common module by @terrelln in #3292
  • Complete migration of ZSTD_c_enableLongDistanceMatching to ZSTD_paramSwitch_e framework by @embg in #3321
  • meson: get version up front by @eli-schwartz in #3327
  • Fix for MSVC C4267 warning on ARM64 (which becomes error C2220 with /WX) by @cwoffenden in #3320
  • Enable dependabot for automatic GitHub Actions updates by @DimitriPapadopoulos in #3284
  • Print checksum value for single frame files in cli with -v -l options by @Cyan4973 in #3332
  • Fix window size resizing optimization for edge case by @daniellerozenblit in #3345
  • [linux-kernel] Fix stack detection for newer gcc by @terrelln in #3348
  • Reserve two fields in ZSTD_frameHeader by @embg in #3349
  • Fix seekable format for empty string by @daniellerozenblit in #3346
  • meson: make backtrace dependency on execinfo for musl libc compatibility by @neheb in #3276
  • Refactor progress bar & summary line logic by @terrelln in #2984
  • Use __attribute__((aligned(1))) for unaligned access by @Hello71 in #2881
  • Separate parameter adaption from display update rate by @terrelln in #3354
  • [decompress] Fix UB nullptr addition & improve fuzzer by @terrelln in #3356
  • [legacy] Simplify legacy codebase by removing esoteric memory accesses and only use memcpy by @terrelln in #3355
  • Fix corruption that rarely occurs in 32-bit mode with wlog=25 by @terrelln in #3361
  • meson: partial fix for building pzstd on MSVC by @eli-schwartz in #3357
  • [CI] Re-enable versions-test by @terrelln in #3371
  • [api][visibility] Make the visibility macros more consistent by @terrelln in #3363
  • [build] Fix ZSTD_LIB_MINIFY build option by @terrelln in #3366
  • [zdict] Fix static linking only include guards by @terrelln in #3372
  • check potential overflow of compressBound() by @Cyan4973 in #3362
  • decompressBound tests and fix by @Cyan4973 in #3373
  • Meson test fixups by @eli-schwartz in #3120
  • [pzstd] Fixes for Windows build by @terrelln in #3380
  • Windows MT layer bug fixes by @yoniko in #3364
  • Update Copyright Comments by @felixhandte in #3173
  • [docs] Clarify dictionary loading documentation by @terrelln in #3381
  • [build][cmake] Fix cmake with custom assembler by @terrelln in #3382
  • Pin actions/checkout Dependency to Specific Commit Hash by @felixhandte in #3384
  • Improve help/usage (-h, -H) formatting by @jonpalmisc in #3385
  • [cmake] Add noexecstack to compiler/linker flags by @terrelln in #3392
  • Fix -Wdocumentation by @terrelln in #3393
  • Support decompression of compressed blocks of size ZSTD_BLOCKSIZE_MAX by @Cyan4973 in #3399
  • spec update : require minimum nb of literals for 4-streams mode by @Cyan4973 in #3398
  • External matchfinder API by @embg in #3333
  • New ZSTD_CCtx_setCParams() entry point, to set all parameters defined in a ZSTD_compressionParameters structure by @Cyan4973 in #3403
  • Move deprecated annotation before static to allow C++ compilation for clang by @danlark1 in #3400
  • Optimal huff depth speed improvements by @daniellerozenblit in #3302
  • improve compression ratio of small alphabets by @Cyan4973 in #3391
  • Fix fuzzing with ZSTD_MULTITHREAD by @danlark1 in #3417
  • minor refactoring for timefn by @Cyan4973 in #3413
  • Add support for in-place decompression by @terrelln in #3421
  • fix when nb of literals is very small by @Cyan4973 in #3419
  • Deprecate advanced streaming functions by @embg in #3408
  • Disable Custom ASAN/MSAN Poisoning on MinGW Builds by @felixhandte in #3424
  • [tests] Fix version test determinism by @terrelln in #3422
  • Refactor timefn unit, restore support for clock_gettime() by @Cyan4973 in #3423
  • Fuzz on maxBlockSize by @daniellerozenblit in #3418
  • Fuzz the External Matchfinder API by @embg in #3437
  • Cap hashLog & chainLog to ensure that we only use 32 bits of hash by @terrelln in #3438
  • [versions-test] Work around bug in dictionary builder for older versions by @terrelln in #3436
  • added c89 build test to CI by @Cyan4973 in #3435
  • added cygwin tests to Github Actions by @Cyan4973 in #3431
  • Huffman refactor by @terrelln in #3434
  • Fix bufferless API with attached dictionary by @terrelln in #3441
  • Test PGO Builds by @felixhandte in #3442
  • Fix CLI Handling of Permissions and Ownership by @felixhandte in #3432
  • Fix -Wstringop-overflow warning by @terrelln in #3440
  • refactor : --rm ignored with stdout by @Cyan4973 in #3443
  • Fix sequence validation and seqStore bounds check by @daniellerozenblit in #3439
  • Fix ZSTD_estimate* and ZSTD_initCStream() docs by @embg in #3448
  • Fix 32-bit build errors in zstd seekable format by @daniellerozenblit in #3452
  • Fuzz large offsets through sequence compression api by @daniellerozenblit in #3447
  • [huf] Add generic C versions of the fast decoding loops by @terrelln in #3449
  • Provide more accurate error codes for busy-loop scenarios by @Cyan4973 in #3455
  • disable --rm on -o command by @Cyan4973 in #3450
  • [Bugfix] CLI row hash flags set the wrong values by @yoniko in #3457
  • [huf] Fix bug in fast C decoders by @terrelln in #3459
  • Disable status updates when stderr is not the console by @Cyan4973 in #3458
  • fix long offset resolution by @daniellerozenblit in #3460
  • Simplify 32-bit long offsets decoding logic by @terrelln in #3467
  • NEW CONTRIBUTORS:
  • @WojciechMula made their first contribution in #3013
  • @trixirt made their first contribution in #3026
  • @brailovich made their first contribution in #3027
  • @u1f35c made their first contribution in #3042
  • @shadchin made their first contribution in #3050
  • @ooosssososos made their first contribution in #3054
  • @TocarIP made their first contribution in #3075
  • @xry111 made their first contribution in #3084
  • @niamster made their first contribution in #3079
  • @dirkmueller made their first contribution in #3059
  • @cyberknight777 made their first contribution in #3088
  • @dpelle made their first contribution in #3095
  • @paulmenzel made their first contribution in #3108
  • @cuishuang made their first contribution in #3117
  • @averred made their first contribution in #3135
  • @JunHe77 made their first contribution in #3145
  • @htnhan made their first contribution in #3184
  • @udayanbapat made their first contribution in #3118
  • @zhuhan0 made their first contribution in #3205
  • @mgord9518 made their first contribution in #3218
  • @qiongsiwu made their first contribution in #3219
  • @orbea made their first contribution in #3217
  • @cgbur made their first contribution in #3215
  • @tomcwang made their first contribution in #3208
  • @mileshu made their first contribution in #3196
  • @zengyijing made their first contribution in #3226
  • @grossws made their first contribution in #3230
  • @wahern made their first contribution in #3241
  • @daniellerozenblit made their first contribution in #3258
  • @DimitriPapadopoulos made their first contribution in #3259
  • @sashashura made their first contribution in #3264
  • @haampie made their first contribution in #3247
  • @Tachi107 made their first contribution in #3252
  • @ctkhanhly made their first contribution in #3268
  • @MaskRay made their first contribution in #3273
  • @ilyakurdyukov made their first contribution in #3281
  • @ojeda made their first contribution in #3294
  • @GermanAizek made their first contribution in #3304
  • @joycebrum made their first contribution in #3309
  • @yiyuaner made their first contribution in #3300
  • @nmoinvaz made their first contribution in #3289
  • @jonpalmisc made their first contribution in #3385

New in Zstandard 1.5.2 (Feb 15, 2022)

  • Zstandard v1.5.2 is a bug-fix release, addressing issues that were raised with the v1.5.1 release.
  • In particular, as a side-effect of the inclusion of assembly code in our source tree, binary artifacts were being marked as needing an executable stack on non-amd64 architectures. This release corrects that issue. More context is available in #2963.
  • This release also corrects a performance regression that was introduced in v1.5.0 that slows down compression of very small data when using the streaming API. Issue #2966 tracks that topic.
  • In addition there are a number of smaller improvements and fixes.
  • FULL CHANGELIST:
  • Fix zstd-static output name with MINGW/Clang by @MehdiChinoune in #2947
  • storeSeq & mlBase : clarity refactoring by @Cyan4973 in #2954
  • Fix mini typo by @fwessels in #2960
  • Refactor offset+repcode sumtype by @Cyan4973 in #2962
  • meson: fix MSVC support by @eli-schwartz in #2951
  • fix performance issue in scenario #2966 (part 1) by @Cyan4973 in #2969
  • [meson] Explicitly disable assembly for non clang/gcc copmilers by @terrelln in #2972
  • Mark Huffman Decoder Assembly noexecstack on All Architectures by @felixhandte in #2964
  • Improve Module Map File by @felixhandte in #2953
  • Remove Dependencies to Allow the Zstd Binary to Dynamically Link to the Library by @felixhandte in #2977
  • [opt] Fix oss-fuzz bug in optimal parser by @terrelln in #2980
  • [license] Fix license header of huf_decompress_amd64.S by @terrelln in #2981
  • Fix stderr progress logging for decompression by @terrelln in #2982
  • Fix tar test cases by @sunwire in #2956
  • Fixup MSVC source file inclusion for cmake builds by @hmaarrfk in #2957
  • x86-64: Hide internal assembly functions by @hjl-tools in #2993
  • Prepare v1.5.2 by @felixhandte in #2987
  • Documentation and minor refactor to clarify MT memory management by @embg in #3000
  • Avoid updating timestamps when the destination is stdout by @floppym in #2998
  • [build][asm] Pass ASFLAGS to the assembler instead of CFLAGS by @terrelln in #3009
  • Update CI documentation by @embg in #2999
  • Zstandard v1.5.2 by @felixhandte in #2995