Release Notes: This release introduces the capability to do Global Deduplication. This performs deduplication across the entire dataset using an in-memory index as opposed to deduplication only within segments. Two kinds of indexes are used based on the dataset size. A full chunk hash index is used for small datasets. A special segmented similarity based index is used when the dataset is very large. The latter index size is just 0.002% of the dataset size with >90% efficiency of a full chunk index based exact dedupe using 4KB chunks. Streaming support allows optimization of network transfer of large data.
Release Notes: This release adds many improvements and fixes, including ones for performance and stability. It adds the KECCAK Sha3 message digest. A new fast Delta variant detects embedded tables of binary numeric data and RLE encodes them. A matrix transform allows better compression of a Dedupe index. LZ4 and XXHash have been updated. The test suite has been expanded. Pcompress now builds without warnings with strict compiler flags. Alternate locations for external libraries are handled properly, and older OpenSSL versions up to 0.9.8e work. The debug statistics mode now prints additional throughput statistics.
Release Notes: This is the first stable release of Pcompress. An extensive test suite has been added and used to validate and fix bugs. The file format is versioned, and backward compatiblity will be maintained. Deduplication parameters have been tweaked to get better distribution of content-aware blocks and an improved dedupe ratio. Adaptive compression modes have been improved with more heuristics. This release also provides best-in-class SHA256 digest performance on x86 platforms. Pcompress today provides a combination of features and performance which makes it unique among data compression utilities.
Release Notes: Initial alpha release.