|
kusano |
7d535a |
LZ4 - Extremely fast compression
|
|
kusano |
7d535a |
================================
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
LZ4 is lossless compression algorithm,
|
|
kusano |
7d535a |
providing compression speed at 400 MB/s per core,
|
|
kusano |
7d535a |
scalable with multi-cores CPU.
|
|
kusano |
7d535a |
It also features an extremely fast decoder,
|
|
kusano |
7d535a |
with speed in multiple GB/s per core,
|
|
kusano |
7d535a |
typically reaching RAM speed limits on multi-core systems.
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
Speed can be tuned dynamically, selecting an "acceleration" factor
|
|
kusano |
7d535a |
which trades compression ratio for more speed up.
|
|
kusano |
7d535a |
On the other end, a high compression derivative, LZ4_HC, is also provided,
|
|
kusano |
7d535a |
trading CPU time for improved compression ratio.
|
|
kusano |
7d535a |
All versions feature the same excellent decompression speed.
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
|Branch |Status |
|
|
kusano |
7d535a |
|------------|---------|
|
|
kusano |
7d535a |
|master | [![Build Status][travisMasterBadge]][travisLink] [![Build status][AppveyorMasterBadge]][AppveyorLink] [![coverity][coverBadge]][coverlink] |
|
|
kusano |
7d535a |
|dev | [![Build Status][travisDevBadge]][travisLink] [![Build status][AppveyorDevBadge]][AppveyorLink] |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
[travisMasterBadge]: https://travis-ci.org/Cyan4973/lz4.svg?branch=master "Continuous Integration test suite"
|
|
kusano |
7d535a |
[travisDevBadge]: https://travis-ci.org/Cyan4973/lz4.svg?branch=dev "Continuous Integration test suite"
|
|
kusano |
7d535a |
[travisLink]: https://ci.appveyor.com/project/YannCollet/lz4
|
|
kusano |
7d535a |
[AppveyorMasterBadge]: https://ci.appveyor.com/api/projects/status/v6kxv9si529477cq/branch/master?svg=true "Visual test suite"
|
|
kusano |
7d535a |
[AppveyorDevBadge]: https://ci.appveyor.com/api/projects/status/v6kxv9si529477cq/branch/dev?svg=true "Visual test suite"
|
|
kusano |
7d535a |
[AppveyorLink]: https://ci.appveyor.com/project/YannCollet/lz4
|
|
kusano |
7d535a |
[coverBadge]: https://scan.coverity.com/projects/4735/badge.svg "Static code analysis of Master branch"
|
|
kusano |
7d535a |
[coverlink]: https://scan.coverity.com/projects/4735
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
> **Branch Policy:**
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
> - The "master" branch is considered stable, at all times.
|
|
kusano |
7d535a |
> - The "dev" branch is the one where all contributions must be merged
|
|
kusano |
7d535a |
before being promoted to master.
|
|
kusano |
7d535a |
> + If you plan to propose a patch, please commit into the "dev" branch,
|
|
kusano |
7d535a |
or its own feature branch.
|
|
kusano |
7d535a |
Direct commit to "master" are not permitted.
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
Benchmarks
|
|
kusano |
7d535a |
-------------------------
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
The benchmark uses the [Open-Source Benchmark program by m^2 (v0.14.3)]
|
|
kusano |
7d535a |
compiled with GCC v4.8.2 on Linux Mint 64-bits v17.
|
|
kusano |
7d535a |
The reference system uses a Core i5-4300U @1.9GHz.
|
|
kusano |
7d535a |
Benchmark evaluates the compression of reference [Silesia Corpus]
|
|
kusano |
7d535a |
in single-thread mode.
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
| Compressor | Ratio | Compression | Decompression |
|
|
kusano |
7d535a |
| ---------- | ----- | ----------- | ------------- |
|
|
kusano |
7d535a |
| memcpy | 1.000 | 4200 MB/s | 4200 MB/s |
|
|
kusano |
7d535a |
|**LZ4 fast 17 (r129)**| 1.607 |**690 MB/s** | **2220 MB/s** |
|
|
kusano |
7d535a |
|**LZ4 default (r129)**|**2.101**|**385 MB/s** | **1850 MB/s** |
|
|
kusano |
7d535a |
| LZO 2.06 | 2.108 | 350 MB/s | 510 MB/s |
|
|
kusano |
7d535a |
| QuickLZ 1.5.1.b6 | 2.238 | 320 MB/s | 380 MB/s |
|
|
kusano |
7d535a |
| Snappy 1.1.0 | 2.091 | 250 MB/s | 960 MB/s |
|
|
kusano |
7d535a |
| LZF v3.6 | 2.073 | 175 MB/s | 500 MB/s |
|
|
kusano |
7d535a |
| zlib 1.2.8 -1 | 2.730 | 59 MB/s | 250 MB/s |
|
|
kusano |
7d535a |
|**LZ4 HC (r129)** |**2.720**| 22 MB/s | **1830 MB/s** |
|
|
kusano |
7d535a |
| zlib 1.2.8 -6 | 3.099 | 18 MB/s | 270 MB/s |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
Documentation
|
|
kusano |
7d535a |
-------------------------
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
The raw LZ4 block compression format is detailed within [lz4_Block_format].
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
To compress an arbitrarily long file or data stream, multiple blocks are required.
|
|
kusano |
7d535a |
Organizing these blocks and providing a common header format to handle their content
|
|
kusano |
7d535a |
is the purpose of the Frame format, defined into [lz4_Frame_format].
|
|
kusano |
7d535a |
Interoperable versions of LZ4 must respect this frame format.
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
Other source versions
|
|
kusano |
7d535a |
-------------------------
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
Beyond the C reference source,
|
|
kusano |
7d535a |
many contributors have created versions of lz4 in multiple languages
|
|
kusano |
7d535a |
(Java, C#, Python, Perl, Ruby, etc.).
|
|
kusano |
7d535a |
A list of known source ports is maintained on the [LZ4 Homepage].
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
|
|
kusano |
7d535a |
[Open-Source Benchmark program by m^2 (v0.14.3)]: http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029
|
|
kusano |
7d535a |
[Silesia Corpus]: http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
|
|
kusano |
7d535a |
[lz4_Block_format]: lz4_Block_format.md
|
|
kusano |
7d535a |
[lz4_Frame_format]: lz4_Frame_format.md
|
|
kusano |
7d535a |
[LZ4 Homepage]: http://www.lz4.org
|