ISA-L compression support evaluation.#1908
ISA-L compression support evaluation.#1908rhpvorderman wants to merge 4 commits intosamtools:developfrom
Conversation
|
ISA-L decompression is also slightly faster: Just 10% though and this is on a intel i7 6700. This is not as clear cut as the compression. |
|
I can confirm that the same sort of results occur when using a ryzen 5 3600 (AMD Zen 2). 10% faster reading and almost twice as fast writing. So it works equally well on both x86 vendors. Since we have a cluster with both AMD Epyc and Intel Xeon I always check if optimizations are not detrimental. I don't have any high performing ARM64 devices to test. The latest release of ISA-L is available on conda (https://github.com/conda-forge/isa-l-feedstock) for ARM64 and MacOS ARM64 architectures. If anyone wants to test. |
This is a quick test for re-evaluation of ISA-L support
see #1780
I ran samtools view on the current repository using libdeflate:
And then using the below patch with ISA-L.
This is significantly faster (slightly less than 2X) with a marginally larger filesize. And this is only the compression, I haven't tested decompression yet.
This needs to be properly integrated in the build system if this is going to be properly supported, so for now it is just hacked in to allow for benchmarks. (Install libisal-dev to compile on linux).