Possible concurrency issue during deflate on libflate 0.1.25

[mjc]

I'm using zip-rs with rayon to zip many files concurrently (I think 16x? Whatever rayon ends up using on this machine) This works fine with compression level store, and with bzip2. (Bzip2 is also way, way faster, 150+MB/s instead of 10MB/s, but that is.... another issue.)

All of the actual zipping is happening inside each loop iteration that is parallelized: open the source and destination files, io::copy, close the file when done.

When not using par_iter it completes successfully (but takes 14 hours to do so on a small test dataset of 283GB.)

With deflate (which uses this library,) I run into the below stack trace eventually, on win64.

It takes about an hour or so to run into the issue and doesn't seem to be on the same file every time, so reducing this test case seems difficult.

I've also gotten an illegal instruction once or twice which makes me think there is undefined behavior somewhere in this lib.

Thread '<unnamed>' panicked at 'index out of bounds: the len is 15 but the index is 15', /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\src\libcore\slice\mod.rs:2695:10
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread '<unnamed>' panicked at 'index out of bounds: the len is 15 but the index is 15', /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\src\libcore\slice\mod.rs:2695:10
stack backtrace:
   0:     0x7ff62b6a7aad - std::sys::windows::backtrace::set_frames
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\sys\windows\backtrace\mod.rs:95
   1:     0x7ff62b6a7aad - std::sys::windows::backtrace::unwind_backtrace
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\sys\windows\backtrace\mod.rs:82
   2:     0x7ff62b6a7aad - std::sys_common::backtrace::_print
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\sys_common\backtrace.rs:71
   3:     0x7ff62b6aacad - std::sys_common::backtrace::print
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\sys_common\backtrace.rs:59
   4:     0x7ff62b6aacad - std::panicking::default_hook::{{closure}}
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:197
   5:     0x7ff62b6aa9aa - std::panicking::default_hook
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:211
   6:     0x7ff62b6ab53f - std::panicking::rust_panic_with_hook
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:474
   7:     0x7ff62b6ab073 - std::panicking::continue_panic_fmt
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:381
   8:     0x7ff62b6aaf58 - std::panicking::rust_begin_panic
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:308
   9:     0x7ff62b6b8b1b - core::panicking::panic_fmt
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libcore\panicking.rs:85
  10:     0x7ff62b6b8ad9 - core::panicking::panic_bounds_check
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libcore\panicking.rs:61
  11:     0x7ff62b54505c - <core::iter::adapters::Rev<I> as core::iter::traits::iterator::Iterator>::try_fold::h4df698d468c8a2ab
  12:     0x7ff62b537e30 - <libflate::deflate::symbol::DynamicHuffmanCodec as libflate::deflate::symbol::HuffmanCodec>::save::h0b55c10f5385fd10
  13:     0x7ff62b54023a - libflate::deflate::encode::Encoder<W>::new::h14c0d67a7d5ae201
  14:     0x7ff62b540664 - libflate::deflate::encode::Encoder<W,E>::finish::h5e76f2387f1275f7
  15:     0x7ff62b53ca1e - zip::write::ZipWriter<W>::finalize::h7a22fb7f667ff968
  16:     0x7ff62b53bbc4 - zip::write::ZipWriter<W>::start_file::hae527b1388e644b5
  17:     0x7ff62b53be81 - zip::write::ZipWriter<W>::finalize::h7a22fb7f667ff968
  18:     0x7ff62b53da4e - <zip::write::ZipWriter<W> as core::ops::drop::Drop>::drop::h457d134e7a1b2002
  19:     0x7ff62b52c73f - core::ptr::real_drop_in_place::h8b27e2444c24fda3
  20:     0x7ff62b52ec4a - mame_coalesce::rom::zip::write_zip::hc9fe388a39a2a3b3
  21:     0x7fff553e102f - <unknown>
  22:     0x7fff553e3595 - is_exception_typeof
  23:     0x7fff553ebb23 - _C_specific_handler
  24:     0x7fff553e2fec - is_exception_typeof
  25:     0x7fff553ebfe0 - _CxxFrameHandler3
  26:     0x7fff6aec47fe - _chkstk
  27:     0x7fff6ae2600b - RtlUnwindEx
  28:     0x7fff553ebe48 - _C_specific_handler
  29:     0x7fff553e2688 - is_exception_typeof
  30:     0x7fff553e29e2 - is_exception_typeof
  31:     0x7fff553e30f2 - is_exception_typeof
  32:     0x7fff553ebfe0 - _CxxFrameHandler3
  33:     0x7fff6aec477e - _chkstk
  34:     0x7fff6ae24bee - RtlWalkFrameChain
  35:     0x7fff6ae289e5 - RtlRaiseException
  36:     0x7fff67d99128 - RaiseException
  37:     0x7fff553e486c - CxxThrowException
  38:     0x7ff62b6b0777 - panic_unwind::imp::panic
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libpanic_unwind\seh.rs:281
  39:     0x7ff62b6b0777 - panic_unwind::__rust_start_panic
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libpanic_unwind\lib.rs:101
  40:     0x7ff62b6ab717 - std::panicking::rust_panic
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:523
  41:     0x7ff62b6ab5ed - std::panicking::rust_panic_with_hook
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:494
  42:     0x7ff62b6ab073 - std::panicking::continue_panic_fmt
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:381
  43:     0x7ff62b6aaf58 - std::panicking::rust_begin_panic
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libstd\panicking.rs:308
  44:     0x7ff62b6b8b1b - core::panicking::panic_fmt
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libcore\panicking.rs:85
  45:     0x7ff62b6b8ad9 - core::panicking::panic_bounds_check
                               at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c\/src\libcore\panicking.rs:61
  46:     0x7ff62b54505c - <core::iter::adapters::Rev<I> as core::iter::traits::iterator::Iterator>::try_fold::h4df698d468c8a2ab
  47:     0x7ff62b537e30 - <libflate::deflate::symbol::DynamicHuffmanCodec as libflate::deflate::symbol::HuffmanCodec>::save::h0b55c10f5385fd10
  48:     0x7ff62b54023a - libflate::deflate::encode::Encoder<W>::new::h14c0d67a7d5ae201
  49:     0x7ff62b53fadd - <libflate::deflate::encode::Encoder<W,E> as std::io::Write>::write::h11ec20597a5b3e80
  50:     0x7ff62b53ab33 - std::io::Write::write_all::h9a21696f30eac3c4
  51:     0x7ff62b538631 - std::io::util::copy::h7fe9620ed861a5de
  52:     0x7ff62b52e3a2 - mame_coalesce::rom::zip::write_zip::hc9fe388a39a2a3b3

[sile]

Thank you for reporting this issue.
Could you share the code for reproducing this problem in my environment?

[Shnatsel]

Address Sanitizer and/or Thread Sanitizer could be used to pinpoint the memory safety issue, see https://github.com/japaric/rust-san

I'm not sure if they work on Windows, though.

[mjc]

Sure, this is the (rather admittedly terrible as I am quite new to rust) project I'm starting https://github.com/mjc/mame_coalesce. I can try and isolate it a bit more if you want.

I can try and reproduce it on Linux also if need be.

[mjc]

Trying ASan, MemorySanitizer, and ThreadSan now.

EDIT: none of the sanitizers detected anything, sadly.

[sile]

@mjc Thank you for your information! I will try to reproduce it on my local machine.

[Shnatsel]

zip crate has recently switched to miniz_oxide as the compression backend. @mjc has anything changed for your program? Is the issue still reproducible?

[mjc]

zip crate has recently switched to miniz_oxide as the compression backend. @mjc has anything changed for your program? Is the issue still reproducible?

excellent I'll retry today. also finally have a fast linux box to try this on too

[mjc]

On both Linux and windows I don't experience this bug with the new zip crate update that doesn't use this library.