Added support for hashmaps in Smt and SimpleSmt

[polydez]

After benchmarking of compute_mutations and apply_mutations we noticed poor performance of multiple key-value insertions into BTreeMap which is used in our SMT implementations. Rewriting to hashbrown's HashMap (which supports no-std) gave us more than 10x boost in apply_mutations operation for large trees.

---

apply_mutations/SimpleSmt: apply_mutations/10000
time: [154.52 ms 163.09 ms 174.46 ms]
change: [-93.242% -92.753% -92.142%] (p = 0.00 < 0.05)
Performance has improved.

---

(More context in the PR: #355)

In this implementation we also introduced smt_hashmaps feature. If it's switched off, the SMT uses binary-tree implementation, as before. This might be useful for backward-compatibility with code, which relies on entry ordering in SMTs.

[bobbinth]

@polydez - merging the latest next branch into this broke it somehow. Could you take a look?

[polydez]

@bobbinth, now it works!

[bobbinth]

Thank you! Looks good! I've reviewed all non-test code so far and left some comments inline.

I also ran the benchmarks (1M tree size and 1K insertions) and got about 20% improvement:

Using BTreeMap

Constructed a SMT with 1000000 key-value pairs in 41.5 seconds
Number of leaf nodes: 1000000

Running an insertion benchmark:
An average insertion time measured by 1000 inserts into an SMT with 1000000 leaves is 404 μs

Running a batched insertion benchmark:
An average insert-batch computation time measured by a 1000-batch into an SMT with 1001000 leaves over 405.3 ms is 405 μs
An average insert-batch application time measured by a 1000-batch into an SMT with 1001000 leaves over 32.5 ms is 32 μs
An average batch insertion time measured by a 1k-batch into an SMT with 1001000 leaves totals to 437.8 ms

Running a batched update benchmark:
An average update-batch computation time measured by a 1000-batch into an SMT with 1002000 leaves over 402.0 ms is 402 μs
An average update-batch application time measured by a 1000-batch into an SMT with 1002000 leaves over 28.1 ms is 28 μs
An average batch update time measured by a 1k-batch into an SMT with 1002000 leaves totals to 430.1 ms

Running a proof generation benchmark:
An average proving time measured by 100 value proofs in an SMT with 1001807 leaves in 9 μs

Using Hashbrown

Constructed a SMT with 1000000 key-value pairs in 42.1 seconds
Number of leaf nodes: 1000000

Running an insertion benchmark:
An average insertion time measured by 1000 inserts into an SMT with 1000000 leaves is 367 μs

Running a batched insertion benchmark:
An average insert-batch computation time measured by a 1000-batch into an SMT with 1001000 leaves over 360.1 ms is 360 μs
An average insert-batch application time measured by a 1000-batch into an SMT with 1001000 leaves over 4.8 ms is 5 μs
An average batch insertion time measured by a 1k-batch into an SMT with 1001000 leaves totals to 364.9 ms

Running a batched update benchmark:
An average update-batch computation time measured by a 1000-batch into an SMT with 1002000 leaves over 370.1 ms is 370 μs
An average update-batch application time measured by a 1000-batch into an SMT with 1002000 leaves over 4.2 ms is 4 μs
An average batch update time measured by a 1k-batch into an SMT with 1002000 leaves totals to 374.2 ms

Running a proof generation benchmark:
An average proving time measured by 100 value proofs in an SMT with 1001806 leaves in 0 μs

[polydez]

I also ran the benchmarks (1M tree size and 1K insertions) and got about 20% improvement:

I tested only application of mutation set, which does a lot of inserts/updates and in your benchmark this operation gave us up to ~7x improvement. Computation time is still slow, but I hope, paralleling of the computation can give us significant speed-up.

[bobbinth]

Looks good! Thank you! There are 2 more things we should do:

• Update the "Crate features" section in the README to mention smt_hashmaps feature (I noticed that concurrent feature description is also missing there).
• Probably update the Makefile to make sure we also run test with smt_hashmaps feature enabled.

[polydez]

• Update the "Crate features" section in the README to mention smt_hashmaps feature (I noticed that concurrent feature description is also missing there).

I will update the list, thank you for noticing! But what about concurrent feature? I can see it in the list (first item):

crypto/README.md

Line 63 in a797a9e

- `concurrent`- enabled by default; enables multi-threaded implementation of `Smt::with_entries()` which significantly improves performance on multi-core CPUs.

[bobbinth]

But what about concurrent feature? I can see it in the list (first item)

Ah yes! I was looking at an old version of the file.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[bobbinth]

All looks good! Thank you!