feat: create a subdirectory per mode in debug/

[sayon]

What ❔

Merging this PR requires merging matter-labs/era-compiler-llvm-context#24 first.

Support creating subfolders in debug/.

Why ❔

Compiler tester overrides IR dumps if they were produced by launching a test repeatedly with different modes.

This change should not break behavior expected from era-compiler-llvm-context.

Initially I wanted to just encode modes in the filenames but it turned out to be very intrusive to the code base (for one, dumping .yul files is done by a very different code than dumping everything else). Therefore we are creating a subdirectory per mode.

[sayon]

Note: run CI after merging matter-labs/era-compiler-llvm-context#24 and before merging this PR.

[hedgar2017]

Address the 2nd comment as well please.

[hedgar2017]

@sayon thanks!
Please note that you have to do cargo update for zksolc and zkvyper as well, so they depend on the correct commit of era-compiler-llvm-context, and then do cargo update here in the tester so it depends on everything up-to-date.

[hedgar2017]

Address the 2nd comment as well please.

@sayon have you checked locally before commiting? There are compile errors.

[github-actions]

Benchmark results:

╔═╡ Size (-%) ╞════════════════╡ All M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════════════╡ All M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════════════╡ All M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════════════╡ All MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════════════╡ All MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════════════╡ All MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════╡ Precompiles M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════╡ Precompiles M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════╡ Precompiles M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞════════╡ Precompiles MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞══════╡ Precompiles MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞════════╡ Precompiles MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞══════════╡ Real life M3B3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞════════╡ Real life M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞══════════╡ Real life M3B3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

╔═╡ Size (-%) ╞══════════╡ Real life MzB3 ╞═╗
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Cycles (-%) ╞════════╡ Real life MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╠═╡ Ergs (-%) ╞══════════╡ Real life MzB3 ╞═╣
║ Mean                                0.000 ║
║ Best                                0.000 ║
║ Worst                               0.000 ║
║ Total                               0.000 ║
╚═══════════════════════════════════════════╝

[hedgar2017]

Can be some weird CI issue.
Let's see why benchmark works and default fails.

[sayon]

I see the problem! It worked indeed on my machine, but I was using a patch in Cargo.toml to override the dependency on llvm context.

Should I manually adjust the commit hash of era-compiler-llvm-context in Cargo.lock and commit it? Or do cargo update and commit it?

As for CI: Maybe CI does "cargo update" before launching tests? Therefore it has Cargo.lock with a newer dependency, not an obsolete reference to a previous commit.

[hedgar2017]

I see the problem! It worked indeed on my machine, but I was using a patch in Cargo.toml to override the dependency on llvm context.

Should I manually adjust the commit hash of era-compiler-llvm-context in Cargo.lock and commit it? Or do cargo update and commit it?

As for CI: Maybe CI does "cargo update" before launching tests? Therefore it has Cargo.lock with a newer dependency, not an obsolete reference to a previous commit.

No, CI shouldn't do it. Please do cargo update and push Cargo.lock!

[hedgar2017]

Apparently you're editing Cargo.lock manually. This is not a common practice as this file is supposed to be managed by cargo. You need to specify branches in Cargo.toml, do cargo update and push Cargo.lock.

Please read the cargo docs on this before proceeding.