-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--early-exit doesn't work as expected #243
Comments
) The --early-exit flag is now properly handled to ensure that Halmos terminates immediately after finding the first counterexample, avoiding the reporting of multiple counterexamples. The previous implementation used a busy-waiting loop with time.sleep(1) to wait for the first counterexample or all futures to complete, which was inefficient and allowed multiple counterexamples to accumulate. The updated code introduces an early_exit flag variable to keep track of whether an early exit is requested. When a counterexample is found and --early-exit is set, the early_exit flag is set to True. The future_callback function checks this flag and immediately returns if it is True, preventing further processing of results. The thread pool is shut down after all the submitted futures have completed or been canceled, ensuring a clean termination.
@karmacoma-eth I've fixed the issue (#284) by making the following changes:
Explanation of the issueThe previous implementation of the It used a busy-waiting loop with The updated code addresses these issues by introducing an With these changes, Halmos now terminates immediately after finding the first counterexample when the Here's the updated output for the example contract you provided: (.venv) 86fb4ce7caf3% python3 -m halmos --function test_manyCexes --early-exit
[⠊] Compiling...
[⠢] Compiling 28 files with 0.8.25
[⠔] Solc 0.8.25 finished in 2.15s
Compiler run successful!
Running 1 tests for test/Test60.t.sol:Test60
Counterexample:
p_x_uint256 = 0x8000000000000000000000000000000000000000000000000000000000000008 (57896044618658097711785492504343953926634992332820282019728792003956564819976)
[FAIL] test_manyCexes(uint256) (paths: 32, time: 0.08s, bounds: [])
Symbolic test result: 0 passed; 1 failed; time: 0.09s As you can see, with the I have opened a pull request (#284), please let me know if you have any further questions or concerns. |
) The --early-exit flag is now properly handled to ensure that Halmos terminates immediately after finding the first counterexample, avoiding unnecessary solver computations. The previous implementation allowed solver threads to continue running even after a counterexample was found, leading to inefficient resource usage and delayed termination. The updated code introduces the following changes: - A global list `solver_contexts` is introduced to store the solver contexts created in the `solve` function. - The `solve` function is modified to append the solver context to the global list before returning. - In the `future_callback` function, if `args.early_exit` is true and a counterexample is found, all the solver contexts in the global list are interrupted using `ctx.interrupt()`. - Exception handling is added to the interruption process to avoid segmentation faults. - After interrupting the solvers, the global list is cleared to prevent memory leaks. With these changes, when `args.early_exit` is true and a counterexample is found, the `future_callback` function interrupts all the running solver contexts, effectively stopping the unnecessary computations and allowing Halmos to terminate early.
…#243) The --early-exit flag is now properly handled to ensure that Halmos terminates immediately after finding the first counterexample, avoiding unnecessary solver computations. The previous implementation allowed solver threads to continue running even after a counterexample was found, leading to inefficient resource usage and delayed termination. The updated code introduces the following changes: - A global list `solver_contexts` is introduced to store the solver contexts created in the `solve` function. - The `solve` function is modified to append the solver context to the global list before returning. - The list of futures is stored in a variable before submitting them to the thread pool. - In the `future_callback` function, if `args.early_exit` is true and a counterexample is found: - All the solver contexts in the global list are interrupted using `ctx.interrupt()`. - The remaining futures are canceled using `future.cancel()` to stop unnecessary computations. - The global lists are cleared to prevent memory leaks. - Exception handling is added to the interruption process to avoid segmentation faults. With these changes, when `args.early_exit` is true and a counterexample is found, the `future_callback` function interrupts all the running solver contexts and cancels the remaining futures, effectively stopping the unnecessary computations and allowing Halmos to terminate early.
…#243) The --early-exit flag is now properly handled to ensure that Halmos terminates immediately after finding the first counterexample, avoiding unnecessary solver computations. The previous implementation allowed solver threads to continue running even after a counterexample was found, leading to inefficient resource usage and delayed termination. The updated code introduces the following changes: - A global list `solver_contexts` is introduced to store the solver contexts created in the `solve` function. - The `solve` function is modified to append the solver context to the global list before returning. - The list of futures is stored in a variable before submitting them to the thread pool. - In the `future_callback` function, if `args.early_exit` is true and a counterexample is found: - All the solver contexts in the global list are interrupted using `ctx.interrupt()`. - The remaining futures are canceled using `future.cancel()` to stop unnecessary computations. - The global lists are cleared to prevent memory leaks. - Exception handling is added to the interruption process to avoid segmentation faults. With these changes, when `args.early_exit` is true and a counterexample is found, the `future_callback` function interrupts all the running solver contexts and cancels the remaining futures, effectively stopping the unnecessary computations and allowing Halmos to terminate early.
Describe the bug
--early-exit
is supposed to print the first counterexample found and stop halmos, but we can actually still have multiple counterexamples reportedTo Reproduce
The text was updated successfully, but these errors were encountered: