-
Notifications
You must be signed in to change notification settings - Fork 82
relations of enumerated types (related to issue 45) #46
Comments
Fixed in commit ddb4f25. This is a bit risky, since it changes the way enumerated types are encoded in Z3, which might impact performance. If you notice any significant performance regressions, please let me know. |
Please re-open this issue because I'm seeing a regression, e.g. in ivy-master/examples/ivy/flash3.ivy Before Issue #46 release: ivy_check diagnose=true trace=true debug=true flash3.ivy OK After Issue #46 release: ivy_check diagnose=true trace=true debug=true flash3.ivy Trace follows...
flash3.ivy: line 61: cache.state(X) := invalid |
Fixed in commit cb17739. Thanks for catching that, Asgeir. |
Ran one of the larger test cases we have (4000 line model + 200 conjectures/invariants + 8 exports) and the test passes (this run didn't exercise any assert statements and I'll do that next) The run-time went from 45min to 16 hours and that's why I wanted to give you an early heads-up before the regression testing is complete |
Thanks, Asgeir. Is that compared to the version just before the fix for this issue (i.e., commit a7585aa)? I was using a Boolean encoding for enumerated types, but it couldn't handle functions or quantifiers over enumerated types. The fix was to switch to native Z3 enumerated types (that is, using inductive datatypes) but perhaps that is not well optimized in Z3. I could try bit vectors, which probably have received more attention. |
Yes, that’s correct. I am comparing the performance with the version just before the fix
… On Oct 30, 2019, at 11:10 AM, Ken McMillan ***@***.***> wrote:
Thanks, Asgeir. Is that compared to the version just before the fix for this issue (i.e., commit a7585aa <a7585aa>)?
I was using a Boolean encoding for enumerated types, but it couldn't handle functions or quantifiers over enumerated types. The fix was to switch to native Z3 enumerated types (that is, using inductive datatypes) but perhaps that is not well optimized in Z3. I could try bit vectors, which probably have received more attention.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#46?email_source=notifications&email_token=ADB2EOTKVEQT33ZVL5ID2KLQRHEYHA5CNFSM4JFI66G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECVHU7I#issuecomment-548043389>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADB2EOSPFIY6ZS2DVDJOOGLQRHEYHANCNFSM4JFI66GQ>.
|
It turns out I was running with z3 version 4.4.2 Once I fixed the PATH to pick up z3 version 4.7.1 the run time improved to 4 1/2 hours |
I have pushed a new version on branch Thanks. |
The run-time for the test case that took 4 1/2 hours with z3 enum is now down to about 112min
The cti also look Ok and I will test that part better tomorrow
I changed from MacOS 10.14 Mojave to 10.15 Catalina recently and tomorrow I’m going to try older ivy releases to verify I still see the 46min on Catalina 10.15 (aka that 10.15 is not causing some of the run-time increase)
… On Oct 30, 2019, at 6:20 PM, Ken McMillan ***@***.***> wrote:
I have pushed a new version on branch bv_enum. This version uses Z3 bit vectors to represent Ivy enumerated types. It passes the regression suite, but there are not many tests of enumerated types. Would you might running your tests on this version to look for both performance and functional regressions?
Thanks.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#46?email_source=notifications&email_token=ADB2EOVEYUNYRGQJRVTC6HLQRIXFXA5CNFSM4JFI66G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECWIMIA#issuecomment-548177440>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADB2EOSZJXRROG6Z5DED2K3QRIXFXANCNFSM4JFI66GQ>.
|
When I modify ivy-bv_enum/examples/pldi16/learning_switch.ivy Before: type packet After: type packet = {arp,ipv4,ipv6,udp,icmp} I get the following failure: |
Thanks, Asgeir. That is fixed on branch Let me know if the |
The performance is Ok i.e. when I run a benchmark test that has many enum but no relations of enum types the run-time is 35min for the baseline and 39min for the bv_enum branch When run a model with a relation of enum types I get an "error: Solver produced inconclusive result" and I think I'm recreating this by modifying the learning_switch.ivy example command line: ivy_check diagnose=true trace=true learning_switch.ivy With the learning_switch.ivy example modified as below produces the same error, but runs without issue using the ivy version that uses the Z3 enum type facility bash-3.2$ diff -w /Users/asgeir/Downloads/ivy-bv_enum-2/examples/pldi16/learning_switch.ivy /Users/asgeir/Downloads/ivy-master/examples/pldi16/learning_switch.ivy
|
Thanks. I can repro that but unfortunately not using smtlib2, so it's a bit hard to submit an issue to Z3. I'm checking whether Z3 actually intends to be a decision procedure for EPR with bit vectors. If not, I have to use a different encoding. |
Please refer to Issue #45
After the fix there is still a failure, i.e. when I change the following (that passes)
type packet
type node
to the following:
type packet
type node = {bridge,router}
With same command line as in #45 I now get
..
Initialization must establish the invariant
learning_switch.ivy: line 24: route(N:node) ... PASS
learning_switch.ivy: line 25: route(N:node) ... PASS
learning_switch.ivy: line 26: route(N:node) ... FAIL
searching for a small model... done
warning: model doesn't give a value for enumerated term @n. returning bridge.
warning: model doesn't give a value for enumerated term @n. returning bridge.
Trace follows...
The text was updated successfully, but these errors were encountered: