-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: remove Types
collection from wdl-analysis
.
#277
Conversation
This commit removes the `Types` collection from `wdl-analysis`. The collection served as an arena to allocate types into which also served as providing type recursion via arena identifiers. The hope was that this would prevent a lot of small allocations from occurring during analysis when calculating expression types. However, each document had a different `Types` collection, requiring "importing" types between collections. Further complicating things, an evaluation "engine" had its own `Types` collection which needed importing of types from other document collections. This was a source of bugs in the implementation of the evaluation engine. Worse, the types collection served as a bottleneck for sharing data across asynchronous tasks in the upcoming implementation of workflow evaluation, requiring a lock. The alternative is to use an `Arc` in appropriate places where type recursion occurs. After running some simple benchmarks, the removal of the types arena doesn't appear to have any real world impact on performance. Additionally, the removal of `Types` allows `Type` to implement `PartialEq` rather than a custom trait that uses the arena; likewise for `Display`. Similarly in `wdl-engine`, `Value` can now directly implement serialization and deserialization traits.
Currently, the `Task` variant of the `Value` enum causes `Value` to be about 200 bytes in size due to its numerous fields. This change puts the immutable fields of a task value behind an indirection, reducing the size of `Value` to 40 bytes. It also puts the `left` and `right` members of `Pair` behind a single allocation rather than two.
Types
collection from wdl-analysis
.
No rush on this review as I'm out this week and next, but the workflow evaluation implementation will be relying on these changes so that we can more easily spawn asynchronous tasks that do WDL evaluation internally (i.e. we no longer have to synchronize a singular |
Looks like gauntlet caught a bug for me to track down too. |
So the gauntlet diff is actually removing false positives fixed by this PR. The actual bug was in the now-deleted For example: bar.wdl: version 1.1
task foo {
input {
Array[String]? bar
}
command <<<>>>
} foo.wdl: version 1.1
import "bar.wdl"
workflow bar {
input {
Array[String]? bar
}
call bar.foo { input: bar = bar }
} Results in this false positive diagnostic:
I'll push a bless of the gauntlet diagnostics to rid the false positives and add a test case to ensure we don't accidentally drop the optional qualifier like this in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Also fix a comment for the `Coercible` trait in `wdl-engine`.
This commit removes the
Types
collection fromwdl-analysis
.The collection served as an arena to allocate types into which also served as providing type recursion via arena identifiers. The hope was that this would prevent a lot of small allocations from occurring during analysis when calculating expression types.
However, each document had a different
Types
collection, requiring "importing" types between collections. Further complicating things, an evaluation engine had its ownTypes
collection which needed importing of types from other document collections. This was a source of bugs in the implementation of the evaluation engine.Worse, the types collection served as a bottleneck for sharing data across asynchronous tasks in the upcoming implementation of workflow evaluation, requiring a lock.
The alternative is to use an
Arc
in appropriate places where type recursion occurs. After running some simple benchmarks, the removal of the types arena doesn't appear to have any real world impact on performance.Additionally, the removal of
Types
allowsType
to implementPartialEq
rather than a custom trait that uses the arena; likewise forDisplay
.Similarly in
wdl-engine
,Value
can now directly implement serialization and deserialization traits.Before submitting this PR, please make sure:
changes (when appropriate).
CHANGELOG.md
(see"keep a changelog" for more information).