You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the current implementation of metrics (#18), metrics are collected and aggregated from all tasks involved at the completion of a session.Run. This has a few drawbacks. First, all tasks may not be available at the end of session.Run (e.g., due to failed machines). Second, it ties stats collection to evaluation: we may in the future, persist task outputs to, e.g., s3, and clear task outputs from individual machines.
The solution to this is clear: we can flow Scopes along with task outputs; the executor can then aggregate them as the data flows.
The text was updated successfully, but these errors were encountered:
In the current implementation of metrics (#18), metrics are collected and aggregated from all tasks involved at the completion of a
session.Run
. This has a few drawbacks. First, all tasks may not be available at the end ofsession.Run
(e.g., due to failed machines). Second, it ties stats collection to evaluation: we may in the future, persist task outputs to, e.g., s3, and clear task outputs from individual machines.The solution to this is clear: we can flow Scopes along with task outputs; the executor can then aggregate them as the data flows.
The text was updated successfully, but these errors were encountered: