bump min python to 3.8, and migrate to aiorpcx 0.22 #7661
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR
Re the aiorpcx changes, note that the external API of that library changed.
Most notable is that when using a TaskGroup in an
async with
statement, exceptions encountered by enclosed tasks are not re-raised in__aexit__
. Instead, it is now the caller's responsibility to inspect the results and exceptions of the tasks.For old code that looks like this:
kyuupichan and upstream curio recommends to write instead (for new code):
Instead, in this PR I introduce class
OldTaskGroup(aiorpcx.TaskGroup)
, which retains the old behaviour of TaskGroup.see
TaskGroup(wait=all)
, weird behaviour and API ofcompleted
kyuupichan/aiorpcX#43Also note that there is a timing issue/race with the
timeout_after
andignore_after
APIs of aiorpcx. I think the issue has been there for a long time - the changes in TaskGroup simply exposed it.Consider example:
When the 0.1 sec timeout expires,
inner_task
will get cancelled bytimeout_after
(=internal cancellation).If around the same time (in terms of event loop iterations) another coroutine cancels
outer_task
(=external cancellation), there will be a race.Both cancellations work by propagating a
CancelledError
out totimeout_after
, which thenneeds to decide (in
TimeoutAfter.__aexit__
) whether it's due to an internal or external cancellation.AFAICT asyncio provides no reliable way of distinguishing between the two.
This patch tries to always give priority to external cancellations.
see
ignore_after
cannot be cancelled sometimes (race?) kyuupichan/aiorpcX#44see aiohttp swallows asyncio.CancelledError during connection timeout aio-libs/async-timeout#229
see https://bugs.python.org/issue42130 and https://bugs.python.org/issue45098
I believe the problem became more relevant with new aiorpcx due to changes to
TaskGroup.join()
(which is called in__aexit__
) andTaskGroup.cancel_remaining()
in particular. When the group encounters an exception in a task, all other tasks are cancelled. Old aiorpcx used to not wait for the cancellations to finish, but new aiorpcx blocks until the cancellations are done. If the timing race occurs, a cancellation gets suppressed, andcancel_remaining()
blocks indefinitely.To fix this, this PR is monkey-patching aiorpcx to include a workaround for the race.