Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add our own versions of FuturesUnordered and FuturesOrdered #2798

Merged
merged 6 commits into from
Aug 19, 2024

Conversation

hrxi
Copy link
Contributor

@hrxi hrxi commented Aug 2, 2024

Unlike their futures_util::stream::* counterparts, they wake themselves when some future is pushed.

This gets rid of a couple of manual waker instances.

It also fixes some waking-related bugs (CC #2550).

Use clippy to warn for usages of the old types.

@hrxi hrxi force-pushed the hrxi/futures_unordered branch from e0ee318 to 3948e91 Compare August 2, 2024 13:34
Copy link
Member

@jsdanielh jsdanielh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we're still using the futures's version in consensus/src/sync/syncer.rs.

@hrxi hrxi force-pushed the hrxi/futures_unordered branch 2 times, most recently from 57c33ba to e2f0ae0 Compare August 14, 2024 14:33
Copy link
Member

@Eligioo Eligioo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some minor feedback points.

CC #2832 to use this new abstraction once this PR is in.

consensus/src/sync/live/state_queue/mod.rs Outdated Show resolved Hide resolved
consensus/src/sync/live/state_queue/mod.rs Outdated Show resolved Hide resolved
consensus/src/sync/peer_list.rs Outdated Show resolved Hide resolved
@hrxi hrxi force-pushed the hrxi/futures_unordered branch 2 times, most recently from 2fd2b5a to f2441da Compare August 15, 2024 13:46
@hrxi hrxi mentioned this pull request Aug 16, 2024
3 tasks
utils/src/stream.rs Show resolved Hide resolved
utils/src/stream.rs Show resolved Hide resolved
utils/src/lib.rs Show resolved Hide resolved
@@ -568,8 +577,23 @@ impl<N: Network> Stream for StateQueue<N> {
}
}

// Check if we have peers.
if self.nonempty_peers.is_none() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to use get_or_insert?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only inserts something if self.chunk_request_component.notify_nonempty_peers() returns a non-None value. This doesn't fit Option::get_or_insert's function signature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notify_nonempty_peers() can return None thus the following assignment can be to None again, which get_or_insert does not allow for.

.map(|notify| Box::pin(async move { notify.notified().await }) as BoxFuture<()>);
}
if let Some(nonempty_peers) = &mut self.nonempty_peers {
if nonempty_peers.as_mut().poll(cx).is_ready() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just take it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only want to remove the future from the member variable when it resolves. Option::take can't do that.

consensus/src/sync/peer_list.rs Outdated Show resolved Hide resolved
@hrxi hrxi force-pushed the hrxi/futures_unordered branch from f2441da to f3744bd Compare August 16, 2024 13:18
consensus/src/sync/live/state_queue/mod.rs Outdated Show resolved Hide resolved
consensus/src/sync/peer_list.rs Show resolved Hide resolved
consensus/src/sync/peer_list.rs Show resolved Hide resolved
@@ -568,8 +577,23 @@ impl<N: Network> Stream for StateQueue<N> {
}
}

// Check if we have peers.
if self.nonempty_peers.is_none() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notify_nonempty_peers() can return None thus the following assignment can be to None again, which get_or_insert does not allow for.

consensus/src/sync/sync_queue.rs Outdated Show resolved Hide resolved
consensus/src/sync/sync_queue.rs Outdated Show resolved Hide resolved
@hrxi hrxi force-pushed the hrxi/futures_unordered branch 3 times, most recently from 6c66ab0 to 86b7fe0 Compare August 16, 2024 15:50
@jsdanielh
Copy link
Member

Rebasing branch to merge it

hrxi added 6 commits August 19, 2024 13:38
Unlike their `futures_util::stream::*` counterparts, they wake themselves when
some future is pushed.
This gets rid of a couple of manual waker instances.

It also fixes some waking-related bugs (CC #2550).
This fixes a waking bug in the `Stream` implementation of `StateQueue`,
as it checked for a nonempty peer list but did not register a waker for
it.

CC #2550
@jsdanielh jsdanielh force-pushed the hrxi/futures_unordered branch from 86b7fe0 to 8171e0c Compare August 19, 2024 11:39
@jsdanielh jsdanielh merged commit 8171e0c into albatross Aug 19, 2024
7 checks passed
@jsdanielh jsdanielh deleted the hrxi/futures_unordered branch August 19, 2024 13:03
@jsdanielh jsdanielh added this to the Nimiq PoS Mainnet milestone Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants