Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[runtime] Enhancement: Add id map to I/O queue table #1498

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

[runtime] Enhancement: Add id map to I/O queue table #1498

wants to merge 2 commits into from

Conversation

iyzhang
Copy link
Contributor

@iyzhang iyzhang commented Feb 5, 2025

This PR adds an id map to the I/O queue table, instead of our previous implementation that just started at 500. We retain the behavior for direct mapping by having the id map start allocating at 500.

@iyzhang iyzhang requested a review from anandbonde February 5, 2025 20:16
@iyzhang iyzhang self-assigned this Feb 5, 2025
@iyzhang
Copy link
Contributor Author

iyzhang commented Feb 5, 2025

closes #1497

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 5, 2025

libos = catpowder
commit id = a46d59a

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::inetstack::poll;inetstack::layer4::poll_once 20773645 661 265 12688512380 46 12680202968 85
bgc::inetstack::poll 10386822 1530 612 14851467379 54 2162954999 14
ioc::network::libos::push 16145 2969 1211 48370825 0 48370825 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch 5858 2504 983 8450249 0 8448041 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter 11620 417 168 5671107 0 5671107 0
ioc::network::libos::pop 9232 443 178 3457974 0 3457974 0
bgc::inetstack::tcp::established::background 11635 1758 699 13075866 0 3300484 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger 11635 618 247 3168689 0 3168689 0
ioc::network::libos::pushto 673 4399 1771 2949722 0 2949722 0
ioc::network::libos::connect 42 42327 17259 1640771 0 1640771 0
demikernel::sgaalloc 6502 150 61 963307 0 963307 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender 11576 870 336 935585 0 935585 0
demikernel::sgafree 8748 113 45 650894 0 650894 0
bgc::inetstack::tcp::passiveopen::background 42 16628 6561 407804 0 407804 0
ioc::network::libos::close 47 6137 2466 278000 0 278000 0
ioc::network::libos::accept 39 2388 942 51794 0 51794 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;udp::receive 200 213 84 43415 0 43415 0
bgc::inetstack::arp::background 7 2308 919 15420 0 15420 0
bgc::inetstack::icmp::background 1 2608 1040 2848 0 2848 0

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 5, 2025

libos = catnip
commit id = a46d59a

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::inetstack::poll;inetstack::layer4::poll_once;catnip::runtime::receive 27737242 208 84 5864309311 20 5864309311 45
bgc::inetstack::poll;inetstack::layer4::poll_once 27737242 362 147 9880561759 34 4011645455 31
bgc::inetstack::poll 13868621 928 375 12852607415 45 2972045656 23
ioc::network::libos::push 16145 1214 497 17435440 0 15288600 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch 2592 3153 1332 4685076 0 4659810 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter 11057 343 139 4584627 0 4583829 0
ioc::network::libos::pushto 2931 1419 610 4077489 0 3298202 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger 11069 287 115 2863547 0 2860898 0
bgc::inetstack::tcp::established::background 11069 1075 432 10949965 0 2798705 0
ioc::network::libos::pop 9093 385 157 2532122 0 2532122 0
ioc::network::libos::push;catnip::runtime::transmit 8763 327 134 2330855 0 2330855 0
ioc::network::libos::connect 42 36557 15005 1473313 0 1417634 0
ioc::network::libos::pushto;catnip::runtime::transmit 2931 286 123 779287 0 779287 0
demikernel::sgafree 8876 89 36 701247 0 701247 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender 11014 324 127 703085 0 692885 0
demikernel::sgaalloc 6643 135 55 659090 0 659090 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;udp::receive 1369 257 107 290689 0 290689 0
bgc::inetstack::tcp::passiveopen::background 42 32975 13217 339925 0 288587 0
ioc::network::libos::close 49 6009 2430 258616 0 258616 0
ioc::network::libos::connect;catnip::runtime::transmit 42 5508 2238 55679 0 55679 0
bgc::inetstack::tcp::passiveopen::background;catnip::runtime::transmit 21 15879 6376 51338 0 51338 0
ioc::network::libos::accept 37 2368 956 45787 0 45787 0
bgc::inetstack::arp::background 7 2441 987 20696 0 20696 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger;catnip::runtime::transmit 20 982 416 16484 0 16484 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;catnip::runtime::transmit 24 578 233 11048 0 11048 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender;catnip::runtime::transmit 21 553 225 10200 0 10200 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter;catnip::runtime::transmit 10 765 328 4967 0 4967 0
bgc::inetstack::icmp::background 1 2352 950 2517 0 2517 0

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

libos = catnap
commit id = a46d59a

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::catnap::transport::epoll 13245180 3865 1589 6977076864 100 6977076864 100
ioc::network::libos::push 17600 497 215 7711587 0 7711587 0
ioc::network::libos::pop 15317 465 200 5505649 0 5505649 0
ioc::network::libos::connect 50 146882 61978 2694090 0 2694090 0
demikernel::sgaalloc 11286 126 52 1170220 0 1170220 0
demikernel::sgafree 11536 77 33 901198 0 901198 0
ioc::network::libos::pushto 1193 390 160 427819 0 427819 0
ioc::network::libos::accept 26 11664 5037 370224 0 370224 0
ioc::network::libos::close 18 23886 10374 336371 0 336371 0

anandbonde
anandbonde reviewed Feb 10, 2025
View reviewed changes
src/rust/collections/id_map/direct_map.rs
// Don't need to pre-allocate, the overhead is a 6ns on the scheduler insert benchmark.
_phantom: PhantomData,
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment can be removed.

anandbonde
anandbonde reviewed Feb 10, 2025
View reviewed changes
src/rust/collections/id_map/direct_map.rs
// Constants
//======================================================================================================================

// Value that we use to offset direct mapping for 32 bit ids. This number is chosen ran
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly incomplete comment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anandbonde anandbonde anandbonde left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@iyzhang iyzhang

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@iyzhang @anandbonde