Accessing API interfaces is very slow

[zhy827827]

version:
parity/polkadot:stable2409-2
parity/substrate-api-sidecar:v19.3.1

Sidecar logs:

2024-11-25 09:07:11 http: GET /blocks/head 200 119623ms
2024-11-25 09:07:11 http: GET /blocks/head 200 119702ms
2024-11-25 09:07:11 http: GET /blocks/head 200 119785ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 71026ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 65072ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 65076ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 65081ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 55210ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 55216ms
2024-11-25 09:07:11 http: GET /blocks/head/header 200 51966ms
2024-11-25 09:07:13 http: GET /blocks/23528703 200 137230ms
2024-11-25 09:07:13 http: GET /blocks/23528708 200 137934ms
2024-11-25 09:07:14 http: GET /blocks/23528710 200 179144ms
2024-11-25 09:07:15 http: GET /blocks/23528710 200 139593ms
2024-11-25 09:07:16 http: GET /blocks/23528602 200 181166ms
2024-11-25 09:07:17 http: GET /blocks/23528602 200 141961ms
2024-11-25 09:07:19 http: GET /blocks/23528697 200 143169ms
2024-11-25 09:07:20 http: GET /blocks/23528605 200 144398ms
2024-11-25 09:07:21 http: GET /blocks/23528709 200 145790ms
2024-11-25 09:07:23 http: GET /blocks/23528601 200 187932ms
2024-11-25 09:07:24 http: GET /blocks/23528601 200 149000ms
2024-11-25 09:07:26 http: GET /blocks/23528596 200 190971ms
2024-11-25 09:07:28 http: GET /blocks/23528596 200 152177ms
2024-11-25 09:07:29 http: GET /blocks/23528699 200 154029ms
2024-11-25 09:07:31 http: GET /blocks/23528603 200 196316ms
2024-11-25 09:07:33 http: GET /blocks/23528603 200 157874ms
2024-11-25 09:07:35 http: GET /blocks/23528570 200 159995ms
2024-11-25 09:07:36 http: GET /blocks/23528684 200 160560ms
2024-11-25 09:07:37 http: GET /blocks/23528692 200 161130ms
2024-11-25 09:07:37 http: GET /blocks/23528689 200 161708ms
2024-11-25 09:07:40 http: GET /blocks/23528675 200 164141ms
2024-11-25 09:07:40 http: GET /blocks/23528705 200 164722ms
2024-11-25 09:07:43 http: GET /blocks/23528598 200 207573ms
2024-11-25 09:07:45 http: GET /blocks/23528598 200 169652ms
2024-11-25 09:07:48 http: GET /blocks/23528693 200 172111ms
2024-11-25 09:07:48 http: GET /blocks/23528683 200 172719ms
2024-11-25 09:07:49 http: GET /blocks/23528706 200 173359ms
2024-11-25 09:07:49 http: GET /blocks/23528685 200 174008ms
2024-11-25 09:07:50 http: GET /blocks/23528679 200 174620ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39019ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39013ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39013ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39007ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39008ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39003ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 39003ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 38999ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 38995ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 38996ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 7ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 8ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 8ms
2024-11-25 09:07:50 http: GET /blocks/head/header 200 9m

[Imod7]

Thank you for reporting this @zhy827827 ! This is a known issue and the source cause is the same as in #1554. The problem occurs when the queried blocks contain an unusually high volume of extrinsics. We are currently evaluating different approaches to address this performance bottleneck and we will update you once we have concrete next steps and action items. Thank you for your patience.

[Imod7]

@zhy827827

Please expect a possible decline in Sidecar performance starting at approximately 14:00 UTC today and lasting for around 40 minutes. This is due to an expected Spammening event on the Polkadot network.

During this period, Sidecar endpoints like blocks, staking-info and others may be affected, as we expect a high volume of 2.000-5.000 transactions per block.

To mitigate the potential performance impact, we recommend the following:

• Increase Sidecar's Heap Memory by using the --max-old-space-size flag in your process startup command.
• Use the noFees=true and finalizedKey=false flags for the blocks endpoint to reduce the data payload.

Related docs:
Showcasing Polkadot’s Capabilities: The Spammening

[zhy827827]

@Imod7
thank you very much