Add request queue with limits #13
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
willmmiles
force-pushed
the
response_queue
branch
from
ed9014b to
96bc0e4
Compare
Add a very simple request queue to the web server core. This can be used to limit the amount of memory consumed by the web server, even under load.
Skip handler processing if the handler was already attached by the queue.
Respond quickly with a static reply rather than using the full stack.
Allow responses to indicate they're not ready, so they can be called back later. Currently we just move them to the back of the queue, allowing other requests to be processed first.
Use heap metrics as the queue decision
Permit the queue to be processed in other contexts.
If the server queue might be changed by a separate task - either by a deferred handler, or some other call - we must ensure it's in a sane and consistent state.
Debug printing calls yield(), permitting the queue to change mid-run.
Ensure we don't leak any AsyncClient objects, and handle extreme low memory case: each AsyncClient needs about 1k to send a 503. If we're truly out of memory, just drop the connection to keep from crashing.
This narrows the lock scope and should also avoid nested queue processing.
Better print formatting, and indicate when idle.
Cleaner API - allow the caller to tell us where to print to.
Do both queue limit disciplines, since ESP32 seems to get in to trouble with too many parallel connections even though it has lots of heap still available.
This version works around deadlocking under heavier load.
Allow clients to query or update the queue limits.
Ensure correct locking, and provide APIs for both pending and all clients.
This is helpful for debugging the underlying client library.
Also use the updated two argument concat() on modern arduino cores.
Use the optimized library implementation rather than a local loop.
Use willmmiles/v1.2.2 for load-related fixes
Add a "Walkable" wrapper for DynamicBuffer to support partial buffer reads. This permits the code to be extended to handle allocation failures, where std::vector gave us no option but to crash. Of course this code still makes no attempt to actually handle an allocation failure.
willmmiles
force-pushed
the
response_queue
branch
from
70bbca9 to
870fffc
Compare
When under memory pressure, the ESP32 TCP stack can also stop operating, resulting in hung requests. Use the logic developed for ESP8266 to avoid allocating buffers too large for the system to actually send. Fixes stalls on low-RAM ESP32 devices, such as the S2.
Include the correct header to ensure it is defined.
Provide some indication of partial buffer writes
When packet buffer allocation fails, try using TCP_MSS (one packet's worth) instead of reaching for the second-largest free block. This can improve the low memory behavior and simplifies the logic quite a bit. Also use the correct available space estimator on ESP32, and minify the debug logs to reduce the performance impact when tracing.
Use the correct functions and set good limits for ESP32. Also separate heap and maximum allocation size checks to improve tracing clarity.
willmmiles
force-pushed
the
response_queue
branch
from
96758be to
d59a712
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a request queue to AsyncWebServer with limits on queue size and available memory. This permits web servers to manage resource usage, so as to avoid crashing when overloaded. Requests that cannot be queued will be served a 503 response (unless the system is truly out of resources, at which point the connections are simply summarily closed.)
The queuing feature also permits handler functions to
defer(), indicating that some other resource is contended on, and the request cannot be immediately serviced. When called, the handler will be re-executed (if resources permit) when the queue is next pumped.The queue is pumped any time a new request is ready to handle, when a request is destructed, and periodically in poll() callbacks.