Merge #4620

4620: Update juliet server settings r=jacek-casper a=jacek-casper

Updating the client settings after doing some measurements and advice from `@marc-casperlabs` .

The rationale is:
- we've measured the average round-trip time for sending a juliet request and receiving a response from the sidecar, which averaged around 5ms and varied between 2-10ms
- we've measured the average time spent in the code that deals with storage (run_query) and it was about 0.6ms
- if we want to saturate the time spent running queries, we'd need to permit at least 4 concurrent requests on average ((average_roundtrip - average_run_query) / 2 / average_run_query), and in the worst case scenario (10ms time and lack of parallelism) we have (10 - 0.6) / 2 / 0.6 which is 8 concurrent requests and we've rounded it up to 10 to create some leeway
- the buffer size was set based on the request limit to allow for a backlog size of double the requests we can handle concurrently

We didn't discuss `max_request_size_bytes`, and `max_response_size_bytes`, might be worth tackling that as well

Co-authored-by: Jacek Malec <[email protected]>

resources/local/config.toml

@@ -312,11 +312,13 @@ max_request_size_bytes = 4_194_304
 # Maximum size of a response in bytes.
 max_response_size_bytes = 4_194_304
 
-# Maximum number of in-flight requests per client.
-client_request_limit = 3
+# Maximum number of in-flight requests.
+# This is based on measurements captured on a local instance of the sidecar,
+# it might make sense to increase this value if the sidecar is running on a separate machine.
+client_request_limit = 10
 
-# Number of requests that can be buffered per client.
-client_request_buffer_size = 16
+# Number of requests that can be buffered.
+client_request_buffer_size = 20
 
 # Maximum number of connections to the server.
 max_connections = 16