Change ping body default from 10KB to 100KB?

[bdd]

Healthchecks announced the ping body limit change for the hosted healthchecks.io instance on Twitter.

I have the defaults change queued up for runitor on a branch https://github.com/bdd/runitor/tree/ping-body-100K

On the other hand the open source distribution defaults have not changed. I have little to no insight about what environments runitor is used in but judging from the issues filed by on GitHub, quite a bit of hurdles are hit by people running their own instances. (This is partly my fault, because I assume most users would be using this against the hosted instance).

This is a community pulse check, if you will. If you are running your own instance, please chime in about this change I'm considering to merge. Your options are either to change the default for you in hc/settings.py or your container environment settings, or pass runitor --ping-body-limit parameter to match your instance's settings.

(Apologies for direct mentions, this GH Discussions are not like mailing lists, so I really wanted to grab your attention)
/cc @lfkdev, @ep1cman, @singpolyma, @fabMrc, @cuu508.

[cuu508]

I could bump up the limit on the hosted healthchecks.io instance because I recently switched to storing ping bodies in S3, they are no longer being stored in the database, and so it is now "cheaper" to support the higher limit.

If a self-hosted instance does not enable the new S3 functionality, it is still "expensive" for them to store big ping bodies. That's why I think it makes sense to keep the default limit of 10KB.

One half-baked idea: perhaps runitor can keep the default limit (and so be compatible with the default configuration of self-hosted instances) but have a special case for hc-ping.com?

[cuu508]

One more thing, we need a special value for an unlimited ping body. I'm thinking about Ping-Body-Limit: -1

• An absent Ping-Body-Limit header means the server does not report the ping body limit.
• If the server sends Ping-Body-Limit: <n>, and n is zero or positive, the server will store the n first bytes of the request body, and will ignore the rest.
• If n is -1, the server will store the complete request body.

[bdd]

How do we interpret value 0?

• Don't send a body.
• Send whatever but it'll be discarded.

Granted there's no such thing as unlimited storage, I think you can get away with, dunno, 2^64-1 if the instance setup doesn't specify a limit. In practice it'll be limited to max HTTP POST request size of the instance or any of its reverse proxies.

When you submit a wip branch for healthchecks, we can discuss there.

[cuu508]

0 means the body will be discarded. It is not an error to send it, but there is also no point in sending it.

Thinking more about the unlimited case – of course in practice it is not unlimited, but it is hard to know the effective maximum. Perhaps there is no need for the "-1" special value after all. Instead, the server could just omit the Ping-Body-Limit header if there's no explicit limit. The client would then:

• use the user-supplied limit (-ping-body-limit)
• or use the default limit (10000)

So, the user would have to make a guess of what the effective maximum is, or get the default 10'000.

[cuu508]

FYI I released Healthchecks v2.1 which adds Ping-Body-Limit: <n> header in ping responses. If the header is present, n is a nonnegative integer. The hosted service is also already sending this header, with the value 100000.

[bdd]

I already have the implementation but I'll work on redoing it so it won't require Go 1.18 (it uses generics because it was the fastest way to prototype this).

I also need to tie down how runitor should interpret Ping-Body-Limit: 0. I'm going to change it to disabling sending ping body altogether. Previously (currently) if one passes -ping-body-limit 0 as an argument, it disables client side tailing and the entirety of accumulated body gets sent to server.

[lfkdev]

Many thanks for drawing attention! 100kb is really great and i'll definitly use it. In my opinion, 100kb is still cheap enough that you can make this the new default. Most people will enable this anyway - and people who maintain a very large number of jobs will probably pay attention to what changes an update brings (maybe it should be made extra clear in the changelog).

[bdd]

Released v0.10.0-beta.3 with hc-ping.com default updated to 100KB, along with custom request headers (useful for local instances behind Cloudflare Zero Trust and the like) and a modification to build script to ensure we continue to build reproducible binaries with Go 1.18 and later.

Please download, test, and report back if you can.

I intend to cut the v0.10.0 release on May 7, Saturday morning, Pacific time.

[bdd]

v0.10.0-beta.5 with Ping-Body-Limit response header discovery (plus the previous custom request headers feature from beta.1) is out.

https://github.com/bdd/runitor/releases/tag/v0.10.0-beta.5