Improve length field prepending on bungee -> server connection

[Janmm14]

Use alternative implementation of Varint21LengthFieldPrepender on bungee -> server connection for improved speed - it uses separate buffer to prepend the length to avoid copying large data buffer.

Not applied bungee -> client because encrypting 1-5 bytes of length separately through expensive jni call could make it not worth (not measured).

Netty's LengthFieldPrepender also uses an extra buffer for the length.

[Janmm14]

This will theoretically synergize well with #3393

[md-5]

Thanks

[bob7l]

It'd probably be much faster in terms of performance to use a heapBuffer considering you're only working with a couple bytes of memory.

[Janmm14]

It'd probably be much faster in terms of performance to use a heapBuffer considering you're only working with a couple bytes of memory.

Afaik epoll requires direct/unsafe byte buffers anyway, buffers without a mem address require extra handling.

The buffer() method used by netty's lengthfieldprepender (i used ioBuffer() ) by default prefers direct buffers too.

Two types of buffer impls in use might also remove some hotspot optimizations which would be appliable if only one buffer impl is used etc.

I wouldn't go against what netty itself uses, if you believe that heap buffers should be used / are faster for a few bytes, you could open an issue at netty about their lengthfieldprepender (after you benchmarked that it's slower).
Maybe you could also just create a discussion or ask in their discord about it and someone might be able to answer you or benchmarks it for you.

[bob7l]

It'd probably be much faster in terms of performance to use a heapBuffer considering you're only working with a couple bytes of memory.

Afaik epoll requires direct/unsafe byte buffers anyway, buffers without a mem address require extra handling.

The buffer() method used by netty's lengthfieldprepender (i used ioBuffer() ) by default prefers direct buffers too.

Two types of buffer impls in use might also remove some hotspot optimizations which would be appliable if only one buffer impl is used etc.

I wouldn't go against what netty itself uses, if you believe that heap buffers should be used / are faster for a few bytes, you could open an issue at netty about their lengthfieldprepender (after you benchmarked that it's slower). Maybe you could also just create a discussion or ask in their discord about it and someone might be able to answer you or benchmarks it for you.

I was more concerned with writing the varInt to a direct buffer using the current loop implementation (Netty uses bulk writes) which would likely be much slower. Allocation I doubt really matters since we're using pooling. But since most packets are small, I doubt there would be any benefit from using a heap buffer. Not worth going through the trouble of all the benchmarking required for a micro-optimization, and it'll obviously lead to slowdowns further down the chain

Onto these bulk operations though. We could easily utilize bulk writes like writeShort/WriteMedium/etc in this implementation (Or really any pertaining to VarInts). We already need to find a VarInt length for allocation, might as well use it for single-writing our VarInt too as opposed to writing the VarInt in a loop. Would most likely get smacked by the JIT and or further runtime optimizations too.

[Janmm14]

Onto these bulk operations though. We could easily utilize bulk writes like writeShort/WriteMedium/etc in this implementation (Or really any pertaining to VarInts). We already need to find a VarInt length for allocation, might as well use it for single-writing our VarInt too as opposed to writing the VarInt in a loop. Would most likely get smacked by the JIT and or further runtime optimizations too.

I did a benchmark with combined writes (short / medium / int / int+byte) and I couldn't find a significant, reproducable difference. Run-by-run differences too high.

Upgrading jvm to 19 from 16 has easily more impact in my benchmark. Nope, just blackhole impl difference or idk what.

Edit: Here is my benchmark code: https://gist.github.com/Janmm14/1afff414893bac7e0daf85492912ed22

Edit:
I slightly altered the benchmark.

New code: https://gist.github.com/Janmm14/67cd16e0966b8f0c2a4bfc3de33c3bd2
Results: https://gist.github.com/Janmm14/9d8396aff212f91135ce8200d8380c42

java ran with CPU frequency locked, priority on high and pc not being actively used for anything else.
Results vary too much to be meaningful - there is at max a slight hint that the custom impl with a switch is a little bit faster, but its definitely not enough. In other runs I have seen orig impl also for max of 60000 producing less than 10ns/op.

Yet another edit:
I upped DATA_COUNT to 1_000_000, this got rid of the hikkups (maybe they came from inaccurate per-invocation measurements). The data suggests that there is a 10% improvement (more for bigger numbers) when we use a switch and consolidated buffer write calls - I also would spend the results of benchmarks without a 2 (the benchmakrs with blackhole) more importance than those without blackhole.
Results: https://gist.github.com/Janmm14/aee19d08ffcb34df251f0ff0ad11113c

[Leymooo]

New code: https://gist.github.com/Janmm14/67cd16e0966b8f0c2a4bfc3de33c3bd2

why use Level.Invocation?
Your becnhmark is broken, you have different test data for each benchmark, one benchmark can have more 4,5 bytes varint while another can have more 1,2 bytes varints.

Also this was measured before https://steinborn.me/posts/performance/how-fast-can-you-write-a-varint/ ,and it shows that unrolling varint writting loop gives a lot of performance(https://github.com/astei/varint-writing-showdown)

[Janmm14]

why use Level.Invocation? Your becnhmark is broken, you have different test data for each benchmark, one benchmark can have more 4,5 bytes varint while another can have more 1,2 bytes varints.

My thought was that 100k or 1 million would be large enough to roughly even the randomness, especially for the random runs with ints <30k/60k. I will run a short test to see if not using invocation level gives changes.

Also this was measured before https://steinborn.me/posts/performance/how-fast-can-you-write-a-varint/ ,and it shows that unrolling varint writting loop gives a lot of performance(https://github.com/astei/varint-writing-showdown)

Testing stuff, the main difference between our benchmarks is not randomness of data, but the data_count difference, making the bottleneck in my benchmark to probably be memory bandwhich.
Lowering DATA_COUNT to 2048 causes my benchmark to reach similar speed to his zen2 result (me being on zen3 without boost clocks for testing) (3.1ns vs 5.1ns without blackhole, 5.7ns vs 7.7ns with b) - 40% reduction.

Using a DATA_COUNT of 32768 (prob causing branch prediction cache to be less relevant) gives me 6.2ns vs 8.1ns (7.7ns vs 9.4ns with blackhole) even when limiting the max number to just 10k. (9.6ns vs 16ns when using full integer range).

Given this, the real-world-speedups inside bungee are nowhere near 80% (most packets and other sent varint numbers are smaller), but they are still emasureable. PR is coming.