Remove some casts that are now unnecessary #1144

Job	Run time
QEMU build & test (stable, ubuntu-latest, s390x-unknown-linux-gnu, s390x-linux-gnu-gcc)	1m 0s
cargo-c dynamic library	10s
Build and test (stable, ubuntu-latest, x86_64-unknown-linux-gnu)	52s
Clippy	18s
Build and test (msrv, ubuntu-latest, x86_64-unknown-linux-gnu)	1m 3s
Clippy (--no-default-features --features="c-allocator")	18s
QEMU build & test (stable, ubuntu-latest, i686-unknown-linux-gnu, i686-linux-gnu-gcc)	44s
Build and test (beta, ubuntu-latest, x86_64-unknown-linux-gnu)	1m 14s
Smoke-test fuzzing targets (default)	2m 36s
Clippy (--no-default-features --features="rust-allocator")	37s
Build and test (stable, macos-latest, x86_64-apple-darwin)	1m 7s
Smoke-test fuzzing targets (c-allocator)	2m 35s
Build and test (stable, macos-14, aarch64-apple-darwin)	53s
Smoke-test fuzzing targets (stable, macos-14, aarch64-apple-darwin)	2m 47s
Smoke-test fuzzing targets (rust-allocator)	2m 34s
vanilla dynamic library	37s
Build and test (stable-x86_64-gnu, windows-2022, x86_64-pc-windows-gnu)	1m 58s
run flate2 test suite	17s
Miri	1m 46s
wasm32	42s
	24m 8s

Provide feedback