redisTestHook,memcachedTestHook: init; prevent hanging Darwin build after test failures

[ofalvai]

Looking at Hydra's "timed out jobs" list, I noticed many Darwin timeouts where the last log lines were all about Redis. It turns out the background Redis job is not terminated correctly on macOS when checkPhase fails (thus postCheck never runs).

I reviewed all Redis test usage in pkgs and added the same workaround, as well as connection retry where it was missing.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[afh]

It seems this PR adds almost the same code to the preCheck phase of various packages. Would it be possible and feasible to refactor that code into a script that can be re-used by these packages?

[ofalvai]

I was thinking the same, but I'm not familiar enough with nixpkgs to know how to approach this. Do you have a specific idea in mind? Maybe a hook?

[toonn]

I asked in the NixOS dev room and emily suggested a hook indeed. And in addition to mark the failing packages as broken or badPlatforms.

[mweinelt]

Create a hook to integrate redis into the check phase.

Check out pkgs/by-name/po/postgresqlTestHook which has a similar use case.

[toonn]

Emily also mentioned the following:

I'm also not sure that MAX_RETRIES stuff is a good idea since Hydra builders are often under heavy load and test timeouts usually just break things unnecessarily

I sort of suspect fixing the underlying issue is just a matter of putting a nohup in front of the server call

I'm not sure nohup would prevent the process being assigned to launchd but worth looking into I suppose.

[mweinelt]

Thanks, that looks pretty good already!

[mweinelt]

Please format codeblocks with nixfmt-rfc-style.

[ofalvai]

Fixed, as well as the redis hook docs

[mweinelt]

Can we also expose redis over a unix domain socket?

[mweinelt]

Please also expose the unix domain socket.

[mweinelt]

I think we can just print that once, otherwise it could become very spammy on busy builders.

[ofalvai]

Fixed, as well as in redisHook

[mweinelt]

I'm not entirely sure we want to propagate these packages. We could just use buildInputs, no? Or use replaceVars on the hook script.

[ofalvai]

True, I pushed a fix using substitutions (makeSetupHook has no buildInputs)

[ofalvai]

Thank you for the review @mweinelt. The easier ones are fixed now. Next, I'll figure out the unix domain socket thing and then resolve the merge conflict.