fix: remove the use of asyncio.get_child_watcher() for python3.14

[cryptomilk]

Use PidfdChildWatcher if os.pidfd_open is available and works, and otherwise use ThreadedChildWatcher which should work in any case.

Fixes #583

[cryptomilk]

Looks like that the CI fails before it runs the tests.

[justinmk]

expand this to see the test failures:

 FAILED test/test_buffer.py::test_options - OSError: EOF
  FAILED test/test_buffer.py::test_get_exceptions - OSError: EOF
  FAILED test/test_window.py::test_options - OSError: EOF

[wookayin]

I will try to diagnose the test failure on python 3.13 with nvim-0.11 nightly and send a separate PR to fix it.

UPDATE: this is an upstream issue, see neovim/neovim#31894

[cryptomilk]

Oh, locally I'm running nvim 0.10.3 and everything works. GH runs nvim nightly?

[cryptomilk]

I've rebased on master and it shows that the patch works fine with nvim-stable but fails with nvim-nightly.

[justinmk]

windows ci shows an issue:

  C:\hostedtoolcache\windows\Python\3.13.1\x64\Lib\asyncio\unix_events.py:40: in <module>
      raise ImportError('Signals are not really supported on Windows')
  E   ImportError: Signals are not really supported on Windows

other than that, the failing vim.current.buffer.options['invalid-option'] tests are a known issue

Note: Tests are still failing on nvim-nightly due to upstream bug, but as soon as neovim/neovim#31894 is fixed CI should be back to green.

[cryptomilk]

Maybe we need to make sure the ThreadedChildWatcher is used on Windoze?

[wookayin]

CI on python3.7 reports:

RuntimeError: Cannot add child handler, the child watcher does not have a loop attached

whereas on all other python versions (>=3.8) CI failure is expected just because of Nvim crashing.

It'd be also appreciated if you could fix the commit message title with an appropriate header (e.g. fix: replace deprecated ...).

[cryptomilk]

I'm sick, it has to wait ...

[justinmk]

If necessary, we can drop python 3.7 support.

[cryptomilk]

The error on 3.7 is now the same as with 3.13

[justinmk]

neovim/neovim#31894 was fixed, will re-run the tests here after a new nightly is posted.

[cryptomilk]

Rebased, tests are mostly passing now. The Windows and MacOS tests with Python 3.13 are failing.

On Windows it looks like we have some closing workaround which fails with Python 3.13 now.

pynvim/pynvim/msgpack_rpc/event_loop/asyncio.py

Line 250 in 9c5ec67

# Windows: for ProactorBasePipeTransport, close() doesn't take in

I have no idea what is wrong on MacOS.

[cryptomilk]

I added a commit which disabled the workaround on Windows and that seems to work. Not sure if this is the right fix. I have no clue about Windows.

The failing test_attach.py::test_connect_stdioThe test on MacOS seems to be race condition. Now it works on 3.13 and fails on 3.12.

[wookayin]

I don't think the macOS failure has something to do with this PR; it'll be a different bug that already existed. I'll see later if we can bring a proper fix.

[justinmk]

disabled the workaround on Windows and that seems to work

that was added in 17fbcbc ; @wookayin do you see any risk with disabling it here?

[wookayin]

I cherry-picked the commit eb14e83 for bumping up msgpack as it's a bit tangential to this fix.

Regarding the workaround in 17fbcbc, we can try disabling the workaround for python3.13 on Windows to see if the bug has been fixed. But I believe for older versions this workaround will still be necessary.

[cryptomilk]

I really start to hate Python.

python/cpython@9d2e1ea
python/cpython#120804

Every 2 versions they completely break the API. WTF.

[cryptomilk]

Looks like supporting 3.14 is a much bigger rewrite.

[wookayin]

I appreciate your patience. Now as I think about the role of child_watcher, it's not quite crucial part of the pynvim functionality and I think we actually don't need it for other than preventing resource leak and some unwanted side effects (fd4247c) when the event loop closes.

Let's make our life simpler and easier: my proposal is to simply just disable child_watcher in python3.14 (i.e. set child_watcher = None), and let pynvim just work for python3.14. (provided that test passes...)

[cryptomilk]

Looks like I misunderstood how things are now. I hope the latest changes are correct now.

[justinmk]

Thank you both!