test_rf_glofas fails on Windows

[emanuel-schmid]

Apparently temporary directories and, possibly, Mock behave differently on Windows.

Running

pytest climada_petals\hazard\rf_glofas\test

eventually yields the following output:

...
========================================================================================= short test summary info =========================================================================================== 
FAILED climada_petals\hazard\rf_glofas\test\test_cds_glofas_downloader.py::TestGloFASRequest::test_cleanup_download_dir - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\[user]\\AppData\\Local\\Temp\\tmpcdhavidt\\tmpplz_2o8d'
FAILED climada_petals\hazard\rf_glofas\test\test_river_flood_computation.py::TestRiverFloodInundation::test_flood_depth - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\[user]\\AppData\\Local\\Temp\\tmp8v017e7s\\cache\\240522-092644-nxawxu46\\return-pe...
FAILED climada_petals\hazard\rf_glofas\test\test_transform_ops.py::TestGlofasDownloadOps::test_basic - NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\[user]\\AppData\\Local\\Temp\\tmp1f948t9g\\file-1.nc'
FAILED climada_petals\hazard\rf_glofas\test\test_transform_ops.py::TestGlofasDownloadOps::test_countries_area - TypeError: name must be a str, not a MagicMock
FAILED climada_petals\hazard\rf_glofas\test\test_transform_ops.py::TestGlofasDownloadOps::test_preprocess - TypeError: name must be a str, not a MagicMock
ERROR climada_petals\hazard\rf_glofas\test\test_cds_glofas_downloader.py::TestGloFASRequest::test_cleanup_download_dir - NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\[user]\\AppData\\Local\\Temp\\tmpcdhavidt\\tmpplz_2o8d'
ERROR climada_petals\hazard\rf_glofas\test\test_river_flood_computation.py::TestRiverFloodInundation::test_return_period_resample - NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\[user]\\AppData\\Local\\Temp\\tmp8v017e7s\\cache\\240522-092644-nxawxu46\\return-period-regrid.nc'
============================================================================ 5 failed, 36 passed, 14 warnings, 2 errors in 20.79s ============================================================================ 

Amendment:

This one is also failing with python 3.11:

python -m unittest climada_petals.hazard.rf_glofas.test.test_transform_ops.TestGlofasDownloadOps.test_preprocess

======================================================================
ERROR: test_preprocess (climada_petals.hazard.rf_glofas.test.test_transform_ops.TestGlofasDownloadOps.test_preprocess)
Test the capabilities of the preprocessing
----------------------------------------------------------------------
Traceback (most recent call last):
  File "...\miniconda3\envs\climada_env\Lib\shutil.py", line 632, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '...\AppData\\Local\\Temp\\tmpacus8_nt\\file-1.nc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\climada_petals\climada_petals\hazard\rf_glofas\test\test_transform_ops.py", line 82, in tearDown
    self.tempdir.cleanup()
  File "...\envs\climada_env\Lib\tempfile.py", line 947, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "...\envs\climada_env\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "...\envs\climada_env\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\envs\climada_env\Lib\shutil.py", line 634, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "...\envs\climada_env\Lib\tempfile.py", line 893, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '...\\AppData\\Local\\Temp\\tmpacus8_nt\\file-1.nc'

apparently a file handle that hasn't been released.

[peanutfun]

It might also be that the behavior changed with more recent Python versions. Or are you still on 3.9? This is the second issue related to my model. I'll see when I find time to check both out. But I have to admit that this currently has a low priority for me 🙇

[emanuel-schmid]

Thanks! I'm still on 3.9 on Windows. No worries, it has low priority for me as well.