Feature/rclone parallel uploads

[alash2k]

No description provided.

[Calimerorulez]

@alash2k I'm wanting to try your improvements, but when running your dev container, it throws errors:

2024-12-09 12:58:12 [   ERROR   ] unifi_protect_backup.unifi_protect_backup_core :  Unexpected exception occurred in main loop:

Traceback (most recent call last):

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

2024-12-09 12:58:22 [   ERROR   ] aiorun                                     :  Unhandled exception; stopping loop: 'Task exception was never retrieved'

Traceback (most recent call last):

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 219, in new_coro

    await coro

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

Traceback (most recent call last):

  File "/usr/bin/unifi-protect-backup", line 8, in <module>

    sys.exit(main())

             ^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1137, in __call__

    return self.main(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1062, in main

    rv = self.invoke(ctx)

         ^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1404, in invoke

    return ctx.invoke(self.callback, **ctx.params)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 763, in invoke

    return __callback(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/cli.py", line 232, in main

    run(event_listener.start(), stop_on_unhandled_errors=True)

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 371, in run

    raise pending_exception_to_raise

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 219, in new_coro

    await coro

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

[alash2k]

Thanks for checking. I made additional changes to suite my needs. I don't think you would want them merged. I would suggest you look at #153

@alash2k I'm wanting to try your improvements, but when running your dev container, it throws errors:

2024-12-09 12:58:12 [   ERROR   ] unifi_protect_backup.unifi_protect_backup_core :  Unexpected exception occurred in main loop:

Traceback (most recent call last):

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

2024-12-09 12:58:22 [   ERROR   ] aiorun                                     :  Unhandled exception; stopping loop: 'Task exception was never retrieved'

Traceback (most recent call last):

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 219, in new_coro

    await coro

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

Traceback (most recent call last):

  File "/usr/bin/unifi-protect-backup", line 8, in <module>

    sys.exit(main())

             ^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1137, in __call__

    return self.main(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1062, in main

    rv = self.invoke(ctx)

         ^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 1404, in invoke

    return ctx.invoke(self.callback, **ctx.params)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/click/core.py", line 763, in invoke

    return __callback(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/cli.py", line 232, in main

    run(event_listener.start(), stop_on_unhandled_errors=True)

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 371, in run

    raise pending_exception_to_raise

  File "/usr/lib/python3.12/site-packages/aiorun.py", line 219, in new_coro

    await coro

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/unifi_protect_backup_core.py", line 252, in start

    downloader = downloader_cls(

                 ^^^^^^^^^^^^^^^

  File "/usr/lib/python3.12/site-packages/unifi_protect_backup/downloader_experimental.py", line 88, in __init__

    raise RuntimeError("The `uiprotect` library is currently missing the features for this to work.")

RuntimeError: The `uiprotect` library is currently missing the features for this to work.

[Calimerorulez]

I tried to debug a little, and I think @ep1cman is using his own version of uiprotect library, because the error that is thrown is that method prepare_camera_video isn't available in uiprotect.

uilibs/uiprotect#249

[ep1cman]

Unfortunately I'm still waiting on uiprotect to merge this feature so it's blocked. They want me to implement tests but the test feature is broken so I can't.

Just don't use the experimental downloader if you want to use the dev container

[alash2k]

Is there a way to increase concurrency on downloading from NVR?