MemoryError (when trying to make a video)

[petermg]

ERROR:
Please report this issue here. Make sure to provide the full stacktrace: Traceback (most recent call last):
File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 473, in run_generate
ret = video_mode.gen_video(
File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 135, in gen_video
fps, input_images = open_path_as_images(os.path.abspath(video.name))
File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 61, in open_path_as_images
frames = [Image.fromarray(x) for x in list(clip.iter_frames())]
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\Clip.py", line 472, in iter_frames
frame = self.get_frame(t)
File "", line 2, in get_frame
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\decorators.py", line 89, in wrapper
return f(*new_a, **new_kw)
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\Clip.py", line 93, in get_frame
return self.make_frame(t)
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\VideoFileClip.py", line 113, in
self.make_frame = lambda t: self.reader.get_frame(t)
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 184, in get_frame
result = self.read_frame()
File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 120, in read_frame
s = self.proc.stdout.read(nbytes)
MemoryError
Time taken: 59.0 sec.

A: 2.05 GB, R: 2.40 GB, Sys: 4.0/23.9883 GB (16.8%)

[semjon00]

Try to make a shorter video and/or free computer RAM and VRAM.

[ThreeDeeJay]

How come bigger videos need more RAM/VRAM?
I'd imagine it'd just handle a few consecutive frames tops on memory when generating a video.
does it not store processed frames into a cache on the disk? or does it store the whole thing in RAM to speed things up? or perhaps we'd just need to specify a smaller memory allocation and flush unused data to make room for new resources?
It makes me wonder how much we'd need to convert a full feature film that's 1080p or even 4K 🤔

[petermg]

How come bigger videos need more RAM/VRAM?
I'd imagine it'd just handle a few consecutive frames tops on memory when generating a video.
does it not store processed frames into a cache on the disk? or does it store the whole thing in RAM to speed things up? or perhaps we'd just need to specify a smaller memory allocation and flush unused data to make room for new resources?
It makes me wonder how much we'd need to convert a full feature film that's 1080p or even 4K 🤔

I wonder the same thing. It seems unnecessary to store the whole video in RAM, doesn't seem practical.

[semjon00]

@ThreeDeeJay Soo.. That implementation is not great tbh. I believe there is code in Depth Anything 2 repository that shows how one can write frames directly to video stream, by appending a video file. Now that's really clever. But... please see the next comment.

[semjon00]

It would take a clever programmer to fix this repo and bring Depth Anything 2 approach to this repo. Are you, the reader, up to the task?

[ThreeDeeJay]

I'm not really a programmer (I'd be more than happy to assist testing new code though) but someone with technical knowledge about what needs to be done could open an issue on the DepthAnything repo.
That way one of the devs (or someone like the person who helped me here) could offer a few pointers👀🤞