ndsharray files

[lchunleo]

Hi

i noted that when using ndsharray to write, it will create file, may i know the purpose of this file and if it is possible to write this file to a remote folder? thanks

[monzelr]

Hi,

yes, kind of.

ndsharray is using Python's mmap, its documentation says:

Memory-mapped file objects behave like both bytearray and like file objects.

Basically, what happens on the hardware is that memory in RAM is allocated and can be accessed by any process that knows the file object. This makes the file object behave like a pointer in RAM. This is done by using mmap with an anonymous memory (-1 for Windows and /dev/shm for Linux).

Your second question: Yes in theory, on linux system you can also change the path for the file. Please note, that the /dev/shm folder in linux is the "RAM" folder for shared memory for all processes. This folder is not on your hard drive, it is your RAM. This article is explaining it:
https://www.namehero.com/blog/what-is-dev-shm-and-why-is-it-so-useful/

ndsharray is not using the hard drive (HDD/SSD) to increase read/write speed of the numpy array. In a normal usage of mmap, it is using the hard drive.

I hope my answer gives you more clarity on how ndsharray works.