Error when calling lmdb.open()

[piergiaj]

Whenever I call open, whatever string I pass in as the path, it gives the following error:

lmdb.open('trainingData')
Traceback (most recent call last):
File "", line 1, in
lmdb.Error: trainingData: Function not implemented

The error line always matches the string I pass in. The folder is created and a lock file is created, but I always get this error. Any ideas how to fix this?

[dw]

Hi there. What OS and filesystem are you using?

[piergiaj]

OS: Linux 3.0.101-0.31.1_1.0502.8394-cray_gem_c

File system is dvs

[dw]

Hi again,

I'm going to guess there's a 90% chance dvs doesn't support mmap, but in order to confirm this, can you upload a gist containing the output of strace -s 100 python your/script.py. If it is mmap, then we can't support LMDB on that filesystem, however if it's another system call returning the error, it might be possible to work around it

[hyc]

The Cray documentation confirms, they only support read-only mmap on DVS. http://docs.cray.com/books/S-0005-5204/ page 39 caveats.

[dw]

Hi there,

hbf suggests you might be fine running with writemap=False, since DVS supports read-only mmaps, however be aware that this still does not allow opening the database from multiple nodes in your cluster, as DVS does not provide any coherency between the in-memory database file and its current representation on disk, and even if it did, LMDB does not support any kind of locking that would work in this scenario.

[amaeme]

sbatch --gres=gpu:1 tr.sh
Where tr.sh is my test script

[aiyulu]

I have met the same problem, which caused by system lock, like notebook cpfs_data folder has system lock, but nas_data folder has not.