Storage

Shared project dir

When you start an experiment with wall2 resources in the experiment MyProject, all your nodes will have this shared directory:

/groups/wall2-ilabt-iminds-be/MyProject/

You can use this dir to share data between nodes in the projects, and between project members. The data is stored permanently (it is not deleted after your experiment is terminated).

You can also make this shared dir available to gpulab jobs.

(Important disclaimer: There are no automatic backups for this storage! You need to keep backups of important files yourself!)

Access data from GPULab

You can access this same storage from within gpulab jobs. You just need to add the /project mountpoint to the jobDefinition:

"jobDataLocations": [
    {
       "mountPoint": "/project"
    }
],

This will cause a directory /project to be bound inside your docker container.

It will contain the same data as in /groups/wall2-ilabt-iminds-be/MyProject/. As the same NFS share is mounted behind the scenes, the data is instantly shared and never deleted.

You can also mount only sub directories of the /project dir this way:

"jobDataLocations": [
    {
       "sharePath": "/project/mycode/",
       "mountPoint": "/work/code/"
    }
],

Access data locally

To access this data share from your own machine, you’ll need to use the jFed experimenter GUI to reserve a resource, and access the data from that resource. You can find a detailed tutorial on how to do this in the Fed4Fire first experiment tutorial. Note that jFed has basic scp functionality, to make transferring files easier.