Swestore-iRODS-at-NSC

Swestore iRODS at NSC

iRODS is a distributed storage management system which enables the user to control the access to his/her data via policies which are expressed as rules. Swestore iRODS is hosted on servers at PDC and NSC. We are running version 4.2.6. Basic information about how to store and access data had been described above. Additional information about iRODS can be found at the iRODS web site.

iRODS system at NSC

At NSC we host a replica metadata server and two storage server nodes at the moment. All data stored is replicated between PDC and NSC.

Accessing iRODS from the compute cluster

Data stored in iRODS is accessible from tetralith. We have got the command line client installed on the cluster, iCommands should work from the login node and also from the batch nodes.

You should have the client configuration files in place under the ~/.irods directory as it has been described under the introductory pages.

Under your home directory you should create a directory called .irods which will contain the iRODS client configuration file. That file is irods_environment.json. It should look like:

 {
     "irods_host": "irods-login.swestore.se",
     "irods_port": 2432,
     "irods_default_resource": "pdc-gpfs",
     "irods_home": "/snic.se/home/<YOUR_SNIC_USER_NAME>",
     "irods_cwd": "/snic.se/home/<YOUR_SNIC_USER_NAME>",
     "irods_user_name": "<YOUR_SNIC_USER_NAME>",
     "irods_zone_name": "snic.se",
     "irods_client_server_negotiation": "request_server_negotiation",
     "irods_client_server_policy": "CS_NEG_REFUSE",
     "irods_encryption_key_size": 32,
     "irods_encryption_salt_size": 8,
     "irods_encryption_num_hash_rounds": 16,
     "irods_encryption_algorithm": "AES-256-CBC",
     "irods_default_hash_scheme": "SHA256",
     "irods_match_hash_policy": "compatible",
     "irods_authentication_scheme": "PAM" 
}

where the token <YOUR_SNIC_USER_NAME> should be replaced with your SNIC user name.

Issue:

 module load irods-client/4.2.6

after which you should authenticate using iinit and then the iCommands like iget, iput, ils should work as expected. The ihelp command will print a short description of the irods commands.

First issue

 iinit

which will ask for your iRODS password. You can specify a longer time limit for the session using the --ttl option, iinit -h --ttl will print help using this. The default generally will suffice.

Use

 ils

just to check if you can access your home directory. Please use the project directory to store files, which will be a subdirectory under //snic.se/projects/<SNIC_PROJECT_NAME>.

To transfer data from the cluster directory it is a good idea to use irsync like:

 irsync -v -r -K local_directory i:irods_directory

where

 -v for verbose.
 -r for recursive copy of the whole directory tree.
 -K to compute and check checksums. This involves higher CPU usage
    but gives confidence for the integrity of the data stored.

i.e.:

 irsync -r -K mydir i:/snic.se/projects/projectdir

If, for whatever reason the transfer had been interrupted the second invocation of the command with the same arguments should pick up and continue from that point.

Detailed description of the command line client can be found at documentation at the iRODS site.