Swestore-irods

National Storage using iRODS

NOTA: THIS PAGE IS OUT OF DATE! IT WILL BE SOON DELETED.

Getting access

Apply for storage

Please follow the instructions on the Apply for storage on SweStore page.

iRODS user authentication

SweStore's iRODS system uses Yubikey one-time passwords (OTP). With a simple touch of a button, a 44 character one-time password is generated and sent to the system. The user will be provided with a SweStore yubikey.

Yubikey has a status as pilot now. It can be changed in the future.

iRODS usage Pilot.
- How to acquire a SweStore yubikey

Please send an email to support@swestore.se and provide the shipping address to where the yubikey should be sent.

Yubikey has a status as pilot now. It can be changed in the future.

Support

If you have any issues using SweStore please do not hesitate to contact support@swestore.se.

Better not to use filename with single quotes. (There were problems with these but they had been fixed.

Usage monitoring

• Per Project Monitoring of Swestore usage

Supported clients

iDrop web - Point your Web browser to iweb.swestore.se

E-iRODS iCommands 3.0 - Command line client Download E-iRODS icommands

SweStore iRODS uses PAM authentication and SweStore yubikeys. With a simple touch of a button, a 44 character one-time password is generated and sent to the system.

Web GUI (iDrop web)

Please see the specific documentation for iDrop web.

Community iRODS version 3.3

The community iRODS client version 3.3 also should work, with PAM authentication.
It is available from SDSC. Please install the OpenSSL include files and libraries:

$ sudo apt-get install libssl-dev (debian based system)
# yum install openssl-devel (redhat-based systems)

Download irods 3.3 from http://irods.sdsc.edu/download.html and unpack the tar.gz archive.

Please enable the following defines in the Makefile iRODS/config/config.mk.in

PAM_AUTH = 1
PAM_AUTH_NO_EXTEND = 1
USE_SSL = 1 

Please run irodssetup to compile the irods community client with PAM authentication.

SweStore iRODS usage documentation

To use the system you need to have the E-iRODS command line client installed or using iDROP web.

Command line client

For Linux systems the iRODS command line client is available as an installable package for various Linux platforms from the e-iRODS website downloads section.

The command line client is natural to use for Unix users. There are versions of the usual ls, rm, mv, mkdir, pwd, rsync commands prefixed with an i for iRODS, i.e. irm, imv, imkdir etc.

As expected iput and iget move files to and from the irods system. All these commands print short help when using the -h option.

iCommands environment file

There is an environment file .irodsEnv in the .irods subdirectory of the home directory ($HOME/.irods/.irodsEnv) which contains information where and how to access the iRODS metadata (iCAT) server.

It looks like (placeholders are in <>):

irodsHost 'irods.swestore.se'
irodsPort 1247
irodsDefResource 'snicdefResc'
irodsHome '/snicZone/proj/<PROJECT_NAME>'
irodsCwd '/snicZone/proj/<PROJECT_NAME>'
irodsUserName '<USERNAME>'
irodsZone 'snicZone'
irodsAuthScheme 'PAM'

The iCAT server is irods.swestore.se. The default irods zone name is snicZone. The default resource is snicdefResc. It is best to set the home directory to the same as the project directory, which would be a subdirectory under the /snicZone/proj directory tree.

Yubikey instructions

Prerequisite: A correct iCommands environment file, see above for instructions.

1. Insert the yubikey in an available USB-slot in your computer.
2. Type iinit
3. Touch the conductive surface on the yubikey to send an one-time password to the system.

$ iinit
Enter your current PAM (system) password:
$ ils
/snicZone/proj/<projectname>:
$

After that we can use the usual iCommands for 8 hours.

More details on the iCommands are available at https://www.irods.org/index.php/icommands

iCommands

Having initialized the session as described above we can use tie iRODS versions of the basic Unix commands. The project directory is under /snicZone/proj, all members of the project should have write access to this directory. We can use the command

icd /snicZone/proj/projectname

to move to the project directory, or to change to an another project directory when we are members of more than one project.

All commands give short help when invoked with the -h flag.

To put files files into the iRODS system we can use:

iput localfile irodsfile

or, to put a whole directory tree:

iput -r localdirectory irodscollection

To load large amout of data it might be more advantageous to use

irsync -r localdirectory irodscollection

It might be a good idea to use -K so then checksums will be computed, stored and checked.

To create directories (collections in iRODSspeak) we use:

imkdir collection

as it would be expected.

To get those files back we can use

iget irodsfile localfile

or

irsync -r irodscollection localdirectory

To remove files we use:

irm

or

irm -r

Removing files like that would put the files into the trashcan (path: /snicZone/trash/). Time to time we would need to empty the trashcan, using

irmtrash

Using iCommands on SNIC HPC clusters

On SNIC-clusters the icommands command line tools are either available in the PATH or by adding the irods module, e.g.

module load irods

If the irods commands are not available at the SNIC HPC cluster, please contact support@swestore.se

We also need to setup the iCommands environment file $HOME/.irods/.irodsEnv

Storage Project directory structure

Your storage project is available at /snicZone/proj/<PROJECT NAME>

/snicZone/home/<USERNAME> is just a small home directory.

iDROP web client

See the iDrop web specific page.

Upstream documentation

Detailed documentation, papers and resources are available from the E-iRODS web site

Community iRODS

User forum