Swestore-iRODS-at-NSC - Revision history

Niklas Edmundsson (HPC2N): Swestore iRODS is decomissioned

2023-02-08T10:13:55Z

Swestore iRODS is decomissioned

Janos Nagy (NSC): /* Accessing iRODS from the compute cluster */

2020-12-09T09:41:33Z

‎Accessing iRODS from the compute cluster

Janos Nagy (NSC) at 09:28, 9 December 2020

2020-12-09T09:28:35Z

Janos Nagy (NSC): /* Accessing iRODS from the compute cluster */

2020-06-01T10:24:15Z

‎Accessing iRODS from the compute cluster

Janos Nagy (NSC) at 10:22, 1 June 2020

2020-06-01T10:22:32Z

Janos Nagy (NSC): /* Swestore iRODS at NSC */

2020-06-01T10:03:07Z

‎Swestore iRODS at NSC

Janos Nagy (NSC): /* Accessing iRODS from the compute cluster */

2020-06-01T10:00:50Z

‎Accessing iRODS from the compute cluster

Janos Nagy (NSC): /* Accessing iRODS from the compute cluster */

2020-06-01T09:59:58Z

‎Accessing iRODS from the compute cluster

Janos Nagy (NSC) at 09:58, 1 June 2020

2020-06-01T09:58:29Z

Janos Nagy (NSC): Created page with " = Swestore IRODS at NSC ="

2020-06-01T09:28:39Z

Created page with " = Swestore IRODS at NSC ="

New page

= Swestore IRODS at NSC =

← Older revision		Revision as of 10:13, 8 February 2023
Line 1:		Line 1:
−	~~= Swestore iRODS at NSC =~~	+	#REDIRECT[[Swestore iRODS is decommissioned]]
−
−	~~iRODS is a distributed storage management system which enables the user to~~
−	~~control the access to his/her data via policies which are expressed as rules.~~
−	Swestore iRODS is ~~hosted on servers at PDC and NSC. We are running version 4.2.6.~~
−	~~Basic information about how to store and access data had been described above.~~
−	~~Additional information about iRODS can be found at [http://www.irods.org the iRODS web site~~].
−
−	~~== iRODS system at NSC ==~~
−
−	~~At NSC we host a replica metadata server and two storage server nodes at the moment.~~
−	~~All data stored is replicated between PDC and NSC.~~
−
−	~~== Accessing iRODS from the compute cluster ==~~
−
−	~~Data stored in iRODS is accessible from <code>tetralith</code>.~~
−	~~We have got the command line client installed on the cluster,~~
−	~~iCommands should work from the login node and also from the batch~~
−	~~nodes.~~
−
−	~~You should have the client configuration files in place under~~
−	~~the <code>~/.irods</code> directory as it has been described under~~
−	~~the introductory pages.~~
−
−	~~Under your home directory you should create a directory called~~
−	~~<code>.irods</code> which will contain the iRODS client configuration~~
−	~~file. That file is <code>irods_environment.json</code>. It should~~
−	~~look like:~~
−	{
−	~~"irods_host": "irods-login.swestore.se",~~
−	~~"irods_port": 2432,~~
−	~~"irods_default_resource": "pdc-gpfs",~~
−	~~"irods_home": "/snic.se/home/<YOUR_SNIC_USER_NAME>",~~
−	~~"irods_cwd": "/snic.se/home/<YOUR_SNIC_USER_NAME>",~~
−	~~"irods_user_name": "<YOUR_SNIC_USER_NAME>",~~
−	~~"irods_zone_name": "snic.se",~~
−	~~"irods_client_server_negotiation": "request_server_negotiation",~~
−	~~"irods_client_server_policy": "CS_NEG_REFUSE",~~
−	~~"irods_encryption_key_size": 32,~~
−	~~"irods_encryption_salt_size": 8,~~
−	~~"irods_encryption_num_hash_rounds": 16,~~
−	~~"irods_encryption_algorithm": "AES-256-CBC",~~
−	~~"irods_default_hash_scheme": "SHA256",~~
−	~~"irods_match_hash_policy": "compatible",~~
−	~~"irods_authentication_scheme": "PAM"~~
−	}
−	~~where the token <code><YOUR_SNIC_USER_NAME></code> should be replaced~~
−	~~with your SNIC user name.~~
−
−	~~Issue:~~
−	~~module load irods-client/4.2.6~~
−	~~after which you should authenticate using <code>iinit</code> and then the iCommands like~~
−	~~<code>iget</code>, <code>iput</code>, <code>ils</code> should work as expected.~~
−	~~The <code>ihelp</code> command will print a short description of the irods commands.~~
−
−	~~First issue~~
−	~~iinit~~
−	~~which will ask for your iRODS password. You can specify a longer time limit~~
−	~~for the session using the <code>--ttl</code> option, <code>iinit -h --ttl</code>~~
−	~~will print help using this. The default generally will suffice.~~
−
−	~~Use~~
−	~~ils~~
−	~~just to check if you can access your home directory. Please use the project~~
−	~~directory to store files, which will be a subdirectory under~~
−	~~<code>//snic.se/projects/<SNIC_PROJECT_NAME></code>.~~
−
−	~~To transfer data from the cluster directory it is a good idea to use~~
−	~~<code>irsync</code> like:~~
−	~~irsync -v -r -K local_directory i:irods_directory~~
−	~~where~~
−	~~-v for verbose.~~
−	~~-r for recursive copy of the whole directory tree.~~
−	~~-K to compute and check checksums. This involves higher CPU usage~~
−	~~but gives confidence for the integrity of the data stored.~~
−	~~i.e.:~~
−	~~irsync -r -K mydir i:/snic.se/projects/projectdir~~
−	~~If, for whatever reason the transfer had been interrupted the second~~
−	~~invocation of the command with the same arguments should pick up and~~
−	~~continue from that point.~~
−
−	~~Detailed description of the command line client can be found at~~
−	~~[https://docs.irods.org/master/icommands/user/ documentation at the iRODS site~~].

@@ Line 20: / Line 20: @@
 You should have the client configuration files in place under
-the <code>.irods</code> directory as it has been described under
+the <code>~/.irods</code> directory as it has been described under
 the introductory pages.
 Under your home directory you should create a directory called
-<code>.irods</code> which contains the iRODS client configuration
+<code>.irods</code> which will contain the iRODS client configuration
 file. That file is <code>irods_environment.json</code>. It should
 look like:
@@ Line 68: / Line 68: @@
 To transfer data from the cluster directory it is a good idea to use
 <code>irsync</code> like:
-   irsync -r -K local_directory i:irods_directory
+   irsync -v -r -K local_directory i:irods_directory
 where
-   -r for recursive copy of the whole directory tree
+  -v for verbose.
-   -K to compute and check checksums
+   -r for recursive copy of the whole directory tree.
 i.e.:
    irsync -r -K mydir i:/snic.se/projects/projectdir

@@ Line 15: / Line 15: @@
 Data stored in iRODS is accessible from <code>tetralith</code>.
-We have got the command line client installed on the cluster.
+We have got the command line client installed on the cluster,
 You should have the client configuration files in place under
 the <code>.irods</code> directory as it has been described under
 the introductory pages.
 Issue:
    module load irods-client/4.2.6
@@ Line 24: / Line 53: @@
 <code>iget</code>, <code>iput</code>, <code>ils</code> should work as expected.
 The <code>ihelp</code> command will print a short description of the irods commands.
 Detailed description of the command line client can be found at
 [https://docs.irods.org/master/icommands/user/ documentation at the iRODS site].

@@ Line 25: / Line 25: @@
 The <code>ihelp</code> command will print a short description of the irods commands.
 Detailed description of the command line client can be found at
-[https://docs.irods.org/master/icommands/user/ documentation at iRODS site].
+[https://docs.irods.org/master/icommands/user/ documentation at the iRODS site].

@@ Line 3: / Line 3: @@
 iRODS is a distributed storage management system which enables the user to
 control the access to his/her data via policies which are expressed as rules.
-Swestore iRODS is hosted on servers at PDC and NSC.
+Swestore iRODS is hosted on servers at PDC and NSC. We are running version 4.2.6.
 Basic information about how to store and access data had been described above.
 Additional information about iRODS can be found at [http://www.irods.org the iRODS web site].
@@ Line 23: / Line 23: @@
 after which you should authenticate using <code>iinit</code> and then the iCommands like
 <code>iget</code>, <code>iput</code>, <code>ils</code> should work as expected.

@@ Line 13: / Line 13: @@
 == Accessing iRODS from the compute cluster ==
-Data stored in iRODS can be access from tetralith.
+Data stored in iRODS is accessible from tetralith.
 We have got the command line client installed on the cluster.
 You should have the client configuration files in place under

← Older revision		Revision as of 09:58, 1 June 2020
Line 1:		Line 1:
		+	= Swestore iRODS at NSC =

−	= ~~Swestore IRODS~~ at NSC =	+	iRODS is a distributed storage management system which enables the user to
		+	control the access to his/her data via policies which are expressed as rules.
		+	Basic information about how to store and access data had been described above.
		+	Additional information about iRODS can be found at [http://www.irods.org the iRODS web site].
		+
		+	== iRODS system at NSC ==
		+
		+	At NSC we host a replica metadata server and two storage server nodes at the moment.
		+	All data stored is replicated between PDC and NSC.
		+
		+	== Accessing iRODS from the compute cluster ==
		+
		+	Data stored in iRODS can be access from tetralith.
		+	We have got the command line client installed on the cluster.
		+	You should have the client configuration files in place under
		+	the <code>.irods</code> directory as it has been described under
		+	the introductory pages.
		+	Issue:
		+	module load irods-client/4.2.6
		+	after which you authenticate using <code>iinit</code> and then the iCommands like
		+	<code>iget</code>, <code>iput</code>, <code>ils</code> should work as expected.