Difference between revisions of "User:Torben Rasmussen (NSC)/SweGrid old/Using national storage resources"
(→Introduction) |
m (moved SweGrid old/Using national storage resources to User:Torben Rasmussen (NSC)/SweGrid old/Using national storage resources) |
||
| (15 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
= Introduction = | = Introduction = | ||
| + | |||
| + | (UNDER DEVELOPMENT) | ||
The SNIC national storage infrastructure is implemented using dCache, which is a distributed storage system presenting the user with a single filesystem view. The front-end host of the system is srm://srm.swegrid.se which is the machine presenting the filesystem for the user. | The SNIC national storage infrastructure is implemented using dCache, which is a distributed storage system presenting the user with a single filesystem view. The front-end host of the system is srm://srm.swegrid.se which is the machine presenting the filesystem for the user. | ||
| − | The system uses VOMS proxy certificates for authentication. Please see, [[ | + | The system uses VOMS proxy certificates for authentication. Please see, [[Grid certificates#VOMS certificates|VOMS Certificates]] , for information on how to generate a voms proxy certificate. |
| + | |||
| + | = Copying files = | ||
| + | |||
| + | Copying files to and from resources is accomplished using the '''arccp''' command. The command can handle many of existing protocols, such as: ''http, https, httpg, ftp, gsiftp, lfc, rls, srm''. | ||
| + | |||
| + | == Copying single files == | ||
| + | |||
| + | Copying single files is accomplished in the same way as using the normal '''cp''' command as shown in the following example: | ||
| + | |||
| + | <pre>[jonas@localhost jonas2]$ arccp archive.tar.gz srm://srm.swegrid.se/ops/</pre> | ||
| + | |||
| + | == Recursive copying == | ||
| + | |||
| + | Recursive copying is accomplished using the '''--recursive''' option. The argument to the option determines the depth of the recursive copy. | ||
| + | |||
| + | <pre> | ||
| + | [jonas@localhost swestore]$ arccp --recursive=3 jonas2/ srm://srm.swegrid.se/ops/jonas/jonas2/ | ||
| + | </pre> | ||
= Listing files = | = Listing files = | ||
| − | = | + | Listing files on a resources is done using the '''arcls''' command. In the simplest form the command just takes a URL as input and displays names and directories without any extra information as shown in the following example: |
| + | |||
| + | <pre> | ||
| + | [jonas@localhost jonas2]$ arcls srm://srm.swegrid.se/ops/ | ||
| + | motd.1 | ||
| + | bla | ||
| + | generated | ||
| + | ops | ||
| + | motd.f343 | ||
| + | GridFTP-1.0.20-linux.gtk.x86.zip | ||
| + | test23 | ||
| + | test24 | ||
| + | ronstestfile | ||
| + | ronstestfile3 | ||
| + | testfile-put-1296854868-6468a2d5df07.txt | ||
| + | </pre> | ||
| + | |||
| + | Additional information can be listed by adding the '''--long''' option. | ||
| + | |||
| + | <pre> | ||
| + | [jonas@localhost swestore]$ arcls --long srm://srm.swegrid.se/ops | ||
| + | <Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency> | ||
| + | motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE | ||
| + | bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE | ||
| + | generated dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE | ||
| + | ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE | ||
| + | motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE | ||
| + | GridFTP-1.0.20-linux.gtk.x86.zip file 30479220 2011-03-09 09:49:20 (n/a) adler32:5c67686b ONLINE | ||
| + | test23 file 0 2011-03-04 09:44:44 (n/a) adler32:00000001 ONLINE | ||
| + | test24 file 0 2011-03-04 09:45:07 (n/a) adler32:00000001 ONLINE | ||
| + | ronstestfile file 4 2010-11-23 17:46:13 (n/a) adler32:03cd014b ONLINE | ||
| + | ronstestfile3 file 4 2010-11-23 17:49:07 (n/a) adler32:03cd014b ONLINE | ||
| + | testfile-put-1296854868-6468a2d5df07.txt file 20 2011-02-04 22:28:47 (n/a) adler32:1a400272 ONLINE | ||
| + | </pre> | ||
| + | |||
| + | Metadatainformation on a specific file can be listed by specifying the '''-m''' or '''--metadata''' option. In the following example the metadata information of the '''test23''' file is shown: | ||
| + | |||
| + | <pre> | ||
| + | [jonas@localhost swestore]$ arcls --metadata srm://srm.swegrid.se/ops/test23 | ||
| + | /ops/test23 | ||
| + | accessperm:rw-r----- | ||
| + | checksum:adler32:00000001 | ||
| + | ctime:2011-03-04 09:44:44 | ||
| + | filestoragetype:PERMANENT | ||
| + | group:25001 | ||
| + | latency:ONLINE | ||
| + | lifetimeassigned:PT1S | ||
| + | lifetimeleft:PT1S | ||
| + | mtime:2011-03-04 09:44:45 | ||
| + | owner:25001 | ||
| + | path:/ops/test23 | ||
| + | size:0 | ||
| + | spacetokens: | ||
| + | type:file | ||
| + | </pre> | ||
| + | |||
| + | = Creating directories = | ||
| + | |||
| + | There are no command line tool yet for creating directories. This command will probably be added in a upcoming release. In the meantime the following procedure can be used: | ||
| + | |||
| + | <pre> | ||
| + | [jonas@localhost swestore]$ mkdir mydir | ||
| + | [jonas@localhost swestore]$ touch mydir/dummyfile | ||
| + | [jonas@localhost swestore]$ arccp ./mydir/ srm://srm.swegrid.se/ops/mydir/ | ||
| + | [jonas@localhost swestore]$ arcls srm://srm.swegrid.se/ops/mydir | ||
| + | dummyfile | ||
| + | </pre> | ||
| + | |||
| + | The dummmy file can be removed using the arcrm command described in the following sections. | ||
= Removing files = | = Removing files = | ||
| − | = | + | <pre>[jonas@localhost swestore]$ arcrm srm://srm.swegrid.se/ops/mydir/dummyfile</pre> |
| + | |||
| + | = Removing directories = | ||
| + | |||
| + | To remove directories they have to be empty. To indicate a directory the URL should end with a '''/'''. | ||
| + | |||
| + | <pre>[jonas@localhost swestore]$ arcrm srm://srm.swegrid.se/ops/mydir/</pre> | ||
Latest revision as of 14:31, 8 July 2011
Contents
Introduction
(UNDER DEVELOPMENT)
The SNIC national storage infrastructure is implemented using dCache, which is a distributed storage system presenting the user with a single filesystem view. The front-end host of the system is srm://srm.swegrid.se which is the machine presenting the filesystem for the user.
The system uses VOMS proxy certificates for authentication. Please see, VOMS Certificates , for information on how to generate a voms proxy certificate.
Copying files
Copying files to and from resources is accomplished using the arccp command. The command can handle many of existing protocols, such as: http, https, httpg, ftp, gsiftp, lfc, rls, srm.
Copying single files
Copying single files is accomplished in the same way as using the normal cp command as shown in the following example:
[jonas@localhost jonas2]$ arccp archive.tar.gz srm://srm.swegrid.se/ops/
Recursive copying
Recursive copying is accomplished using the --recursive option. The argument to the option determines the depth of the recursive copy.
[jonas@localhost swestore]$ arccp --recursive=3 jonas2/ srm://srm.swegrid.se/ops/jonas/jonas2/
Listing files
Listing files on a resources is done using the arcls command. In the simplest form the command just takes a URL as input and displays names and directories without any extra information as shown in the following example:
[jonas@localhost jonas2]$ arcls srm://srm.swegrid.se/ops/ motd.1 bla generated ops motd.f343 GridFTP-1.0.20-linux.gtk.x86.zip test23 test24 ronstestfile ronstestfile3 testfile-put-1296854868-6468a2d5df07.txt
Additional information can be listed by adding the --long option.
[jonas@localhost swestore]$ arcls --long srm://srm.swegrid.se/ops <Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency> motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE generated dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE GridFTP-1.0.20-linux.gtk.x86.zip file 30479220 2011-03-09 09:49:20 (n/a) adler32:5c67686b ONLINE test23 file 0 2011-03-04 09:44:44 (n/a) adler32:00000001 ONLINE test24 file 0 2011-03-04 09:45:07 (n/a) adler32:00000001 ONLINE ronstestfile file 4 2010-11-23 17:46:13 (n/a) adler32:03cd014b ONLINE ronstestfile3 file 4 2010-11-23 17:49:07 (n/a) adler32:03cd014b ONLINE testfile-put-1296854868-6468a2d5df07.txt file 20 2011-02-04 22:28:47 (n/a) adler32:1a400272 ONLINE
Metadatainformation on a specific file can be listed by specifying the -m or --metadata option. In the following example the metadata information of the test23 file is shown:
[jonas@localhost swestore]$ arcls --metadata srm://srm.swegrid.se/ops/test23 /ops/test23 accessperm:rw-r----- checksum:adler32:00000001 ctime:2011-03-04 09:44:44 filestoragetype:PERMANENT group:25001 latency:ONLINE lifetimeassigned:PT1S lifetimeleft:PT1S mtime:2011-03-04 09:44:45 owner:25001 path:/ops/test23 size:0 spacetokens: type:file
Creating directories
There are no command line tool yet for creating directories. This command will probably be added in a upcoming release. In the meantime the following procedure can be used:
[jonas@localhost swestore]$ mkdir mydir [jonas@localhost swestore]$ touch mydir/dummyfile [jonas@localhost swestore]$ arccp ./mydir/ srm://srm.swegrid.se/ops/mydir/ [jonas@localhost swestore]$ arcls srm://srm.swegrid.se/ops/mydir dummyfile
The dummmy file can be removed using the arcrm command described in the following sections.
Removing files
[jonas@localhost swestore]$ arcrm srm://srm.swegrid.se/ops/mydir/dummyfile
Removing directories
To remove directories they have to be empty. To indicate a directory the URL should end with a /.
[jonas@localhost swestore]$ arcrm srm://srm.swegrid.se/ops/mydir/
