|
|
| (37 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | [[Category:Grid computing]] | + | #REDIRECT[[Swestore Documentation Moved]] |
| − | [[Category:Storage]]
| |
| − | [[Category:SweStore]]
| |
| − | [[Category:SweGrid user guide]]
| |
| − | [[Getting started with SweGrid|< Getting started with SweGrid]]<br>
| |
| − | [[Swestore National Storage|< Swestore National Storage]]
| |
| − | | |
| − | (PAGE UNDER DEVELOPMENT)
| |
| − | | |
| − | = The ARC client =
| |
| − | | |
| − | This guide describes how to use the [http://www.nordugrid.org Nordugrid]
| |
| − | ''ARC'' client for storing and retrieving files from SweStore National
| |
| − | Storage. The ARC client is usually used for sending grid jobs to grid
| |
| − | clusters, but it also contain commands for data management.
| |
| − | | |
| − | If your system doesn't have the ARC client installed you can look at
| |
| − | the [[ARC_client_installation|ARC client installation]] page in this
| |
| − | wiki or the official Nordugrid
| |
| − | [http://www.nordugrid.org/documents/arc-client-install.html ARC
| |
| − | installation] page for more information. Please note that the windows
| |
| − | and MacOS clients are a bit unstable.
| |
| − | | |
| − | A complete user guide for the ARC client can be found in
| |
| − | http://www.nordugrid.org/documents/arc-ui.pdf.
| |
| − | | |
| − | ARC can handle many different file transfer protocols, such as:
| |
| − | ''http, https, httpg, ftp, gsiftp, lfc, rls, srm''. But only the srm
| |
| − | protocol is currently is supported when accessing SweStore using the
| |
| − | ARC client.
| |
| − | | |
| − | == ARC 0.8 versus 1.0 ==
| |
| − | | |
| − | In late spring 2011 Nordugrid release the 1.0 version of ARC
| |
| − | (sometimes called 11.05). One of the new features of 1.0 compared to
| |
| − | the previous 0.8 release was a new command set. Basically most of the
| |
| − | ng* commands was replaced with the new arc* commands. Some
| |
| − | functionality moved between commands (ngstat became arcinfo and
| |
| − | arcstat) and some new commands was introduced (arcproxy as an
| |
| − | replacement for grid-proxy-init, which wasn't an arc command at all
| |
| − | but a part of the Globus Toolkit). There are still legacy compatibility
| |
| − | binaries in place for the old ng* commands, but I strongly suggest
| |
| − | that you use arc* when available.
| |
| − | | |
| − | If you on the same local account switch between ng* and arc* commands you may get warnings:
| |
| − | | |
| − | Bad format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 2.2
| |
| − | Unwrapped data does not fit into buffer
| |
| − | Connection to server failed: Connection refused
| |
| − | Connection to server failed: Connection refused
| |
| − | or
| |
| − | WARNING: Bad or old format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 gsi 2.2
| |
| − | WARNING: Bad or old format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 gsi 2.2
| |
| − | | |
| − | There is a file, srm.conf, that gets automatically updated when
| |
| − | accessing a resource. ngls and arcls does not agree on the content of
| |
| − | that file. There are bug reports about it. That warning is just
| |
| − | confusing and shouldn't be displayed. Another attempt using the same
| |
| − | command will probably not display those errors again.
| |
| − | | |
| − | == arcproxy 1.0.1 ==
| |
| − | | |
| − | There us a bug in dCache which makes proxy certificates from arcproxy
| |
| − | 1.0.1 unusable. The error you get is:
| |
| − | | |
| − | ERROR: Failed listing files
| |
| − | | |
| − | All other version of arcproxy should be fine. If you encounter this
| |
| − | version av arcproxy, please use grid-proxy-init if available. The
| |
| − | generated proxy certificates should be equivalent.
| |
| − | | |
| − | = Preparations =
| |
| − | | |
| − | In addition to the installed ARC client you need the proper access rights and a valid grid proxy certificate. Please check the [[Grid certificates]] page for more information.
| |
| − | | |
| − | = Copying files =
| |
| − | | |
| − | Copying files to and from resources is accomplished using the '''arccp''' command.
| |
| − | | |
| − | == Copying single files ==
| |
| − | | |
| − | Copying single files is accomplished in the same way as using the
| |
| − | normal '''cp''' command as shown in the following example:
| |
| − | | |
| − | $ arccp archive.tar.gz srm://srm.swegrid.se/ops/
| |
| − | | |
| − | Please note the trailing / which marks the destination as a directory.
| |
| − | Without a / the destination will be a file, which may or may not be
| |
| − | what you wanted. All required directories are created when needed so
| |
| − | the destination may be a nonexisting directory.
| |
| − | | |
| − | == Recursive copying ==
| |
| − | | |
| − | Recursive copying is accomplished using the '''--recursive''' option
| |
| − | to arccp. The argument to the option determines the depth of the
| |
| − | recursive copy.
| |
| − | | |
| − | $ arccp --recursive=3 foobar/ srm://srm.swegrid.se/ops/foobar/
| |
| − | | |
| − | = Listing files =
| |
| − | | |
| − | Listing files on a resources is done using the '''arcls''' command. In
| |
| − | the simplest form the command just takes a URL as input and displays
| |
| − | names and directories without any extra information as shown in the
| |
| − | following example:
| |
| − | | |
| − | $ arcls srm://srm.swegrid.se/ops/
| |
| − | motd.1
| |
| − | bla
| |
| − | generated
| |
| − | ops
| |
| − | motd.f343
| |
| − | | |
| − | Additional information can be listed by adding the '''--long''' option:
| |
| − | | |
| − | $ arcls --long srm://srm.swegrid.se/ops
| |
| − | <Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency>
| |
| − | motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE
| |
| − | bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE
| |
| − | foobar dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE
| |
| − | ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE
| |
| − | motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE
| |
| − | | |
| − | Metadatainformation on a specific file can be listed by specifying the '''-m''' or '''--metadata''' option. In the following example the metadata information of the '''motd.1''' file is shown:
| |
| − | | |
| − | $ arcls --metadata srm://srm.swegrid.se/ops/motd.1
| |
| − | /ops/motd.1
| |
| − | accessperm:rw-r--r--
| |
| − | checksum:adler32:582d7718
| |
| − | ctime:2008-06-05 12:28:23
| |
| − | filestoragetype:PERMANENT
| |
| − | group:25001
| |
| − | latency:NEARLINE
| |
| − | lifetimeassigned:PT1S
| |
| − | lifetimeleft:PT1S
| |
| − | mtime:2011-03-24 12:57:42
| |
| − | owner:25001
| |
| − | path:/ops/motd.1
| |
| − | size:354
| |
| − | spacetokens:
| |
| − | type:file
| |
| − | | |
| − | = Creating directories =
| |
| − | | |
| − | There is no command line tool for creating directories. This command
| |
| − | will probably be added in an upcoming release. In the meantime the
| |
| − | following procedure can be used:
| |
| − | | |
| − | $ touch dummyfile
| |
| − | $ arccp dummyfile srm://srm.swegrid.se/ops/newdir/
| |
| − | $ arcls srm://srm.swegrid.se/ops/newdir/
| |
| − | dummyfile
| |
| − | | |
| − | The dummmyfile can be removed using the arcrm command described in the following section.
| |
| − | | |
| − | = Removing files or directories =
| |
| − | | |
| − | $ arcrm srm://srm.swegrid.se/ops/newdir/dummyfile
| |
| − | $ arcrm srm://srm.swegrid.se/ops/newdir/
| |
| − | | |
| − | To remove directories they have to be empty.
| |
