Difference between revisions of "Accessing Swestore with the ARC client"

From SNIC Documentation
Jump to: navigation, search
(ARC 0.8 versus 1.0)
(Swestore documentation moved)
(Tag: New redirect)
 
(37 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:Grid computing]]
+
#REDIRECT[[Swestore Documentation Moved]]
[[Category:Storage]]
 
[[Category:SweStore]]
 
[[Category:SweGrid user guide]]
 
[[Getting started with SweGrid|< Getting started with SweGrid]]<br>
 
[[Swestore National Storage|< Swestore National Storage]]
 
 
 
(PAGE UNDER DEVELOPMENT)
 
 
 
= The ARC client =
 
 
 
This guide describes how to use the [http://www.nordugrid.org Nordugrid]
 
''ARC'' client for storing and retrieving files from SweStore National
 
Storage. The ARC client is usually used for sending grid jobs to grid
 
clusters, but it also contain commands for data management.
 
 
 
If your system doesn't have the ARC client installed you can look at
 
the [[ARC_client_installation|ARC client installation]] page in this
 
wiki or the official Nordugrid
 
[http://www.nordugrid.org/documents/arc-client-install.html ARC
 
installation] page for more information. Please note that the windows
 
and MacOS clients are a bit unstable.
 
 
 
A complete user guide for the ARC client can be found in
 
http://www.nordugrid.org/documents/arc-ui.pdf.
 
 
 
ARC can handle many different file transfer protocols, such as:
 
''http, https, httpg, ftp, gsiftp, lfc, rls, srm''. But only the srm
 
protocol is currently is supported when accessing SweStore using the
 
ARC client.
 
 
 
== ARC 0.8 versus 1.0 ==
 
 
 
In late spring 2011 Nordugrid release the 1.0 version of ARC
 
(sometimes called 11.05). One of the new features of 1.0 compared to
 
the previous 0.8 release was a new command set. Basically most of the
 
ng* commands was replaced with the new arc* commands. Some
 
functionality moved between commands (ngstat became arcinfo and
 
arcstat) and some new commands was introduced (arcproxy as an
 
replacement for grid-proxy-init, which wasn't an arc command at all
 
but a part of the Globus Toolkit). There are still legacy compatibility
 
binaries in place for the old ng* commands, but I strongly suggest
 
that you use arc* when available.
 
 
 
If you on the same local account switch between ng* and arc* commands you may get warnings:
 
 
 
Bad format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 2.2
 
Unwrapped data does not fit into buffer
 
Connection to server failed: Connection refused
 
Connection to server failed: Connection refused
 
or
 
WARNING: Bad or old format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 gsi 2.2
 
WARNING: Bad or old format detected in file /home/jens/.arc/srms.conf, in line srm.swegrid.se 8443 gsi 2.2
 
 
 
There is a file, srm.conf, that gets automatically updated when
 
accessing a resource. ngls and arcls does not agree on the content of
 
that file. There are bug reports about it. That warning is just
 
confusing and shouldn't be displayed. Another attempt using the same
 
command will probably not display those errors again.
 
 
 
== arcproxy 1.0.1 ==
 
 
 
There us a bug in dCache which makes proxy certificates from arcproxy
 
1.0.1 unusable. The error you get is:
 
 
 
ERROR: Failed listing files
 
 
 
All other version of arcproxy should be fine. If you encounter this
 
version av arcproxy, please use grid-proxy-init if available. The
 
generated proxy certificates should be equivalent.
 
 
 
= Preparations =
 
 
 
In addition to the installed ARC client you need the proper access rights and a valid grid proxy certificate. Please check the [[Grid certificates]] page for more information.
 
 
 
= Copying files =
 
 
 
Copying files to and from resources is accomplished using the '''arccp''' command.
 
 
 
== Copying single files ==
 
 
 
Copying single files is accomplished in the same way as using the
 
normal '''cp''' command as shown in the following example:
 
 
 
$ arccp archive.tar.gz srm://srm.swegrid.se/ops/
 
 
 
Please note the trailing / which marks the destination as a directory.
 
Without a / the destination will be a file, which may or may not be
 
what you wanted. All required directories are created when needed so
 
the destination may be a nonexisting directory.
 
 
 
== Recursive copying ==
 
 
 
Recursive copying is accomplished using the '''--recursive''' option
 
to arccp. The argument to the option determines the depth of the
 
recursive copy.
 
 
 
$ arccp --recursive=3 foobar/ srm://srm.swegrid.se/ops/foobar/
 
 
 
= Listing files =
 
 
 
Listing files on a resources is done using the '''arcls''' command. In
 
the simplest form the command just takes a URL as input and displays
 
names and directories without any extra information as shown in the
 
following example:
 
 
 
$ arcls srm://srm.swegrid.se/ops/
 
motd.1
 
bla
 
generated
 
ops
 
motd.f343
 
 
 
Additional information can be listed by adding the '''--long''' option:
 
 
 
$ arcls --long srm://srm.swegrid.se/ops
 
<Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency>
 
motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE
 
bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE
 
foobar dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE
 
ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE
 
motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE
 
 
 
Metadatainformation on a specific file can be listed by specifying the '''-m''' or '''--metadata''' option. In the following example the metadata information of the '''motd.1''' file is shown:
 
 
 
$ arcls --metadata srm://srm.swegrid.se/ops/motd.1
 
/ops/motd.1
 
accessperm:rw-r--r--
 
checksum:adler32:582d7718
 
ctime:2008-06-05 12:28:23
 
filestoragetype:PERMANENT
 
group:25001
 
latency:NEARLINE
 
lifetimeassigned:PT1S
 
lifetimeleft:PT1S
 
mtime:2011-03-24 12:57:42
 
owner:25001
 
path:/ops/motd.1
 
size:354
 
spacetokens:
 
type:file
 
 
 
= Creating directories =
 
 
 
There is no command line tool for creating directories. This command
 
will probably be added in an upcoming release. In the meantime the
 
following procedure can be used:
 
 
 
$ touch dummyfile
 
$ arccp dummyfile srm://srm.swegrid.se/ops/newdir/
 
$ arcls srm://srm.swegrid.se/ops/newdir/
 
dummyfile
 
 
 
The dummmyfile can be removed using the arcrm command described in the following section.
 
 
 
= Removing files or directories =
 
 
 
$ arcrm srm://srm.swegrid.se/ops/newdir/dummyfile
 
$ arcrm srm://srm.swegrid.se/ops/newdir/
 
 
 
To remove directories they have to be empty.
 

Latest revision as of 09:55, 8 February 2023