Difference between revisions of "Accessing Swestore with rclone"
(→Configuration) |
(→Configuration) |
||
Line 49: | Line 49: | ||
: <code>n</code> for '''New remote''' | : <code>n</code> for '''New remote''' | ||
: <code>swestore</code> for '''name''' | : <code>swestore</code> for '''name''' | ||
− | : <code> | + | : <code>37</code> for '''Storage''' (Or what ever number is Webdav, this changes between releases) |
: <code><nowiki>https://webdav.swestore.se</nowiki></code> for '''url''' | : <code><nowiki>https://webdav.swestore.se</nowiki></code> for '''url''' | ||
: <code>4</code> for '''vendor''' (Other) | : <code>4</code> for '''vendor''' (Other) |
Revision as of 15:51, 15 February 2021
This guide describes how to use the Rclone WebDAV client for storing and retrieving files from Swestore. Rclone is versatile and supports many protocols through a simple command line interface (CLI).
Contents
Requirements
To access Swestore using the rclone you need to be a member of a Swestore storage project, see Swestore#Getting access to Swestore.
To install rclone on your own computer, please follow your systems instructions at the official rclone documentation pages here.
Quickstart
Swestore access URL
The WebDAV access URL for Swestore should be specified for rclone
as;
Basic commands
rclone config
- configure rclone. See Rclone configuration for details.rclone ls
- for listing files. Works similarly tols
. See Rclone ls.rclone copyto
- for copying directories or separate files. Works similarly tocp
with wildcards. See Rclone copyto.rclone copy
- for copying contents of directories. Works similarly tocp -R path/*
. See Rclone copy.rclone mkdir
- for creating directories. Works similarly tomkdir -p
. See Rclone mkdir.rclone deletefile
- for removing specific files. Works similarly torm
. See Rclone deletefile.- More powerful removal functions are available, but be careful with these;
rclone delete
- for removing all files under path - i.e. you loose your data. Works similarly tofind path -type f | xargs rm
. See Rclone delete.rclone rmdir
- removing path if empty. Works similarly torm -d
. See Rclone rmdir.rclone rmdirs
- removing all empty directories under path. Works similarly tofind path -type d | awk '{ print length, $0 }' | sort -nsr | cut -d" " -f2- | xargs rm -d
(example modified from stackoverflow). See Rclone rmdirs.rclone purge
- removing all data under path, including path - i.e. you loose your data. Works similarly torm -rf
. See Rclone purge.
- More powerful removal functions are available, but be careful with these;
Use man
and --help
to get more info on rclone and its commands. Examples: man rclone
, rclone --help
or rclone copy --help
.
Paths
The rclone commands supports multiple storage protocols. Given that you have configured https://webdav.swestore.se
as the swestore
remote, we recommend using WebDAV with paths on the form swestore:/snic/YOUR_PROJECT_DIR/...
.
Configuration
You have to configure rclone with your access protocol, URL and user login credentials for it to work. Simply issue the following configuration command to interactively configure rclone;
$ rclone config
For configuring WebDAV, this amounts to answering something along the lines;
n
for New remoteswestore
for name37
for Storage (Or what ever number is Webdav, this changes between releases)https://webdav.swestore.se
for url4
for vendor (Other)yourusername
for user, should be on the formats_user
y
for Yes type in my own password- then enter your swestore password twice
- Just press
<Enter>
for bearer_token n
for Edit advanced config?y
if you think the resulting config is correct, otherwisee
to edit again.q
to Quit config
To see your configuartion afterwards, run
$ cat ~/.config/rclone/rclone.conf
You can also list your configured remotes by issuing
$ rclone listremotes
In the following sections, we are assuming your swestore remote is named swestore
Copying files
Copying files to and from resources is accomplished using the rclone copy and copyto command.
Copying single files
Copying single files is accomplished in the same way as using the normal cp command as shown in the following example:
$ rclone copyto archive.tar.gz swestore:/snic/YOUR_PROJECT_DIR/archive.tar.gz
You can also use copyto in order to rename the file in the process of copying it to the destination, by specifying a different filename on the remote.
Recursive copying
Recursive copying of a directory is accomplished using the copy command. The command will only copy files that have changed on the source compared to the destination, which is determined by checksums and timestamps. Observe that the source directory is not copied over, only its contents. Also, empty directories are omitted.
Example:
$ rclone copy /path/to/src swestore:/snic/YOUR_PROJECT_DIR/DESTINATION_DIRECTORY
The option --no-traverse can be used to not list files on the destination (good for huge directories). --max-age can be used to select the most recently modified files for transfer, and -P gives you status on progress.
Example, copying the last days modifications, with progress:
$ rclone copy --max-age 24h --no-traverse -P /path/to/src swestore:/snic/YOUR_PROJECT_DIR/DESTINATION_DIRECTORY
NOTE: The above example will copy all files in the directory src
into
the destination directory DESTINATION_DIRECTORY
. If you want the directory src
to be part of the destination path you have to explicitly supply it as shown in the example below:
$ rclone copy /path/to/src swestore:/snic/YOUR_PROJECT_DIR/DESTINATION_DIRECTORY/src
Listing
Rclone supports listing all files under a path; you can list recursively with the -R flag.
Listing files and directories on a resources is done using the ls or one of the ls* commands (see below). The simple ls command only lists objects and their sizes;
$ rclone ls swestore:/snic/YOUR_PROJECT_DIR
Further functionality can be be achieved from using any of the the following ls* commands;
lsl
long listing with additional infolsd
list only directorieslsf
list objects and directories, in a fashion good for scriptinglsjson
gives advanced output in JSON format
Example:
$ rclone lsl swestore:/snic/YOUR_PROJECT_DIR
Creating directories
Directories are generally created on demand. If you copy a file with the destination /snic/YOUR_PROJECT_DIR/newdir/dummyfile the newdir directory will be created if missing. But you can explicitly create directories using the mkdir command.
$ rclone mkdir swestore:/snic/YOUR_PROJECT_DIR/newdir
Removing files or directories
Beware that the command delete will recursively delete all file objects under specified path!
To remove the file dummyfile
under /snic/YOUR_PROJECT_DIR/newdir
,
$ rclone deletefile swestore:/snic/YOUR_PROJECT_DIR/newdir/dummyfile
To remove a directory, they have to be empty, and you use the command
$ rclone rmdir swestore:/snic/YOUR_PROJECT_DIR/newdir/
To remove all empty directories under a path, use
$ rclone rmdirs swestore:/snic/YOUR_PROJECT_DIR/newdir/
To recursively remove all files under /snic/YOUR_PROJECT_DIR/newdir
, leaving the empty directory structure in place, be careful;
$ rclone delete swestore:/snic/YOUR_PROJECT_DIR/newdir/
Do not use the command purge, as that will delete everything under the path specified.
FAQ
Q: I used rclone purge
or rclone delete
and have now deleted all my files; can I get them back somehow?
- A: No. Swestore does not currently support recovery of data for which the user has explicably requested to be deleted from the system, be it intentionally or by mistake. Therefore caution is advised while using powerful tools such as
rclone
.