Research Platform Services Wiki

If you're a researcher, we'll help you do stuff.

User Tools

Site Tools


data_management:mediaflux:howto:downloaddata

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
data_management:mediaflux:howto:downloaddata [2018/07/19 15:04]
Neil Killeen
data_management:mediaflux:howto:downloaddata [2019/10/31 15:03] (current)
Neil Killeen [1 Authenticated Users]
Line 9: Line 9:
 ==== 1 Authenticated Users ==== ==== 1 Authenticated Users ====
  
-This means you have an account and can log in directly to Mediaflux. ​ The details on how to login with various protocols and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]). ​  +This means you have an account and can log in directly to Mediaflux. ​ The details on how to login with various protocols and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]). ​
  
 === 1.1 Generic Users === === 1.1 Generic Users ===
  
-This means you have an account and can log in directly ​to Mediaflux. ​ The details on how to login with various protocols ​and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]).  + 
 + 
 +We recommend that best way to download data is either ​to use sFTP (if you are already familiar ​with it and have a client installed), or use the download client that UniMelb supplies ​(see below). 
  
 == 1.1.1 From the Vendor [Arcitecta] == == 1.1.1 From the Vendor [Arcitecta] ==
Line 23: Line 25:
       * Has the ability to send data to end points called ''​sinks''​ (e.g. a remote SCP or sFTP server)       * Has the ability to send data to end points called ''​sinks''​ (e.g. a remote SCP or sFTP server)
    * [[data_management:​mediaflux:​howto:​downloaddata:​desktop | Mediaflux Desktop ]] - Web-browser based Interface ​    * [[data_management:​mediaflux:​howto:​downloaddata:​desktop | Mediaflux Desktop ]] - Web-browser based Interface ​
-   * [[data_management:​mediaflux:​howto:​downloaddata:​aterm | ATERM Graphical Terminal ]] - Java 8 Interactive terminal application (you type Mediaflux commands into it)Advanced ​users only.+   * [[data_management:​mediaflux:​howto:​downloaddata:​aterm | ATERM Graphical Terminal ]] - Java 8 Interactive terminal application (you type Mediaflux commands into it) - **Advanced ​Users Only**.
  
 ==  1.1.2 From Research Platform Services, The University of Melbourne ==  ==  1.1.2 From Research Platform Services, The University of Melbourne == 
  
-    * [[data_management:​mediaflux:​howto:​unimelb-mf-clients:​unimelb-mf-download ​ | UniMelb download client ]] - Efficient, restartable,​ synchronizable, ​ download client. +Our clients ​can be downloaded from our    ​[[data_management:​mediaflux:​downloads|Available downloads]] page.
-    [[data_management:​mediaflux:​howto:​unimelb-mf-clients:​unimelb-mf-check  ​UniMelb checking client ​]] - Efficient client that checks the local file system against assets in Mediaflux.+
  
 +    * [[data_management:​mediaflux:​howto:​unimelb-mf-clients | download client ]] - Efficient, restartable,​ parallel, synchronizable, ​ download client (one of several clients described in this page)
 +    * [[data_management:​mediaflux:​howto:​unimelb-mf-clients ​ | checking client ]] - Efficient client that checks the local file system against assets in Mediaflux (one of several clients described on this page)
 ==== 2 Non-Authenticated Users ==== ==== 2 Non-Authenticated Users ====
  
Line 40: Line 43:
 Direct shareable links are ok for a small amount of data (they download in serial into a container and are not restartable). ​ The more complex indirect shareable links are much better for big data as they don't pack the data into a container, can be restarted and can download the data in parallel. Direct shareable links are ok for a small amount of data (they download in serial into a container and are not restartable). ​ The more complex indirect shareable links are much better for big data as they don't pack the data into a container, can be restarted and can download the data in parallel.
  
-   * [[data_management:​mediaflux:​howto:​downloaddata:​direct_shareable_links | Direct Shareable Links ]] +   * [[data_management:​mediaflux:​howto:​downloaddata:​direct_shareable_links | Direct Shareable Links - Details ]] 
-   * [[data_management:​mediaflux:​howto:​downloaddata:​indirect_shareable_links | In-direct Shareable Links ]]+      * A direct shareable link downloads data to a container (e.g. a tar or zip container). Once that is downloaded, the user must unpack the container (so double the storage is temporarily required)  
 +      * Direct links are not good for big data because if the process fails (e.g.network interruption) ​ you have to start again from the beginning and because twice the storage is required 
 +      * Direct shareable links can be provisioned by users via the Mediaflux Explorer (see the [[https://​wiki.cloud.unimelb.edu.au/​resplat/​doku.php?​id=data_management:​mediaflux:​howto:​explorer_video_guide| shareable link how to video on the Explorer how to videos page]] 
 +   * [[data_management:​mediaflux:​howto:​downloaddata:​indirect_shareable_links | In-direct Shareable Links - Details ​]]  
 +      * An indirect shareable link downloads a downloader (e.g. a script or application) which itself downloads the data 
 +      * These are best for big data because they can restart, may be able to download data in parallel and because they don't pack the data into a container (so you don't need double the storage)  
 +      * Currently, indirect shareable links can only be provisioned by specific users via scripts provided via ResPlat
data_management/mediaflux/howto/downloaddata.1531976674.txt.gz · Last modified: 2018/07/19 15:04 by Neil Killeen