Research Platform Services Wiki

If you're a researcher, we'll help you do stuff.

User Tools

Site Tools


data_management:mediaflux:howto:downloaddata

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
data_management:mediaflux:howto:downloaddata [2018/07/17 13:48]
Neil Killeen
data_management:mediaflux:howto:downloaddata [2019/10/31 15:03] (current)
Neil Killeen [1 Authenticated Users]
Line 9: Line 9:
 ==== 1 Authenticated Users ==== ==== 1 Authenticated Users ====
  
-This means you have an account and can log in directly to Mediaflux. ​ The details on how to login with various protocols and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]). ​ +This means you have an account and can log in directly to Mediaflux. ​ The details on how to login with various protocols and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]). ​
  
-=== 1.1 Spartan ​Users at the University of Melbourne ​===+=== 1.1 Generic ​Users ===
  
  
-Some of the Mediaflux tools listed below are provide for Spartan ​ users via the Spartan module system. Please read the [[ data_management:​mediaflux:​howto:​downloaddata_with_spartan | details here.]] 
  
-=== 1.2 Generic Users ===+We recommend that best way to download data is either to use sFTP (if you are already familiar with it and have a client installed), or use the download client that UniMelb supplies (see below).
  
-This means you have an account and can log in directly to Mediaflux. ​ The details on how to login with various protocols and interfaces are found in (see [[data_management:​mediaflux:​howto:​access|access mechanisms]]).  ​ 
  
-== 1.2.1 From the Vendor [Arcitecta] ==+== 1.1.1 From the Vendor [Arcitecta] ==
  
    * [[data_management:​mediaflux:​howto:​downloaddata:​sftp | Secure FTP]] - e.g. FileZilla or CyberDuck    * [[data_management:​mediaflux:​howto:​downloaddata:​sftp | Secure FTP]] - e.g. FileZilla or CyberDuck
    * [[data_management:​mediaflux:​howto:​downloaddata:​smb2 | SMB2 (Network File Share of which CIFS is a type of SMB) ]] - OSX Finder and Windows Explorer have this built in    * [[data_management:​mediaflux:​howto:​downloaddata:​smb2 | SMB2 (Network File Share of which CIFS is a type of SMB) ]] - OSX Finder and Windows Explorer have this built in
    * [[data_management:​mediaflux:​howto:​downloaddata:​explorer | The Mediaflux Explorer ]] - Specialised Java 8 Interactive Drag-and-drop graphical application ​    * [[data_management:​mediaflux:​howto:​downloaddata:​explorer | The Mediaflux Explorer ]] - Specialised Java 8 Interactive Drag-and-drop graphical application ​
 +      * Has the ability to send data to end points called ''​sinks''​ (e.g. a remote SCP or sFTP server)
    * [[data_management:​mediaflux:​howto:​downloaddata:​desktop | Mediaflux Desktop ]] - Web-browser based Interface ​    * [[data_management:​mediaflux:​howto:​downloaddata:​desktop | Mediaflux Desktop ]] - Web-browser based Interface ​
-   * [[data_management:​mediaflux:​howto:​downloaddata:​aterm | ATERM Graphical Terminal ]] - Java 8 Interactive terminal application (you type Mediaflux commands into it)Advanced ​users only.+   * [[data_management:​mediaflux:​howto:​downloaddata:​aterm | ATERM Graphical Terminal ]] - Java 8 Interactive terminal application (you type Mediaflux commands into it) - **Advanced ​Users Only**.
  
-==  1.2.2 From Research Platform Services, The University of Melbourne == +==  1.1.2 From Research Platform Services, The University of Melbourne == 
  
-    * [[data_management:​mediaflux:​howto:​downloaddata:​unimelb-mf-download  ​UniMelb download client ​]] - Efficient, restartable,​ synchronizable, ​ download client.+Our clients can be downloaded from our    ​[[data_management:​mediaflux:​downloads|Available downloads]] page.
  
 +    * [[data_management:​mediaflux:​howto:​unimelb-mf-clients | download client ]] - Efficient, restartable,​ parallel, synchronizable, ​ download client (one of several clients described in this page)
 +    * [[data_management:​mediaflux:​howto:​unimelb-mf-clients ​ | checking client ]] - Efficient client that checks the local file system against assets in Mediaflux (one of several clients described on this page)
 ==== 2 Non-Authenticated Users ==== ==== 2 Non-Authenticated Users ====
  
Line 42: Line 43:
 Direct shareable links are ok for a small amount of data (they download in serial into a container and are not restartable). ​ The more complex indirect shareable links are much better for big data as they don't pack the data into a container, can be restarted and can download the data in parallel. Direct shareable links are ok for a small amount of data (they download in serial into a container and are not restartable). ​ The more complex indirect shareable links are much better for big data as they don't pack the data into a container, can be restarted and can download the data in parallel.
  
-   * [[data_management:​mediaflux:​howto:​downloaddata:​direct_shareable_links | Direct Shareable Links ]] +   * [[data_management:​mediaflux:​howto:​downloaddata:​direct_shareable_links | Direct Shareable Links - Details ]] 
-   * [[data_management:​mediaflux:​howto:​downloaddata:​indirect_shareable_links | In-direct Shareable Links ]]+      * A direct shareable link downloads data to a container (e.g. a tar or zip container). Once that is downloaded, the user must unpack the container (so double the storage is temporarily required)  
 +      * Direct links are not good for big data because if the process fails (e.g.network interruption) ​ you have to start again from the beginning and because twice the storage is required 
 +      * Direct shareable links can be provisioned by users via the Mediaflux Explorer (see the [[https://​wiki.cloud.unimelb.edu.au/​resplat/​doku.php?​id=data_management:​mediaflux:​howto:​explorer_video_guide| shareable link how to video on the Explorer how to videos page]] 
 +   * [[data_management:​mediaflux:​howto:​downloaddata:​indirect_shareable_links | In-direct Shareable Links - Details ​]]  
 +      * An indirect shareable link downloads a downloader (e.g. a script or application) which itself downloads the data 
 +      * These are best for big data because they can restart, may be able to download data in parallel and because they don't pack the data into a container (so you don't need double the storage)  
 +      * Currently, indirect shareable links can only be provisioned by specific users via scripts provided via ResPlat
data_management/mediaflux/howto/downloaddata.1531799338.txt.gz · Last modified: 2018/07/17 13:48 by Neil Killeen