DERIVA Tools - Bulk upload and export

The DERIVA client tools are used for authenticating to the FaceBase server, bulk uploading of data, and bulk downloading of data. We provide both graphical desktop clients and command-line interface clients.

  • DERIVA Client Tools
    • DERIVA-Auth: a desktop client for establishing an authentication token.
    • DERIVA-Upload: a desktop client for uploading data files.
  • DERIVA Python command-line clients (Python 3.x)
    • deriva-upload-cli: a command-line utility for uploading data files.
    • deriva-download-cli: a command-line utility for downloading data files.

Installing DERIVA Client Tools

The DERIVA-Upload, DERIVA-Auth, and Command-Line clients are bundled for installation on Mac, Windows, and Linux.

System Requirements

  • macOS,
  • Windows, or
  • Linux (recent distros of Ubunto and Fedora preferred with Python 3+)

Installation steps for Windows and MacOS

  1. Go to the Deriva Client Tools Releases page.
  2. Download the appropriate file for your Operating System.
  3. Run the installer.

Installation steps for Linux

  1. Open a terminal
  2. Use the Python PIP command: pip3 install --user deriva-client

Note:

  • on Windows and MacOS, the installation will create launchers and icons for the desktop applications;
  • on Linux, the desktop applications can be invoked from a new terminal window with the commands deriva-auth and deriva-upload.

Authentication Tokens

The command-line clients (cli) can be run from the local host or a remote server, such as a compute cluster used to process data. When running on a remove server, the deriva-upload-cli and deriva-download-cli utilities require an authentication token, so you must be able to install and run the desktop DERIVA-Auth utility on a Windows, macOS, or Linux desktop even if you plan to upload/download data files using the CLI from another server. The CLI can be run remotely from the DERIVA-Auth.

Most likely if you are transferring large files to/from a cluster, you will run DERIVA-Auth on your local desktop to establish and maintain the authentication token, while you run the deriva-upload-cli or deriva-download-cli from your cluster just passing it the authentication token that you established with DERIVA-Auth as a parameter when you invoke the CLI.

DO NOT SHARE YOUR TOKEN

The authentication token is equivalent to a short-term, temporary password, in simple terms. Treat it as you would your FaceBase username and password.

  • Do not share it with anyone.
  • Do not copy and paste it into an email.

Authentication Token Lifetime

The authentication token will expire in 30 minutes by default. However, the DERIVA-Auth client will refresh the token so long as you leave the DERIVA-Auth client open and running on your desktop.

For long transfers, keep the authentication agent running on your local system while the remote transfer is in progress. If you close the DERIVA-Auth client, any ongoing transfers (past the 30 minute limit) will fail and report ‘not authorized’ errors.

Establish an Authentication Token

[[images/deriva-clients-auth-tok.jpeg]]

  1. Open the DERIVA-Auth application
  2. First time use:
    • In the “Server:” input box enter www.facebase.org
    • Click Add
  3. Subsequent use:
    • Ensure that the “Server:” field indicates www.facebase.org
    • Click Login
  4. Follow the prompts in the main panel to login
  5. Copy the “bearer” token:
    • Click “Show Token”
    • Click “Show Details…”
    • Copy the token
    • Click “Close”

If there are any errors, they should be reported in the status panel beneath the file listing panel.

Terminate an Authentication Token

When you are finished using the authentication token, click “Logout” and exit the application. When you do this, the token will be invalidated immediately.