Set Up File Provisioning Plugins
Note
Currently, the Dockstore CLI’s GS plugin does not support files that are located in requester-pays buckets.
File provisioning plugins can be used to extend the Dockstore CLI to allow for input and output files to be downloaded or uploaded to new and/or proprietary services. By default, the Dockstore CLI tries to read input/output files from the local filesystem and copy/hard-link to them. If that does not work, it will look at whether any of the installed plugins can handle the files. Finally, it will try using commons-vfs2 to handle the files.
Generally, the Dockstore CLI downloads a default set of file
provisioning plugins, Amazon S3, Synapse, DOS, GS, and ICGC Storage
Client, when it first installs. However, if you chose to download and
configure the CLI yourself via step 3 of Part 1 during the
quickstart tutorial and did not
have a Dockstore account set up, then no file provisioning plugins will
have downloaded. To check if you have the plugins, run the command
dockstore plugin list
. You should see something like

If not, run dockstore plugin download
.
The specification file for the plugins will be located at
~/.dockstore/plugins.json
by default and is created the first time
you run the command dockstore plugins download
. The location can be
overridden by using the key plugins-json-location
in your Dockstore
config file.
Configuration for plugins can be added to your
~/.dockstore/config
and is indexed by id. For example:
token = <your generated by the dockstore site>
server-url = https://www.dockstore.org/api
# options below this are optional
use-cache = false #set this to true to cache input files for rapid development
cache-dir = /home/<user>/.dockstore/cache #set this to determine where input files are cached (should be the same filesystem as your tool working directories)
[dockstore-file-synapse-plugin]
[dockstore-file-s3-plugin]
endpoint = #set this to point at a non AWS S3 endpoint
[dockstore-file-icgc-storage-client-plugin]
client = /media/large_volume/icgc-storage-client-1.0.23/bin/icgc-storage-client
To try an example and learn more about file provisioning, check out this tutorial.
If you would like to create your own plugins, follow along here.