Write-API Conversion Process
This is a service aimed at two tasks
Providing a concrete reference implementation of a proposed GA4GH Write API.
Providing a utility for developers to convert plain CWL/WDL files and Dockerfiles into GitHub repos storing those plain CWL/WDL files and Quay.io repos storing Docker images built from those Dockerfiles. This can be used by those converting tools described in other formats into “Dockstore-friendly” tools that can be quickly registered and published in Dockstore by using the Write API Client’s publish command or programmatically via the Dockstore API. It is an alternative to using a GUI to register tools on Dockstore.
This is intended to be used by:
Tool Migrators - Developers that have access to a large number of tools in some different format and wants to migrate them all programmatically to Dockstore with minimal effort.
Tool Developers - Developers of tools that wants a quick and simple way of creating one without spending a large amount of time to post a single Dockerfile and CWL descriptor to implement each tool.
Building/Downloading the API jars
You have the option to either build or download the API jars.
git clone https://github.com/dockstore/write_api_service.git
mvn clean install -DskipTests
The built jars will be available as:
Additionally, you can download the Write API jars using the following:
Note that the client one is a shaded jar.
Web Service Prerequisites
GitHub token - Learn how to create tokens on GitHub here. You will need the scope “repo”.
GitHub organization(s) - Your GitHub token must have access to at least one existing GitHub organization. The organization can be changed as long as the the GitHub token still has access to it. The Write API web service currently does not create GitHub organizations. The name of this organization must match the Quay.io organization. This organization will contain the repository that will be created.
Quay.io organization and Quay.io token
You will need an existing Quay.io organization. The Write API currently does not create Quay.io organizations. The name of this organization must match the GitHub organization. Once you have an organization, a token must be created for it.
This organization will contain the repository that will be created. Changing the Quay.io organization requires the token to be recreated/changed too.
Learn how to create a token on Quay.io for your organizations here under the heading “Generating a Token (for internal application use)”. You will need to provide these permissions:
View all visible repositories
Read/Write to any accessible repository
The web service alone only requires a GitHub and Quay.io token. There are two ways to specify your tokens.
Environmental variables. You can set them in Bash like the following:
export quayioToken=<your token here> export githubToken=<your token here>
The YAML configuration file. The tokens can be entered in the top two lines. An example of a YAML configuration file can be seen here.
Run the service using a configuration file:
java -jar write-api-service-*.jar server example.yml The example.yml
shown previously uses port 8082 by default, this can be changed. Note
this port number, it will later be used for the Write API Client
After running the webservice, you can check out the web service
endpoints through swagger. By default, it is available at
The basic workflow is that GitHub repos are created when posting a new tool. When files are posted or put to a version of a tool, we will create or delete and re-create a GitHub release/branch/tag with a matching name. When Dockerfiles are added, the tool will be created and built as a Quay.io repo. After adding both Dockerfiles and descriptors, you basically have a tool that is ready to be quickly registered and published under the Dockstore web service. Go to Dockstore, do a refresh, and then hit quick register on the repos that you wish to publish. You can also do this programmatically through the write api client
Write API web service and all its prerequisites
By now, then web service should be up and running with valid GitHub and Quay.io tokens. If not, please return to the web service usage section to get that running first. It is advised to ensure that the Write API web service is functioning correctly before using the client.
Follow the “Getting Started with Dockstore” tutorial to get a Dockstore token. Note this down, it will later be used in the Write API client properties file.
The Dockstore tutorial earlier would’ve specified the server-url
alongside the token. Unless you’re running your own dockstore
webservice, the Dockstore production server-url is
https://www.dockstore.org/api. Note this down as it will be used later
in the Write API client properties file.
In order to publish to Dockstore, Quay.io must be linked to Dockstore. See Dockstore on how to link your Quay.io account to Dockstore.
Write API web service URL
You will need to know the URL of the Write API web service you ran
previously. If you’ve been using the example.yml, it should be
The configuration file used by the write-api-client is located at ~/.dockstore/write.api.config.properties It should look something like this:
“dockstoreToken” is acquired from your account page on the dockstore website . “server-url” is the dockstore server url. “organization” is the organization/user of the repository to create. “repo” is the repository to create. “write-api-url” is the url of the write-api-service
Here is the general usage information for the client:
$ java -jar write-api-client-*-shaded.jar --help
Usage: client [options] [command] [command options]
Prints help for the client.
add Add the Dockerfile and CWL file(s) using the write API.
Usage: add [options]
The Dockerfile to upload
The cwl descriptor to upload
The optional secondary cwl descriptor to upload
Prints help for the add command
The version of the tool to upload to
publish Publish tool to dockstore using the output of the 'add'
Usage: publish [options]
Prints help for the publish command.
The json output from the 'add' command.
Sample Client Output
client add --Dockerfile Dockerfile --cwl-file Dockstore.cwl --cwl-secondary-file Dockstore2.cwl --version 3.0
You can pipe the output like this:
client add --Dockerfile Dockerfile --cwl-file Dockstore.cwl --cwl-secondary-file Dockstore2.cwl --version 3.0 > test.json
client publish --tool test.json