Register a Workflow on Dockstore

Tutorial Goals

  • Discover how to register a workflow on Dockstore

  • Publish your workflow

This tutorial walks through the process of registering and sharing more complex workflows which are usually comprised of multiple tools, strung together in some sort of order (often a directed acyclic graph (DAG)). Workflows also differ from tools since they are not required to define their own environment. Instead, a workflow engine like Arvados or Cromwell, or an infrastructure like Galaxy will provide the ability to execute a CWL, WDL, or Galaxy workflow respectively.

This tutorial does not go through the creation of a workflow and its registration to GitHub, Bitbucket or GitLab. It assumes that you already have a repository which contains a workflow and are now trying to register it in Dockstore.

Register Your Workflow in Dockstore

Note

You do not need an account to search for workflows on Dockstore or to launch them with our compute partners. However, to register content on Dockstore, you must have an account on Dockstore and link the necessary third-party accounts. Once this is done you can register workflows from the My Workflows page, tools from the My Tools page, or services from the My Services page.

There are a variety of ways to get your workflows into Dockstore. GitHub App registration is the recommended way to register for all new workflows on Dockstore using GitHub. The legacy registration process is used for Bitbucket and GitLab.

A note on permissions

If you are adding the GitHub App to an organization for which you are not an admin, GitHub may block your ability to install the app, even if you have maintainer access to the repository you are hoping to give the GitHub App permission to view. Please see this FAQ entry for more information.

See also

Ensuring sychronization

Once the GitHub App is installed and a .dockstore.yml is present, please make sure to push one additional commit to your repository. This helps make sure your workflows, tools, and services show up in Dockstore.

Once you’ve installed our GitHub app on a repository or organization, you’ll need to add a dockstore.yml file to the root directory of a branch of the repository that contains your workflow. This file contains information like workflow path, test parameter file, workflow name, etc. When a push is made on GitHub to a branch with a .dockstore.yml, Dockstore will add that branch to the corresponding workflow on Dockstore. If the workflow doesn’t already exist on Dockstore, one will be created (but will not automatically be published publically). Note that a single dockstore.yml file can describe multiple workflows, if all of those workflows are in the same repository.

Below is a simple example of a .dockstore.yml file for an alignment workflow to show you how easy it is to use. Note that all file paths in the file must be absolute.

version: 1.2
workflows:
   - subclass: CWL
     primaryDescriptorPath: /aligner.cwl
     testParameterFiles:
     - /test/aligner.cwl.json

If you had our GitHub App installed on the repository myorg/alignments and then add the above .dockstore.yml to the develop branch, the following would occur:

  • A CWL workflow with the ID github.com/myorg/alignments will be created on Dockstore

  • The version develop is added to the workflow github.com/myorg/alignments

  • The version has the primary descriptor file set to /aligner.cwl

  • The version has one test parameter file: /test/aligner.cwl.json

Now that your workflow has been added, any time there is a push to a branch on GitHub for this repository that has a .dockstore.yml, it is automatically updated on Dockstore! Anytime there is a deletion of a branch on GitHub that has a .dockstore.yml, the version is removed from Dockstore.

For more information on this method, as well as general troubleshooting advice, please check our Dockstore GitHub Apps Overview page.

Legacy Registration Methods

Important

Workflows and tools added to Dockstore via our legacy registration methods do not automatically stay in sync with their source repository. Instead, someone with access to the entries on Dockstore must periodically log into Dockstore and press a button to trigger a refresh. Although this process is quick and will bring in all new tags, commits, and branches with the click of a button, it is easy to forget to do this and might not be appropriate for frequently-updated tools and workflows. For this reason, we recommend using GitHub App registration instead.

If you are using BitBucket or GitLab and would prefer not to use GitHub, or if you are using GitHub but do not wish to install our app, our legacy registration methods have you covered. Several options are available to you and described in our legacy registration methods documentation.

Sharing Your Workflow

After you have successfully added your workflow onto Dockstore and have it synced with GitHub, Bitbucket, or GitLab, you are now ready to share your workflow with the public! Assuming that your workflow has at least one valid version, you can publish your workflow for everyone to use. Simply select the workflow on the /my-workflows page and click publish.

For Terra Users : You have the ability to share hosted workflows through Dockstore. This allows for you to share workflows wth other users who have used their Google account to register on Terra. Learn more at Workflow Sharing.

Next Steps

You may not want to store your files directly with a service like GitHub. Perhaps you want your descriptor files to not be public. The solution is to use Hosted Tools and Workflows.