Register a Workflow on Dockstore
Discover how to register a workflow on Dockstore
Publish your workflow
This tutorial walks through the process of registering and sharing more complex workflows which are usually comprised of multiple tools, strung together in some sort of order (often a directed acyclic graph (DAG)). Workflows also differ from tools since they are not required to define their own environment. Instead, a workflow engine like Arvados or Cromwell, or an infrastructure like Galaxy will provide the ability to execute a CWL, WDL, or Galaxy workflow respectively.
This tutorial does not go through the creation of a workflow and its registration to GitHub, Bitbucket or GitLab. It assumes that you already have a repository which contains a workflow and are now trying to register it in Dockstore.
You do not need an account to search for workflows on Dockstore or to launch them with our compute partners. However, to register content on Dockstore, you must have an account on Dockstore and link the necessary third-party accounts. Once this is done you can register workflows from the My Workflows page, tools from the My Tools page, services from the My Services page, or notebooks from the My Workflows page (currently in preview mode).
There are a variety of ways to get your workflows into Dockstore. GitHub App registration is the recommended way to register for all new workflows on Dockstore using GitHub. The legacy registration process is used for Bitbucket and GitLab.
This is the newest way of getting content onto Dockstore and is by far the most automated. Using GitHub Apps, Dockstore can react to changes on GitHub as they are made, keeping Dockstore synced with GitHub automatically.
Installing the GitHub App is simple. Navigate to
/my-services by navigating to My Dashboard then selecting the desired option in the left sidebar. In these screenshots, we will go via
/my-tools, but the process is essentially the same for any of the other options.
Since Dockstore Notebooks is still in preview mode, it does not have a dedicated
/my-notebooks page yet. To register a notebook, navigate to
/my-workflows in the step above.
Register a Tool button on the left hand sidebar.
A window will appear asking how you would like to register your tool, workflow, or service. Select
Register using GitHub Apps.
+ Manage Dockstore Installations on GitHub.
You’ll then be redirected to GitHub where you can install the app in an organization or your personal account. For example, the username for my personal GitHub account is aofarrel. If I want to install the GitHub App so it could access aofarrel/mycoolrepo, I would choose the first option here.
After selection of an organization or a personal account, you can select whether to give access to all current and future repositories or only select ones. If the organization or personal account you choose is intended to be just for Dockstore tools/workflows/services/notebooks, you may want to allow access to all repositories. Otherwise, it may be more intuitive to select only certain repositories. Click save and you will be taken back to the page you started on in Dockstore – either
/my-services, depending on where you started.
The GitHub user who first adds a workflow onto Dockstore must correspond to a user on Dockstore.
On Dockstore, under the GITHUB section, you should see the names of GitHub accounts that you have access to, such as organizations that you belong to and your personal account. If your repositories that you chose to keep track of contained a .dockstore.yml at the time of installing the GitHub App, then you will see the repositories under the GitHub personal/organization account name that it belongs to. Here’s an example involving
You will not see unpublished notebooks because there is not a dedicated
/my-notebooks page yet. To view your notebook, which should’ve been configured to automatically publish on Dockstore via the .dockstore.yml, navigate to
and locate your notebook in the list of published notebooks.
Only organization admins and repository admins can install the Dockstore GitHub App.
Organization admins will have the easiest time installing the Dockstore GitHub App because they can install it to any repository in the organization on the installation page. Users who are not organization admins can only install the Dockstore GitHub App on repositories that they are an admin of.
For more information on troubleshooting GitHub App permissions, please see this FAQ entry.
Automatic Syncing with GitHub Apps and .dockstore.yml - details on writing a .dockstore.yml file
Migrating Your Existing Workflows - a tutorial on converting already registered workflows
Troubleshooting and FAQ - tips on resolving Dockstore Github App issues.
Upon installing the GitHub App, Dockstore will find branches in your repository that contain a .dockstore.yml and attempt to register your workflows, tools, services, and notebooks. There may be some cases where Dockstore is unable to find all branches containing a .dockstore.yml, for example, if the GitHub repository has many branches.
If your workflow, tool, service, or notebook is not showing up on Dockstore after 5 minutes, push one additional commit to the branch in your repository that contains the .dockstore.yml that’s not being synchronized. Dockstore will synchronize the branch that was updated, which helps make sure that your workflows, tools, services, and notebooks show up in Dockstore.
If your workflow, tool, service, or notebook still doesn’t show up, check the GitHub App logs to see if Dockstore encountered an error while processing your .dockstore.yml.
Once you’ve installed our GitHub app on a repository in an organization or your personal account, you’ll need to add a dockstore.yml file to the root directory of a branch of the repository that contains your workflow. This file contains information like workflow path, test parameter file, workflow name, etc. When a push is made on GitHub to a branch with a .dockstore.yml, Dockstore will add that branch to the corresponding workflow on Dockstore. If the workflow doesn’t already exist on Dockstore, one will be created (but will not automatically be published publically). Note that a single dockstore.yml file can describe multiple workflows, if all of those workflows are in the same repository.
Below is a simple example of a .dockstore.yml file for an alignment workflow to show you how easy it is to use. Note that all file paths in the file must be absolute.
version: 1.2 workflows: - subclass: CWL primaryDescriptorPath: /aligner.cwl testParameterFiles: - /test/aligner.cwl.json
If you had our GitHub App installed on the repository
myorg/alignments and then add the above .dockstore.yml to the develop branch,
the following would occur:
A CWL workflow with the ID
github.com/myorg/alignmentswill be created on Dockstore
The version develop is added to the workflow
The version has the primary descriptor file set to
The version has one test parameter file:
Now that your workflow has been added, any time there is a push to a branch on GitHub for this repository that has a .dockstore.yml, it is automatically updated on Dockstore! Anytime there is a deletion of a branch on GitHub that has a .dockstore.yml, the version is removed from Dockstore.
For more information on this method, as well as general troubleshooting advice, please check our Dockstore GitHub Apps Overview page.
Workflows and tools added to Dockstore via our legacy registration methods do not automatically stay in sync with their source repository. Instead, someone with access to the entries on Dockstore must periodically log into Dockstore and press a button to trigger a refresh. Although this process is quick and will bring in all new tags, commits, and branches with the click of a button, it is easy to forget to do this and might not be appropriate for frequently-updated tools and workflows. For this reason, we recommend using GitHub App registration instead.
If you are using BitBucket or GitLab and would prefer not to use GitHub, or if you are using GitHub but do not wish to install our app, our legacy registration methods have you covered. Several options are available to you and described in our legacy registration methods documentation.
You may not want to store your files directly with a service like GitHub. Perhaps you want your descriptor files to not be public. The solution is to use Hosted Tools and Workflows.