Overhauling Dockstore Tools

This documentation explains:

  • How Dockstore tools worked historically before the 1.12 release

  • How tools registered with GitHub Apps work differently

  • How Dockstore envisions future support for tools

At a Glance

Differences

Legacy Tool

GitHub App Tool

Use case

User owns the Docker image

User doesn’t need to own the image

(or their tool does not use Docker)

Dockerfile

Dockerfile required

Dockerfile not required

Versioning

Based on image’s tags

Based off of branches/tags from git

repository

Path

Docker container location

GitHub repository location

Dockstore CLI

Run in tool mode

Run in workflow mode

Dockstore Tools Prior to 1.12 (Legacy Tools)

In a very broad sense, we defined a tool on Dockstore as a CWL or WDL program that performs a single task in a unique Docker container. Dockstore tool versions were based on the Docker image’s tags and we required that the image be owned by the Dockstore user. However, working on GitHub App registration for tools caused a lot of internal discussion because we realized that Dockstore tool support was confusing.

For starters, we created distinctions between tools and workflows even when one did not exist in one language’s specification. When Dockstore was created, CWL was the first descriptor language we supported. CWL does have a very clear distinction between a tool and a workflow. WDL was the next language Dockstore supported, but it does not have separate concepts for tools and workflows. At the time, we decided to create our own definition; we defined WDL tools as a WDL descriptor file that contained a single task and a WDL workflow as one that had more than one task.

Secondly, the requirement that the Docker image tied to the Dockstore tool must be owned by the user is problematic for three different reasons:

  1. Neither CWL or WDL require that a Docker image be specified (although using one is best practice)

  2. If you do specify a Docker image, it is not necessary that it be one you created or owned. It is perfectly valid to use someone else’s Docker image.

  3. Even when an image was specified the user, Dockstore did not enforce that the image registered was actual the one referenced in the descriptor file.

GitHub App Tools

In the 1.12 release, Dockstore introduced a new registration path for tools via GitHub Apps. We introduced this feature for two main reasons:

  1. Allow Dockstore tools to be automatically synced with the work that has been pushed to GitHub, and

  2. Let users register a tool without needing to specify a Docker image that the user owned

But, we also saw this feature as an opportunity to start eliminating some of the confusion surrounding Dockstore tools. As a result, we decided that GitHub App tools should:

  • Be the recommended way of registering tools

  • Not support “WDL tools” (but continue to support WDL workflows)

  • Have versioning based on the versioning on GitHub

Future of Dockstore Tools

Dockstore wants to continue simplifying and automating the registration process for tools. Eventually we want to:

  • Completely eliminate WDL tools and only support WDL workflows in order to match the language specification

  • Deprecate the other tool registration methods (legacy tools)

  • Continue to display already published tools that were created using the legacy tool registration methods