Language Support

Ideally, all features in Dockstore would be available in all languages. However, due to time constraints and gaps in our knowledge of different workflows, some features of Dockstore are not available in all languages.

To help lay out what parts of Dockstore are available in which languages, we cover the following guide for what features are available on the Dockstore site and in the Dockstore command-line utility.

Feature CWL WDL Nextflow Galaxy
Dockstore site
Tool registration Yes Yes No No
Workflow registration Yes Yes Yes Yes
Hosted Workflows Yes Yes Yes Yes
DAG Display Yes (cwl version <= 1.1) [1] Yes (wdl version <= 1.0) Limited support Limited support
Tool Tab Display Yes (cwl version <= 1.1) Yes (wdl version <= 1.0) Yes Limited support
Launch-with Platforms for workflows BioData Catalyst [2]
Cavatica [3]
CGC [4]
AnVIL [5]
BioData Catalyst [6]
DNAnexus
Terra [7]
Nextflow Tower[8] Galaxy Galaxy (in Terra)
Metadata Display Yes Yes (wdl version<=1.0) Yes Yes
Open Data Yes [11] Yes[12] No No
Dockstore CLI
Local workflow engines cwltool, Cromwell Cromwell [9] Nextflow None
File Provisioning In Local File System
HTTP
FTP
GCS via plugins
S3 via plugins
Data Object Service
Local File System
HTTP
FTP
GCS via plugins
S3 via plugins
Data Object Service
Local File System
HTTP
FTP
S3
No
Plugins Support GCS
s3
s3cmd
icgc-get
Data Object Service
GCS
s3
s3cmd
icgc-get
Data Object Service
No No
File Provisioning Out Local File System
HTTP
FTP
GCS via plugins
S3 via plugins
[10]
Local File System
HTTP
FTP
GCS via plugins
S3 via plugins
Local File System
S3
No
Notifications Yes Yes No No

[1] Available in both classic and CWL Viewer modes

[2] Does not support http(s) based imports. See BioData Catalyst Limitations for limitations.

[3] Does not support http(s) based imports. See Cavatica Limitations for limitations.

[4] Does not support http(s) based imports. See CGC Limitations for limitations.

[5] Does not support some file-path based imports. See AnVIL Limitations for limitations.

[6] Does not support some file-path based imports. See BioData Catalyst Limitations for limitations.

[7] Does not support some file-path based imports. See Terra Limitations for limitations.

[8] Gets workflow content directly from GitHub. See Nextflow Limitations for details.

[9] All verified Dockstore WDL tools/workflows were tested successfully. However, we anticipate that more testing is needed for WDL workflows that use language features not contained within that dataset.

[10] Use the Dockstore CLI optional parameter –wdl-output-target which allows you to specify a remote path to provision output files to ex: s3://oicr.temp/testing-launcher/

[11] For workflows and GitHub App Tools, not for Legacy Tools.

[12] For workflows only, not tools.

Converting File-path Based Imports to Public http(s) Based Imports for WDL

See https://cromwell.readthedocs.io/en/develop/Imports/ for general knowledge on imports.

Imports allow you to reference other files in your workflow. There are two types of resources that are supported in imports: http(s) and file-path based. Any public http(s) based URL can be used as the resource for an import, such as a website, GitHub, GA4GH compliant TRS endpoint, etc.

There are times when you may want to convert file-path based imports to public http(s) imports. One such reason is to ensure compatibility with Terra since it currently does not support file-path based imports. There are many different ways to convert to a public http(s) based import, the following are two examples.

You can host your file on GitHub and import it in the workflow descriptor like this:

import "https://raw.githubusercontent.com/DataBiosphere/topmed-workflows/1.11.0/variant-caller/variant-caller-wdl/topmed_freeze3_calling.wdl" as TopMed_variantcaller
import "https://raw.githubusercontent.com/DataBiosphere/topmed-workflows/1.11.0/variant-caller/variant-caller-wdl-checker/topmed-variantcaller-checker.wdl" as checker
...

Similarly, you can also host your file on a public google bucket and import it in the workflow descriptor like this:

import "https://storage.googleapis.com/..."
...