Changelog

Fixes include

  • parsing of Nextflow workflows that specify Docker container in process.container
  • wdl imports read fix
  • lazy load starred entries for performance boost
  • cwl relative import fix for workflows that use “../”
  • fix generated URLs in GA4GH TRS
  • fix for null verification sources

See a full list of our changes on GitHub

No description.

Highlighted new features include

  • initial support for Nextflow workflows
    • starts with support for registering, searching, indexing Nextflow workflow authors+descriptions, and visualizing Nextflow workflows with a rough and ready DAG, and display of Docker images used by a Nextflow workflow
    • full details on support language-by-language
  • initial support for hosted workflows with CWL and WDL
    • allows users to create and edit a limited number (and versions of) of workflows directly on the dockstore.org site for quick prototyping and iteration
    • hosted workflows also support all the regular features that you’re used to for regular workflows
    • download zip bundles of your hosted (actually, all) workflows to ensure that you can always get back your workflows
  • limited beta of permissions based sharing
    • this will eventually allow all users to share workflows with a limited number of other Dockstore users
    • initially, this feature will be limited to users (or rather Google accounts) that are known to the Firecloud ecosystem, but we hope to roll this out to all users over the coming months
  • login with Google and associate your account allowing you to login with either GitHub or Google
    • limited ability to change your username before performing major work on Dockstore
  • a file provisioning client plugin that supports the nascent GA4GH DOS service
    • this allows local development and launching of CWL and WDL workflows to download input files referenced by DOS
  • Launch-with support for DNANexus
  • language support updates
    • updates for WDL support from wdl4s_2.11 to wdl4s-wom_2.11 and Cromwell from 29 to 30.2
    • updates for cwltool support from 1.0.20170828135420 to 1.0.20180403145700
    • to elaborate, verified workflows have been run in-house succesfully with these versions
  • updates to use the Bitbucket 2.0 API and the Gitlab V4 API
  • a large number of UI improvements and usability improvements including, but not limited to
    • a much improved experience on mobile devices
    • display of commit ids and dates for the files we’re bringing in and whether they came from a tag or branch
    • CWL, WDL, Nextflow syntax highlighting thanks to the official grammars provided by these languages to GitHub
    • experimental display of extended verification information separated by platforms and contributed by curators
  • we’ve also added a privacy policy and are working on a terms of service

  • for those eager for inside baseball, we updated from Angular 4 to 6 including updates to the version of Typescript we’re using for the UI, much of the UI has been updated to use Angular Material, and we’ve refreshed many of our Java dependencies

See a full list of our changes on GitHub

Breaking changes

Major

  • beta support for the GA4GH tool registry schema service moves from 2.0.0-beta.1 to 2.0.0-beta.2
    • among other improvements, this simplifies the fields for containerfile, descriptors, and test json with further details on the breaking changes in particular here
  • old iterations of the Dockstore CLI that were previously able to launch unpublished entries of your own creation will no longer be able to without an update
    • published entries should be unaffected

Minor

  • the recommended default root of our API is now https://dockstore.org/api/ rather than https://dockstore.org:8443/
    • for example, https://dockstore.org:8443/swagger.json is deprecated in favour of https://dockstore.org/api/swagger.json
  • hotfix for dockstore tool/workflow convert for some WDL workflows
  • See https://github.com/ga4gh/dockstore/milestone/21?closed=1 for more details
  • handle WDL recursive imports (i.e. imports in imported secondary WDLs)
  • refresh improvements
  • DAG display for WDL improvements
  • improvements to metadata parsing
  • TRS improvments to improve support for GA4GH testbed and match documentation of standard
    • fixed URL formatting (was displaying paths incorrectly for tests)
    • conform to specification by displaying relative paths properly
    • fixed URL pass through to GitHub/Bitbucket/Gitlab in cases of descriptors in nested folders

Full details at https://github.com/ga4gh/dockstore/milestone/19?closed=1

Minor bugfixes for

  • regression for secondary files previously done against hotfix branches
  • refresh issue for tools

Highlighted new features include:

  • support for registering checker workflows, workflows that validate whether a workflow executed successfully to lead-up to the GA4GH workflow testbed
    • see a presentation and our tutorial in progress at https://github.com/ga4gh/dockstore/wiki/WIP-Checker-Workflow-Support-Tutorial
  • launch-with Firecloud coming soon!
  • support for the next iteration of the tool registry schema, see v2
  • usability improvements such as the ability to link to specific versions of workflows such as Sanger 2.0.3, search by workflow or tool language, and sorting by number of stars
  • user requests such as the ability to link to unencoded workflows
  • documentation has been moved out to docs.dockstore.org/ allowing us to update it more often
  • CLI
    • ability to configure Cromwell and Bunny (rabix) version or use the generic cwl-runner interface to run cwl
    • ability to POST notifications during workflow execution to a custom endpoint
    • ability to run unpublished tools and workflows (if you are the owner or in the organization for a tool or workflow)
  • WIP for displaying CWL visualizations using view.commonwl.org

See a full list of our changes on GitHub.

Breaking changes

Major

File provisioning fixes and a gently evolving understanding of CWL Details at https://github.com/ga4gh/dockstore/milestone/17?closed=1

  • fix behaviour of extra cwltool flags
  • use dct:creator and schema:author as alternatives to each other

A hotfix for #948

Highlighted new features include:

  • search interface for quickly locating tools by keyword, author, labels, organization, etc.
  • search niceties such as search term autocomplete, an advanced search, and a tag cloud highlighting significant terms
  • updates for our CLI development environment to use cwltool 1.0.20170828135420 and cromwell 29.
  • our deployment environment is now fully described by Docker Compose allowing developers to spin up a private version of Dockstore including 1.3.0, 1.2.10, and search
  • integration with https://discuss.dockstore.org/
  • user requests such as the ability to allow for more exotic Dockerfile names and a reorganized tutorial for users of tools and workflows (as opposed to tool and workflow developers)
  • and a bit of inside baseball, we migrated from Angular 1 to Angular 4 as a foundation for future improvements

See a full list of our changes on GitHub.

https://github.com/ga4gh/dockstore/milestone/13

No description.
  • A large number of fixes to file provisioning (with more to come)
  • CLI now provisions outfiles to working directory if not otherwise specified
  • File provisioning now supports directories (see https://github.com/ga4gh/dockstore/tree/1.2.5/dockstore-client/src/test/resources/file_provision for some examples)
  • making Dockstore script more robust
  • adding retry functionality to uploads
  • Some TBA launch-with goodness

  • database constraint fixes

More details at https://github.com/ga4gh/dockstore/milestone/12

Hotfixes

  • default path, gitlab, starred tools
  • redirecting logging output
  • correctly understand relative paths for workflows
  • globbed files provisioned to directory support

See also https://github.com/ga4gh/dockstore/milestone/11

  • hotfix for default branch selection
  • hotfix for default path customization

New Features

  • support for private Docker repositories (starting with Amazon ECR)
  • starring tools and workflows for bookmarking
    • also see who has starred a particular tool
  • streamlined CLI
  • passing through json-ld
  • metadata for write api https://github.com/dockstore/write_api_service

Bug Fixes

  • passive ftp downloading for improved compatibility
  • fixes for secondary file provisioning
  • better handling of maven shading and logging
  • many bugfixes, see github issues for details https://github.com/ga4gh/dockstore/milestone/10
  • move S3 to use high-level TransferManager to improve performance
  • fix to icgc provisoning
  • migrate to artifacts.oicr.on.ca
No description.

various small fixes including

  • passive ftp mode for compatibility with more ftp sites
  • improved messages
  • secondary files provisioning on output
  • Support for GitLab as a source code repository
  • Updating naive launch of WDL workflows to Cromwell 0.21 and WDL parsing code to WDL4S 0.6
  • Support for multiple test json/yaml files for a tool or workflow
  • Updates to common-vfs to support file provisioning from ICGC portal
  • Support for administrators to verify versions of tools and workflows
  • CWL generation of parameter files now supports YAML as well as JSON
  • updates to ga4gh endpoints

Production release

  • fix for content-type GA4GH endpoint
  • fixed tool tab issues
  • Dockstore launcher now understands CWL secondary files
  • tweaks to match GA4GH requests for tool-registry-schema
  • yaml/yml support as synonyms for CWL
  • fixes to dag display
  • odd refresh bugfix
  • update cwltool to use enable net
  • fix issue with quay returning only a subset of builds for a given repo
  • various fixes to issues found on staging
  • a couple usability issues

Manually registered images should now automatically populate tags and references

Bug fixes: #77

0.2.1

  • Manual container registration with support for DockerHub
  • Register a pre-existing container under a different tool name
  • Edit tags (reference, Dockerfile and Dockstore.cwl paths)
  • Individual container refresh

Support specifying specific tags in Quay.io, ie multiple file versions for different tags

  • Bitbucket integration
  • Community labelling

Many to many relationship for containers and users. This means multiple users can have the same container if they are from the same organization.

Others:

  • bug fixes for deletion of containers
  • refactoring of refresh
  • sorting for the CLI

Pulls the proper tags from Quay.io. Properly deletes containers on refresh.

CLI bug fixes

fixed CLI bug

Bug fixes for CLI

  • Major revisions to the endpoints
  • renaming collab.cwl to dockstore.cwl

Added prompt for token and server-url in CLI bootstrap.

  • adds authentication
  • re factored Maven modules
  • added CLI
  • getting onto standard Maven release procedure