Approaches for containerized scientific workflows in cloud environments with applications in life science

Ola Spjuth; Marco Capuccini; Matteo Carone; Anders Larsson; Wesley Schaal; Jon Novella; Paolo Di Tommaso; Cedric Notredame; Pablo Moreno; Payam E Khoonsari; Stephanie Herman; Kim Kultima; Samuel Lampa

doi:10.7287/peerj.preprints.27141v1

Approaches for containerized scientific workflows in cloud environments with applications in life science

Ola Spjuth ¹, Marco Capuccini^1,2, Matteo Carone¹, Anders Larsson³, Wesley Schaal¹, Jon Novella¹, Paolo Di Tommaso⁴, Cedric Notredame⁴, Pablo Moreno⁵, Payam E Khoonsari⁶, Stephanie Herman⁶, Kim Kultima⁶, Samuel Lampa¹

1 Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden

2 Department of Information Technology, Uppsala Universitet, Uppsala, Sweden

3 Department of Cell and Molecular Biology, Uppsala University, Uppsala, Sweden

4 Centre for Genomic Regulation (CRG), The Barcelona Institute for Science and Technology, Barcelona, Spain

5 European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, United Kingdom

6 Department of Medical Sciences, Uppsala Universitet, Uppsala, Sweden

DOI: 10.7287/peerj.preprints.27141v1

Published: 2018-08-24
Accepted: 2018-08-24

Subject Areas: Bioinformatics, Computational Biology
Keywords: containers, scientific workflow, cloud computing

Licence: This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.

Cite this article: Spjuth O, Capuccini M, Carone M, Larsson A, Schaal W, Novella J, Di Tommaso P, Notredame C, Moreno P, Khoonsari PE, Herman S, Kultima K, Lampa S. 2018. Approaches for containerized scientific workflows in cloud environments with applications in life science. PeerJ Preprints 6:e27141v1 https://doi.org/10.7287/peerj.preprints.27141v1

Abstract

Containers are gaining popularity in life science research as they encompass all dependencies of provisioned tools and simplifies software installations for end users, as well as offering a form of isolation between processes. Scientific workflows are ideal to chain containers into data analysis pipelines to sustain reproducible science. In this manuscript we review the different approaches to use containers inside the workflow tools Nextflow, Galaxy, Pachyderm, Luigi, and SciPipe when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.

Author Comment

This is a preprint submission to PeerJ Preprints.