Supporting data for"Hot-starting software containers for STAR aligner"

Dataset type: Software
Data released on June 19, 2018

Zhang P; Hung LH; Lloyd W; Yeung KY (2018): Supporting data for"Hot-starting software containers for STAR aligner" GigaScience Database. http://dx.doi.org/10.5524/100468

DOI10.5524/100468

Using software containers has become standard practice to reproducibly deploy and execute biomedical workflows on the cloud. However, some applications which contain time-consuming initialization steps will produce unnecessary costs for repeated executions.
We demonstrate that hot-starting, from containers that have been frozen after the application has already begun execution, can speed up bioinformatics workflows by avoiding repetitive initialization steps. We use an open source tool called Checkpoint and Restore in Userspace (CRIU) to save the state of the containers as a collection of checkpoint files on disk after it has read in the indices. The resulting checkpoint files are migrated to the host and CRIU is used to regenerate the containers in that ready-to-run hot-start state. As a proof-of-concept example, we create a hot-start container for the STAR aligner and deploy this container to align RNA sequencing data. We compare the performance of the alignment step with and without checkpoints on cloud platforms using local and network disks.
We demonstrate that hot-starting Docker containers from snapshots taken after repetitive initialization steps are completed, significantly speeds up the execution of the STAR aligner on all experimental platforms, including Amazon Web Services (AWS), Microsoft Azure and local virtual machines. Our method can be potentially employed in other bioinformatics applications in which a checkpoint can be inserted after a repetitive initialization phase.





File NameSample IDData TypeFile FormatSizeRelease Date 
GitHub archivearchive3.66 KB2018-06-05
ReadmeTEXT2.5 KB2018-06-05
GitHub archivearchive1.63 KB2018-06-05
Displaying 1-3 of 3 File(s).
Funding body Awardee Award ID Comments
National Institutes of Health Yeung-Rhee R01GM126019
National Institutes of Health A Ma'ayan, M Medvedovic, S Schurer U54HL127624
Date Action
June 19, 2018 Dataset publish
June 19, 2018 Funder updated : National Institutes of Health - Authors received funding from U54HL127624 as a subcontract.