Milky way analysis through a Science Gateway: workflows and resource monitoring
- Published
- Accepted
- Subject Areas
- Data Mining and Machine Learning, Distributed and Parallel Computing, Scientific Computing and Simulation
- Keywords
- Workflow Systems, Science Gateways, Collaborative Environments, DCIs, Milky Way Analysis, Astrophysics, Infrastructure Tests, Monitoring
- Copyright
- © 2017 Sciacca et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
- Cite this article
- 2017. Milky way analysis through a Science Gateway: workflows and resource monitoring. PeerJ Preprints 5:e2818v2 https://doi.org/10.7287/peerj.preprints.2818v2
Abstract
This paper presents the latest developments on the VIALACTEA Science Gateway in the context of the FP7 VIALACTEA project. This science gateway operates as a central workbench for the VIALACTEA community in order to allow astronomers to process the new-generation (from Infrared to Radio) surveys of the Galactic Plane to build and deliver a quantitative 3D model of our Milky Way Galaxy. The final model will be used as a template for external galaxies to study star formation across the cosmic time. The adopted AGILE software development process allowed to fulfill the community needs in terms of required workflows and underlying resources monitoring. The scientific requirements arose during the process highlighted the needs for easy parameter setting, fully embarrassingly parallel computations and large-scale input dataset processing. Therefore the science gateway based on the WS-PGRADE/gUSE framework has been able to fulfill the requirements mainly exploiting the parameter sweep paradigm and parallel jobs execution of the workflow management system. Moving from the development to the production environment an efficient resource monitoring system has been implemented to easily analyse and debug sources of failure due to workflows computations. The results of the resource monitoring system are exploitable not only for IT experts administrators and workflow developers but also for the final users of the gateway. The affiliation to the STARnet Gateway Federation ensures the sustainability of the presented products after the end of the project, allowing the usage of VIALACTEA Science Gateway to all the stakeholders and not only to the community members.
Author Comment
This is the abstract of a full paper appearing in IWSG2016 proceedings. In this version author list has been updated.