Serverless OpenHealth at data commons scale - traversing the 20 million patient records of New York's SPARCS dataset in real-time

, , ,
1 Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, New York, United States
2 Radiology, State University of New York at Stony Brook, Stony Brook, New York, United States
DOI
10.7287/peerj.preprints.27209v1
Published
Accepted
Subject Areas
Bioinformatics, Epidemiology, Public Health, Computational Science, Data Science
Keywords
Serverless Computing, OpenHealth, SPARCS, Public Health, Epidemiology Data Commons
Copyright
© 2018 Almeida et al.
Licence
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
Cite this article
Almeida JS, Hajagos J, Saltz J, Saltz M. 2018. Serverless OpenHealth at data commons scale - traversing the 20 million patient records of New York's SPARCS dataset in real-time. PeerJ Preprints 6:e27209v1

Abstract

In a previous report, we explored the serverless OpenHealth approach to the Web as a Global Compute space. That approach relies on the modern browser full stack, and, in particular, its configuration for application assembly by code injection. The opportunity, and need, to expand this approach has since increased markedly, reflecting a wider adoption of Open Data policies by Public Health Agencies. Here, we describe how the serverless scaling challenge can be achieved by the isomorphic mapping between the remote data layer API and a local (client-side, in-browser) operator. This solution is validated with an accompanying interactive web application (bit.ly/loadsparcs) capable of real-time traversal of New York’s 20 million patient records of the Statewide Planning and Research Cooperative System (SPARCS), and is compared with alternative approaches. The results obtained strengthen the argument that the FAIR reproducibility needed for Population Science applications in the age of P4 Medicine is particularly well served by the Web platform.

Author Comment

This is a submission to PeerJ for review.