Innovative management of accelerometry, inertial, acoustic, and satellite data using netCDF and Postgres

In 2015, as part of the Ocean Tracking Network’s bioprobe initiative, 20 grey seals (Halichoerus grypus) were tagged with a high-resolution (> 30 Hz) inertial tags (> 30 Hz), a depth-temperature satellite tag (0.1 Hz), and an acoustic transceiver on Sable Island for 6 months. Comparable to similar large-scale studies in movement ecology, the unprecedented size of the data (gigabytes for a single seal) collected by these instruments raises new challenges in efficient database management. Here we propose the utility of Postgres and netCDF for storing the biotelemetry data and associated metadata. While it was possible to write the lower-resolution (acoustic and satellite) data to a Postgres database, netCDF was chosen as the format for the high-resolution movement (acceleration and inertial) records. Even without access to cluster computing, data could be efficiently (CPU time) recorded, as 920 million records were written in < 3 hours. ERDDAP was used to access and link the different datastreams with a user-friendly Application Programming Interface. This approach compresses the data to a fifth of its original size, and storing the data in a tree-like structure enables easy access and visualization for the end user.