Luminophonics experiment: A user study on visual sensory substitution device
- Published
- Accepted
- Subject Areas
- Human-Computer Interaction, Computer Vision, Emerging Technologies, Multimedia, Visual Analytics
- Keywords
- Image Processing, Computer Vision, Auditory Display, Image Sonification
- Copyright
- © 2015 Tan et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ PrePrints) and either DOI or URL of the article must be cited.
- Cite this article
- 2015. Luminophonics experiment: A user study on visual sensory substitution device. PeerJ PrePrints 3:e1289v2 https://doi.org/10.7287/peerj.preprints.1289v2
Abstract
Loss of vision is a severe impairment to the dominant sensory system. It often has a catastrophic effect upon the sufferer, with knock-on effects to their standard of living, their ability to support themselves, and their care-givers lives. Research into visual impairments is multi-faceted, focusing on the causes of these debilitating conditions as well as attempting to alleviate the daily lives of affected individuals. One of the methods is through the usage of sensory substitution device. Our proposed system, Luminophonics, focuses on visual to auditory cross modalities information conversions. A visual to audio sensory substitution device a type of system that obtains a continual stream visual inputs which it converts into corresponding auditory soundscape. Ultimately, this device allows the visually impaired to visualize the surrounding environment by only listening to the generated soundscape. Even though there is a huge potential for this kind of devices, public usage is still minimal (Loomis, 2010). In order to promote the adoption from the visually impaired, the overall performance of these devices need to be improved in terms of soundscape interpretability, information preservation and listening comfort amongst other factors. Luminophonics has developed 3 type of prototypes, which we have used to explore different ideas pertaining to visual to audio sensory substitution. In addition to these, one of the prototypes has been converted to include depth information using time of flight camera. Previously, an automated measurement method is used to evaluate the performance of the 3 prototypes (Tan, 2013). The results of the measurement cover the effectiveness in terms of interpretability and information preservation. The main purpose of the experiment reported herein, was to test the prototypes on human subjects in order to gain greater insight on how they perform in real-life situations.
Author Comment
Removed excel dataset
Supplemental Information
Figure 1 - Front View of Experiment Site
Figure 1 shows the front view of experiment site. A 90m long path was drawn with yellow floor-tape stuck on to the floor. Yellow colour was chosen because it is easier to be detected and recognized for the first-time user. Barricade floor-tape with alternating red and white stripe were taped on both left and right side of the yellow tape. This tape was intended as an indicator for users when they were not focusing at the correct angle. The yellow path guided users from the origin to the target point, where they were expected to perform a specific task.
Figure 2 - Rear View of Experiment Site
Figure 2 shows the experiment site captured from rear.
Figure 3 - Course Path Design
Course Path Design shows overall design of the navigation course. The red wavy line indicates the path the user needed to walk along. Obstacles were placed next to, and all along, this path. The green flag indicates the end of the path. When the user reaches the green flag, they were required to perform a task involving object recognition.
