CNR ExploRA

Articolo in rivista, 2015, ENG

PARLOMA - A Novel Human-Robot Interaction System for Deaf-blind Remote Communication

L. Orlando Russo, G. Airò Farulla; Pianu, D.; Salgarella, A. R.; Controzzi, M.; Cipriani, C.; Oddo, C. M.; Geraci, C.; Rosa, S.; Indaco, M.

Politecnico di Torino, Politecnico di Torino, CNR IEIIT Torino, Politecnico di Torino, The BioRobotics Institute Scuola Superiore Sant'Anna, The BioRobotics Institute Scuola Superiore Sant'Anna, The BioRobotics Institute Scuola Superiore Sant'Anna, CNRS Paris, Politecnico di Torino, Politecnico di Torino

We present a novel system for remotely controlling an anthropomorphic robotic hand using gestures. Our system uses a low-cost depth sensor as the only input device. The basic idea behind the resented system is that a user is able to perform hand shapes in front of the depth sensor; the system is able to segment the user's hand from the background and to recognize a set of hand postures; recognized postures are then sent over the web and reproduced by the a remote robotic hand. The system is expected to enhance communication among Deaf-blind signers by allowing, for the first time, remote transmission of Sign Language (SL) in the tactile modality. Usually, Deaf-blind signers can only receive external information by tactile exploration of the environment. Communication is therefore possible by using a special variety of SL, the tactile-SL (t-SL). The proposed system will basically work as a "telephone" for Deaf-blind people. It will receive a SL input (which can be produced by all signers, including Deaf-blind) and it will provide a t-SL output (the only one accessible to Deaf-blind signers). The system will convert a SL message into its tactile variety. It does not manipulate the semantic nor the structural content of the message, and therefore it is language-independent (i.e. it can be used with any SL, t-SL pair). The system has been designed after consulting the main Italian Deaf-blind associations. A Deaf-blind representative also tested a preliminary version of the system. In this paper, we report the results of a first set of experiments, showing that the developed system is accurate enough to reproduce handshapes (a crucial component of the SL message). This system is a first prototype for the Parloma project, which aims at designing a remote communication system for Deaf-blind people, but could also be useful in other scenarios (e.g., tele-rehabilitation, tele-presence . . . ).

International journal of advanced robotic systems (Online)