An evaluation of real-time requirements for automatic sign language recognition using ANNs and HMMs - The LIBRAS use case


  • Ednaldo Brigante Pizzolato Universidade Federal de São Carlos
  • Mauro dos Santos Anjo Universidade Federal de São Carlos
  • Sebastian Feuerstack Universidade Federal de São Carlos


gesture recognition, computer vision, sign language


Sign languages are the natural way Deafs use to communicate with other people. They have their own formal semantic definitions and syntactic rules and are composed by a large set of gestures involving hands and head. Automatic recognition of sign languages (ARSL) tries to recognize the signs and translate them into a written language. ARSL is a challenging task as it involves background segmentation, hands and head posture modeling, recognition and tracking, temporal analysis and syntactic and semantic interpretation. Moreover, when real-time requirements are considered, this task becomes even more challenging. In this paper, we present a study of real time requirements of automatic sign language recognition of small sets of static and dynamic gestures of the Brazilian Sign Language (LIBRAS). For the task of static gesture recognition, we implemented a system that is able to work on small sub-sets of the alphabet - like A,E,I,O,U and B,C,F,L,V - reaching very high recognition rates. For the task of dynamic gesture recognition, we tested our system over a small set of LIBRAS words and collected the execution times. The aim was to gather knowledge regarding execution time of all the recognition processes (like segmentation, analysis and recognition itself) to evaluate the feasibility of building a real-time system to recognize small sets of both static and dynamic gestures. Our findings indicate that the bottleneck of our current architecture is the recognition phase.


Download data is not yet available.

Author Biographies

Ednaldo Brigante Pizzolato, Universidade Federal de São Carlos

Departamento de Computação

Mauro dos Santos Anjo, Universidade Federal de São Carlos

Departamento de Computação

Sebastian Feuerstack, Universidade Federal de São Carlos

Departamento de Computação