Abstract

Sign language is a fundamental part of the deaf culture, and it has its own linguistic and grammatical rules that allow perfect communication between its speakers (signers). In computer science, some automatic mechanisms have been studied to favor this communication. One of them is the Sign Language Recognition (SLR) which covers the whole process of tracking and identification of the signs made, and their conversion into words and expressions. In this context, this work focuses on developing a dataset for Isolated Chilean Sign Language Recognition (LSCh) by manually segmenting videos of deaf students, pre-processing them with a pose estimation tool and classifying them with Transformer-based deep learning models. Furthermore, it explores the impact of data augmentation on model performance and compares it with results from the Argentine sign language corpus (LSA64) to assess recognition ability with new signers. The results show the importance of data augmentation on smaller datasets and demonstrate that a single model can learn from signs made by both left-handed and right-handed signers. Metrics from the experiments show an accuracy of 0.75 and 0.94 and recall of 0.74 and 0.94 for LSCh and LSA64 respectively, using the best performing models (Mirror + Angle).