Researchers at the National University of Chilecito (UNdeC) have developed software capable of transcribing speech into text and translating it into sign language in real time. The aim is to provide hearing-impaired students with a tool that facilitates their academic career.
“The development of this pilot software favors the inclusion of students with sensory disabilitiessuch as deafness and hearing loss, during their academic training process”, says Gabriel Quiroga Salomão, project Manager. The idea, presented and approved by the University in 2020, had to be implemented from a distance due to the isolation imposed by the pandemic.
Baptized with the name “Enseñame”, the program already works in its first version, although it only has a reduced dictionary generated to be able to do tests. Among its features, five main functions stand out. The first is the literal one, which sequentially translates (one by one) the words heard and a 3D avatar is responsible for showing the translation.
The second is Argentine Sign Language (LSA) which has its own grammatical structure of tense and subject, object, verb, negation and question. When the teacher speaks, the system translates to text, interprets the structure, switches to LSA and returns the translation through the avatar.
“We developed an algorithm so that the system recognizes the voice, passes it to the text and simulates the representation made by the interpreter“, mentions Quiroga Salomão. And he adds that, if the pilot software performs these two functions well, it also has a very reduced dictionary of words and passwords: “To increase the dictionary we work with an interpreter and we develop the animations of our avatar, to later load them into the database of the system”.
He orthography is the third function, which translates letter by letter into signs those words that have no translation or are not in the dictionary. The fourth allows conjugate verbs and the last one is calledself learn”, which allows the translation not to dwell on words not found in the dictionary, spell them out and keep track of the sign that is added to the database.
present and future
“Enseñame” has been presented at several national and international scientific events. In 2022, he was a finalist for the “Lueny Morrel” award for educational innovation in higher education and winner of the “José María Antón 2022” award of Virtual Educa, an organization created in 2001 to strengthen education in Latin America and the Caribbean.
Currently, the research team continues to work on finding tools and applications necessary to improve the 3D avatar and to develop the system on multiple platforms. In this sense, points out the project director: “It is a software that currently runs in a browser, but we intend it to also run as an app on a mobile device”.
For this, they look for funding programs or companies that can sponsor and invest in the initiative. “We need to continue working on the dictionary of signs and words for the 3D avatar. With this application we want to reach educational institutions of all levels, organizations and institutions of the State, favoring inclusion, the right and access to training”, highlights Quiroga Salomón.