EnglishPortugueseSpanish

Using Computer Vision and Machine Learning Systems to Develop Software to Transform Sign Language into Words

3X place

Project summary

The Brazilian Sign Language (Libras) is the main means of communication for the deaf, dumb and hearing impaired, but it was mostly restricted to a few people, encompassing more than 5 million users in a country with 10 million deaf people, corresponding respectively 3 % and 5% of the Brazilian population. Observing this issue, it is possible to expand communication and accessibility to the deaf community in society, enabling the creation of a tool capable of translating Libras signs into text. Tools that automate the translation process were researched, such as Artificial Intelligences (or known as AI, which can create computational Neural Networks, applying insight and autonomy to machines), Machine Learning (or Machine Learning, providing greater precision when training intelligences artificial) and MediaPipe (an open source tool capable of generating a graphical matrix based on 543 Cartesian points demarcated on the face, hands and body to recorded or real-time video). To apply them, a training of 200 videos per signal was started, using MediaPipe to identify movement variations in each sign of Libras, training around 100 signals and totaling 20 thousand trained videos. When performing this procedure, the AI ​​may be able, with the sign of Libras present in the video, to associate it with its corresponding word and output the result obtained to the user in a text file. Currently, the resulting word is generated separately from the video, but it was successful in applying subtitles according to the signal in some videos, but in both cases, the process is done only in videos that have already been recorded. Many people do not have knowledge of Libras, lacking accessibility for the deaf in, for example, hospitals, subways and companies. With the software, it will be possible to facilitate communication in such places, helping the communication of this previously excluded community.

Students

Fabricio Holanda de Almeida
Vinicius Luciano Navarrete da Silva
Luciano Dos Anjos Oliveira

Guidance counsellors

Cleiton Fabiano Patricio
Rosa Mitiko Shimizu

Institution

Etec Lauro Gomes
  SP -
  Brazil

Popular vote*

Did you like it? So vote and share now:

14+

Leave your comment

What did you think of this project? Participate by leaving your comment below:

Subscribe now
Notify of
guest
0 Comments
Inline feedback
View all comments

Popular vote*

Did you like it? So vote and share now:

14+

Students

Fabricio Holanda de Almeida
Vinicius Luciano Navarrete da Silva
Luciano Dos Anjos Oliveira

Guidance counsellors

Cleiton Fabiano Patricio
Rosa Mitiko Shimizu

Institution

Etec Lauro Gomes
  SP –
  Brazil

Educational Incentives and Awards Offered

Research Areas

Get to know the projects that are competing in all of Mostratec Virtual's research areas:

Cellular and Molecular Biology and Microbiology
Biochemistry and Chemistry
Environmental Sciences
Animal and Plant Sciences
Computer Science
Planetary and Earth Sciences and Mathematics and Physics
Health Sciences
Education and Humanities
Environmental and Sanitary Engineering
Engineering and Materials
Electrical engineering
Electronic Engineering
Mechanical Engineering
History and Social Sciences
0
Click to leave your comment and participate!x
()
x

Popular Jury Awards

For the People's Jury awards, only single votes (one per user) and that are cast between 0/00 and 26:10 on 23/59 will be considered valid.

Entrepreneur Profile Test

Fill in your details below to perform your test: