Signage | Diploma Project

•

Introduction

Accessibility for the Disabled has indeed become an important area in inclusive design. My project focuses on a design solution for the creation of an Open Interactive Platform to enable Sign Language Teaching and Learning.

This project was done as my diploma project at the Srishti School of Art, Design and Technology.

Need

Deaf people have means and ways of learning sign language, as most deaf people go to specialized schools.

In the case of people with hearing capabilities, our language learning doesn’t stop at school. We have families who speak the same language as us, and a peer group who does as well.

Many deaf children lack this out-of-school support system, as in most cases, deaf children have hearing- capable families and hearing-capable peer groups who don’t always know how to use sign language.

The following needs stood out through my research:

1) Families and Social circles need to learn sign language, to provide a complete learning environment for the deaf person.
2) The need to provide an interactive platform to learn the sign language at one’s own pace. 3) The need to bridge the communication gap between the hearing disabled and others.

Technology

Following are the technology requirements of the platform:
1) A Gesture Recognition Device such as the LEAP Motion or Microsoft’s Kinect

2) A Back-End comprising of An Artificial Neural Network to recognize gestures and provide feedback

3) A Front-End Interface for users to interact with the platform
4) A cloud based service to allow users to access the platform from anywhere.

American Sign Language (ASL)

For this project I will be using ASL as an example for the language being taught on my platform.

ASL is the most widely used sign language in the English-speaking world as it is used in more than 50 countries.

It is the lingua franca in the deaf communities of several nations and is therefore widely learned as a second language within the hearing community.

ASL is composed of a system of conventional gestures, mimic, hand signs and finger spelling, as well as hand positions to represent the letters of the alphabet. Signs in ASL usually represent complete ideas, not only individual words.

Research - Existing Applications - Drawbacks

Most Applications use video, however, very few have video speed control

Most Web based applications have a search-able database, however, none allow users to contribute to the database, reducing the scope of learning.

Most applications do not insist on the need of a facilitator to be present while using the app, however, none of them give feedback or correct the user if he/she is signing wrong.

Very few applications allow users to test themselves on the knowledge acquired using the application. No application takes into account, the 3-D component of sign language, hence, ignoring the

accuracy of the sign being learnt.

Most applications are free, however some are limited in their usage unless the user pays a monthly fee. This provides a barrier for truly inclusive accessibility.

Proposed Platform - Features

Open Access: To allow complete access to the product at no cost.

Open Source Design: Physical products, machines and systems through use of publicly shared design information.

Gesture Recognition Device: To provide feedback to users while learning the language. A gesture recognition device allows for users to test themselves and see where they go wrong.

No Need of Facilitation: The Gesture recognition device eliminates the need of a teacher, allowing users to learn remotely without being bound to a classroom.

Data Collection

Crowd Sourcing:

Users will contribute to an existing library of videos and gestures.
Users will have the ability to record only video or video and gesture.
The uploads will require moderation to filter out incorrect/ unclear or rude signs.

Augmented System Design