1994 Project Reports

RALPH: a fourth generation fingerspelling hand

David L Jaffe, MS

Abstract - A fourth generation computer-controlled electromechanical fingerspelling hand called Ralph (for Robotic Alphabet) has been developed at the Rehab R&D Center (Figure 1). The device offers deaf-blind individuals improved access to computers and communication devices in addition to person-to-person conversations. Enhancements in this design include better intelligibility, smaller size, and the ability to optimize hand positions.

Figure 1. Ralph, the fingerspelling hand.

Background - The majority of the estimated 20,000 adults in the US who are deaf and blind have Usher's Syndrome, a disease characterized by deafness at birth and a gradual loss of sight in young adulthood. Most children with Usher's are brought up in the deaf community, learning sign language and fingerspelling (and/or speech and lipreading) rather than braille. As their sight diminishes they resist learning braille, which is difficult to master as an adult and is an admission of a dual sensory loss. For this reason, augmentative communication devices employing braille may be inappropriate for people who are deaf and blind.

Problem - Many deaf-blind people are able to communicate with others by using a hand-on-hand variation of the American One-Hand Manual Alphabet. Interpreters for the deaf use this hand gesture system (fingerspelling) to spell out words for which there are no sign language equivalents. Instead of visually recognizing the gestures as deaf people do, deaf-blind individuals feel and interpret the motion and positions of the hand as the message is spelled out, one letter at a time by the interpreter.

However, fingerspelling using a human interpreter has several disadvantages:

• Both communication partners must know fingerspelling.
• The communication partners must be in physical proximity with each other.
• Professional interpreters are difficult to locate and schedule.
• The presence of interpreter may intrude on the privacy of a conversation.
• Extreme informational and social isolation can occur without communication.

Rationale - An electromechanical hand provides a deaf-blind person some independence in communication. Such a device typically translates keypresses or standard computer-produced serial ASCII representations of letters into movements of the fingers of a mechanical hand. These movements are felt by a deaf-blind user and interpreted as the fingerspelling equivalents of the letters comprising a message. They enable the user to receive fingerspelled messages from the mechanical hand in response to person-to-person communication as well as gain access to sources of computer-based information. With a fingerspelling hand, a deaf-blind individual need not rely on a human interpreter for all communication.

Project goal - The goal of this project is to design, develop, and pursue commercialization of a fingerspelling hand that is smaller, lighter, and more intelligible than our previous prototypes. Specifically, this design implemented an improved mechanical system.

Operation - A sighted person wishing to talk to a deaf-blind individual interacts with the fingerspelling hand through a hand-held computer via a serial connection. (Actually, any device that produces RS232 serial data, including terminals, modems, computers, OCR scanners, speech recognizers, or modified closed caption systems, could be used to control the hand.) The user interface is implemented as a menu system that provides easy access to the unit's various functions, such as displaying and setting the microcontroller's parameters, testing the hand motions, editing hand position data, and entering letters to be fingerspelled.

In the fingerspelling mode, keypresses are entered on the keyboard. The hand's software translates these keypresses into commands for the DC servo motors. As the motor shafts rotate, they push/pull on the rods that connect to the fingers' mechanical linkages. It is by this coordinated series of motor commands that keyboard input is transformed into choreographed motion representing fingerspelling.

Testing and evaluation - Evaluation of previous fingerspelling hands has been performed by Gallaudet University (Washington, DC) and AI duPont Institute (Wilmington, DE). Nine deaf-blind people tested the hands over a two month period. They identified confusing letter combinations and suggested improvements for a commercial prototype.

Technology transfer - A collaborative effort with prospective manufacturer has begun. This activity should result in the production of a commercial device that can be employed by deaf-blind people.

A potential solution exists for the provision of fingerspelling hands to deaf-blind people. Within California (and some other states), all telephone subscribers support a fund which provides telephone access equipment for persons with disabilities. Commercial versions of this fingerspelling hand could be furnished at no charge to deaf-blind people under this program.

All encounters with Ralph and previous fingerspelling hands have been enthusiastic, positive, and at times, highly emotional. The increased communication capability and ability to "talk" directly with people other than interpreters are powerful motivations for using fingerspelling hands. They have the potential to provide deaf-blind users with untiring personal communication at rates approaching that of a human interpreter.

Republished from the 1994 Rehabilitation R&D Center Progress Report.