Subvocal Speech Capture Keeps Things Quiet While Improving Recognition Accuracy
By sensing activity in the nerves in the throat that control speech, the NASA Ames Research Center has begun to computerize human silent reading.
Button-sized sensors stuck under the chin and on either side of the Adam's apple could gather nerve signals created by "subvocal speech" and send them to a processor that executes a program to translate them into words. Eventually, the technology could be used in spacesuits, in noisy places like airport control towers, or even in traditional voice-recognition programs to increase accuracy.
"The processor analyzes the silent, or subauditory, speech such as when a person silently reads or talks to himself," explained Chuck Jorgensen, a scientist whose team is developing the technology. "Biological signals arise when reading or speaking to oneself with or without actual lip or facial movement. A person using the subvocal system thinks of phrases and talks to himself so quietly, it cannot be heard, but the tongue and vocal cords do receive speech signals from the brain."
In their first experiment, scientists "trained" software to recognize six words and 10 digits that the researchers repeated subvocally. Initial word recognition results had an average of 92% accuracy. The first subvocal words the system learned were stop, go, left, right, alpha, omega, and the numbers zero through nine. Silently speaking these words, scientists conducted simple searches on the Internet by using a number chart representing the alphabet to control a Web browser program. This proved that users could browse the Web without touching a keyboard.
Scientists are now testing new, noncontact sensors that can read muscle signals even through a layer of clothing. A second demonstration will attempt to control a mechanical device using a simple set of subvocal commands, according to Jorgensen. His team is planning tests with a simulated Mars rover. Silent "spoken" words will control rover movement. This would enable people in noisy environments to use the system when privacy is needed, such as during telephone conversations on buses or trains.
An expanded muscle-control system could help injured astronauts control machines. After suffering from muscle weakness due to a long stint in microgravity, astronauts could send signals to software that would assist with landings or other maneuvers. A logical spinoff would be an aid to handicapped individuals who could use this system to help with many tasks.
Scientists at Ames are studying the complex nerve-signal patterns to learn more about the signals that control vocal cords, muscles, and tongue position. The researchers use an amplifier to strengthen the electrical nerve signals. The signals are processed to remove noise and then processed again to differentiate one word from another.
After the signals are amplified, software "reads" the signals to recognize each word and sound. The researchers will continue to expand the vocabulary with sets of English sounds to eventually enable a full speech-recognition program to interpret the words.
The Computing, Information, and Communications Technology Program, part of NASA's Office of Exploration Systems, funds the subvocal word-recognition research. There is a patent pending for the new technology.
For further information, go to http://amesnews.arc.nasa.gov.
