A full report on our research demonstrating the efficacy of ReadMyQuips.
As part of an extensive, three-year development process for ReadMyQuips, we performed two major experiments under the auspices of the NIH to evaluate its efficacy as a tool for improving auditory-visual speech reception in indviduals with mild to moderate hearing impairment. These experiments, which involved over twenty new and experienced hearing aid users, demonstrate conclusivley that ReadMyQuips can make a difference. Our findings show average improvements in speech comprehension of around 30%, with some subjects experiencing well over 50% improvement.[1] The following article is a full report of the first experiment. Stay tuned for a write-up of the second!
Initial Findings
A pilot experiment was performed on a group of 10 adult hearing aid users with sensorineural hearing losses ranging from 14 to 57 dB (pure tone average at 500, 1000, 2000Hz). The subjects had no additional handicapping conditions or evidence of cognitive problems. Their ages ranged from 50 to 80 years of age. All of the subjects had English as a first language. The subjects were recruited from audiological clinics and senior citizen centers in the San Francisco area. All but one of the subjects were experienced hearing aid users (more than 1 year of hearing-aid use). Subject #10 had just been provided with acoustic amplification for the first time.
Figure 1: Decrease in Speech-to-Noise Ratio for 50% Intelligibility. The subjects are shown in order of decreased speech-to-noise ratio (i.e., improved performance). Subjects 1 to 9 are experienced hearing-aid users. The vertical bars show the average test-retest standard error for the observed increase in speech-to-noise ratio.
Each subject was given an IBM T-30 laptop computer to use over a period of three weeks. Each computer was loaded with 15 puzzles. Two of the puzzles were used to explain the procedure and to allow the subject to get used to using the system. The subject then used the remaining puzzles over a 3-week period. The subjects were asked to use the system whenever it was convenient. It was recommended that the subjects use the training system for about one half-hour each day, but they could use it for longer or shorter periods if they wished. At the start of the experiment a video recording of the IEEE sentence test (Hawley, et al. 1999) was administered in noise using an up-down adaptive paradigm to estimate the speech-to-noise ratio at 50% correct sentence identification. The test was administered twice to obtain an estimate of test-retest variability. The IEEE sentence test was administered again at the end of the 3-week training period followed by a structured interview. The video recordings of the IEEE sentences were made by the same speaker who recorded the test items for the cross-sentence puzzle.
The results of the experiment are shown in Figures 1 to 3. Figure 1 shows the decrease in speech-to-noise ratio at which the IEEE sentences were recognized 50% of the time. (Note: ability to understand speech at a poorer speech-to-noise ratio represents improved performance). The subjects are shown in order of the decrease in speech-to-noise ratio that they were able to handle at the end of the training program. The vertical bars show average test-retest standard error for the observed change in speech-to-noise ratio. Subject 10 was the first-time user of acoustic amplification. She showed an improvement of 15.5 dB which was significantly larger than that for any of the experienced hearing aid users.
A statistical analysis was performed on the data of the experienced hearing aid users (Subjects 1 to 9). Subject 10 was omitted from the statistical analysis since the large improvement shown by this subject (15.5 dB) resulted from the joint effect of auditory training and adaptation to acoustic amplification. A repeated measures analysis of variance for the experienced hearing-aid users showed a statistically significant decrease in the speech-to-noise ratio corresponding to 50% intelligibility (F = 8.8 (df = 1,8), p = 0.017). The average improvement was 2.8 dB. A statistical analysis of each subject’s performance showed that three subjects (Subjects 1 to 3) did not show a statistically significant change in their speech-to-noise ratio for 50% intelligibility at the end of the training program. The remaining subjects showed a significant decrease in the speech-to-noise ratio, p ranging from
An important feature of the training program is that it is intended to be entertaining so as to motivate students to use the system over long periods of time. The time spent on the system by each subject for each training session was monitored by the computer. Figure 3 provides a summary of these data for all of the subjects. Each bar in the diagram corresponds to an interval of time during which the system was used in a single training session. The height of each bar shows the frequency with which the system used for this time interval. For example, the first bar corresponds to a time interval of less than 20 minutes. The height of this bar shows that the system was used 28 times for this time interval in a single training session. The second bar corresponds to a time interval of 21 to 40 minutes. The height of this bar shows that the system was used 37 times for periods of 21 to 40 minutes in a training session.
Figure 2: Duration and Frequency of Use. Each bar corresponds to a time interval during which the system was used in a training session. The height of each bar corresponds to the frequency with which the system was used for that time interval. The data are for all 10 subjects. Note that on a few occasions the system was used continuously within a training session for as much as two hours or more.
It is revealing to note that the system was used frequently for periods of time in excess of 40 minutes. The average time spent on the system in a training session was 45 minutes, but there were large differences among subjects. On several occasions, the more highly motivated subjects used the system continuously for periods of up to two hours or more, as shown by the bars to the right of the figure. Not surprisingly, the subjects showing the largest improvements in performance were the ones who spent more time using the system.
Figure 3 shows the relationship between time-on-task (the total time spent using the system over the 3-week training period) and the change in speech-to-noise ratio corresponding to 50% intelligibility. The data shown are for the 9 experienced hearing-aid users. The correlation between Time on Task and Increase in Speech-to-Noise Ratio was found to be 0.61 (p = 0.6, 8 df).
Figure 3: Correlation of Speech-to-Noise Ratio with Time on Task. Time on Task is equal to the total time spent by each subject on the training system during the training program. Correlation coefficient = 0.61 (p = 0.6, 8 df)
The system also monitored the student’s progress during training. Whenever a puzzle was completed, feedback was provided to the student in the form of an index between 0 and 100 which served as a measure of the student’s performance. The performance index was based on a weighted average of the speech-to-noise ratio required for solving the puzzle and the average number of attempts at solving the test items. Figure 5 shows the test scores obtained during the training program for a subject showing a significant change in the speech-to-noise ratio for 50% intelligibility. Positive feedback of this type was found to be very helpful in maintaining motivation.
The final stage of the evaluation consisted of a structured interview with each subject. With one exception, all of the subjects responded positively regarding the value of the system, whether they enjoyed using the system and whether they felt that their ability to understand speech in noise improved as a result of the training program. The most revealing information was obtained from their responses to the open ended question: What did you like most about the system?
- “The challenge of the puzzles.”
- “It made me stop and think about what I was doing and how I was listening.”
- “The quotations.”
- “I found out that I could do some lipreading.”
- “The masking and the noise is accurate/realistic and mimics what I experience. The system has credibility.”
- “It was fun to try and figure it out.”
- “Fascinating and I think it helped.”
- “After I began using it I became aware how much lipreading can help to understand speech. This started a continuing self-training using daily situations.”
- “It is a useful system. I hope it can continue in use.”
- “Lovely, easy-on-the-eye graphics”
- “Clever concept with fun sayings. I enjoyed doing them.”
- “Test adjusted to my level of hearing.”
The responses indicated that all but one of the subjects enjoyed using the system and that they felt that their speech reception ability in noise had improved and that they would continue using the system if it was available. The one subject who did not enjoy using the training system did not like the style of humor that was used. This problem can be rectified by including different types of humor for different tastes. Several very useful suggestions were also made for improving the system, such as speeding up the rate of adjustment for matching the difficulty of the training program to each student’s level of performance; each test item in a puzzle should be a challenge to the student, neither too difficult or too easy.
Conclusions
The feasibility study showed significant improvements in speech recognition in noise using the computer-based speech-reception training system.
The average improvement for the 9 experienced hearing-aid users was 2.8 dB. A much greater improvement (15.5 dB) was obtained with a first-time user of acoustic amplification. This result, however, should be considered as no more than a promising case study and that more detailed experiments are needed with new hearing-aid users to determine how much of an improvement can be attributed to the training program and how much is a result of acclimatization to acoustic amplification. It should be noted that the training period was of short duration (3 weeks) and that there was no evidence of a slowing down in the improvement over time for the subjects showing significant improvements in performance. It should also be noted that the feasibility study focused on improving speech reception with both visual and auditory cues. The system can also be used with less sophisticated instrumentation for improving listening skills for audition only.
A key feature of the training program that distinguishes it from traditional auditory training programs is that it is designed to be entertaining. A training system that is fun to use will not only maintain motivation, it will also be used more intensively and for longer periods of time. The underlying assumption is that increased use of the training system will result in greater improvements in speech reception ability. The results of the feasibility study support both of these points. All but one of the subjects enjoyed using the system and most of the subjects used the system for longer periods of time per training session than was initially recommended (45 minutes, on average, as opposed to the recommended period of one half-hour per training session). Several of the subjects used the system continuously for periods exceeding an hour at a time and, on average, the subjects who used the system for longer periods of time showed larger improvements in performance. The one subject who did not enjoy using the system did not like the style of humor that was used. This problem can be addressed by developing training materials with different styles of humor to suit different tastes. The entertaining aspect of the training system is expected to be a major plus in marketing the system to prospective users.
An additional strength of the training system is its adaptive format in which the difficulty of the puzzle items is adjusted automatically to match the student’s level of performance. Some of the subjects complained that the rate of adjustment was too slow resulting in some puzzle items being either much too difficult or much too easy. This problem will be addressed in Phase II in which more efficient adaptive methods will be used to speed up the rate of convergence to each student’s level of performance. The use of computer-interactive techniques also allowed for detailed, unobtrusive record keeping as well as on-line computation of relative performance for providing helpful feedback to the student.
