This research also foresaw that augmented reality brings the possibility of not
only enhancing our current senses, but of possibly “making up” for missing ones. In this
thesis, the author designed and implemented an augmented reality application for hearing
augmentation where hearing-impaired users can see visual cues of what is being said to
them in a natural and intuitive way to understand. The application, dubbed iHeAR, uses
the iOS platform and an iPad2 as the supporting device. It is implemented using current
algorithms for speech recognition and face detection in order to output the “heard”
speech in real time next to the speaker’s face in a “text-bubble”. The speech recognition
system used is the open source OpenEars which is a wrapper for iOS application of the
PocketSphinx system for device speech recognition. A detailed explanation of OpenEars
was provided in section 4.3. Face detection is achieved using OpenCV’s Viola-Jones
method implementation for face detection, whose explanation was provided in section
4.4. In order to make the face detection algorithm work in real time and perform
calculations for both speech recognition and face detection on the device, the author
optimized the system to run the face detection algorithm only when speech is detected
and only when a previous frame is not already being analyzed for a face since the
detection algorithm runs slower than the video feed. In this way, the final system is not
overloaded with heavy calculations. The system built assumes the following conditions
and limitations:
This research also foresaw that augmented reality brings the possibility of not only enhancing our current senses, but of possibly “making up” for missing ones. In this thesis, the author designed and implemented an augmented reality application for hearing augmentation where hearing-impaired users can see visual cues of what is being said to them in a natural and intuitive way to understand. The application, dubbed iHeAR, uses the iOS platform and an iPad2 as the supporting device. It is implemented using current algorithms for speech recognition and face detection in order to output the “heard” speech in real time next to the speaker’s face in a “text-bubble”. The speech recognition system used is the open source OpenEars which is a wrapper for iOS application of the PocketSphinx system for device speech recognition. A detailed explanation of OpenEars was provided in section 4.3. Face detection is achieved using OpenCV’s Viola-Jones method implementation for face detection, whose explanation was provided in section 4.4. In order to make the face detection algorithm work in real time and perform calculations for both speech recognition and face detection on the device, the author optimized the system to run the face detection algorithm only when speech is detected and only when a previous frame is not already being analyzed for a face since the detection algorithm runs slower than the video feed. In this way, the final system is not มากเกินไปกับการคำนวณที่หนัก ระบบที่สร้างขึ้นถือว่าเงื่อนไขต่อไปนี้ และข้อจำกัด:
การแปล กรุณารอสักครู่..