Creating Realistic Animated Speech

Our TalkingHead system is a text-to-audiovisual-speech system. That is, given some text, TalkingHead will produce computer-generated speech and animate a 3D model to create computer-generated animation that is lip-synchronized to the speech. Producing lip-synched animation of speech is difficult because we produce the same sounds in different ways, that may sound similar, but look quite different. We are currently researching how prosody (stress, volume, speech rate, etc.) affects speech using motion capture. Students have also worked on modeling real-time hair, controlling prosody of computer generated speech, and creating a 3D model of eyes. The project team ranges from Freshmen to Graduate Students. For more information on this and related projects, visit our Pixel Island web site.