talkTV
developed:
Erik Blankinship
information: organized
MIT Media Laboratory, USA
and
Alma Media, Finland
http://www.media.mit.edu/explain/projects/talkTV/
what:
talkTV lets you search digitized video for words in the closed captions (CC). For example, search for the words "how are you" and video clips with that phrase are returned. You can then send this small video clip greeting to others via email or chat programs. You can also string small video clips together to make small scenes. For example, you can have Archie Bunker ask a question and have Captain Kangaroo answer.
requirements:
instructions:
Launch talkTV by double clicking on the talkTV.jar (Mac OS) or talkTV.exe (Windows). If it is your first time running talkTV, you will be prompted to select the folder where you keep your digitized videos and closed captioning text files (we call this the video library folder). You might also be prompted to select a format for video clips to use in chat and email programs (note: you will need Quicktime Pro to set this). Sorenson Video 3, size 120x90, with Qualcomm PureVoice audio is a good choice for small files. You can change this setting later by selecting "options/drag settings" from the menubar.
When talkTV has launched, you can check which videos are available by selecting "/shows/video library" from the menubar. You can label videos by genre, series, and title. If there are no videos listed, you might need to select the correct directory for your video library ("/options/set video library directory" from the menubar).
In the main talkTV frame, type a phrase like "hello" into the search field (A) in the upper left and hit the search button. If this phrase is said in any of the talkTV .xml files in your video library folder, a small screen shot of that scene is displayed in a results list (B). Selecting one of the returned screenshots will play that clip for you (C).
Pressing the 
ADD button inserts the current video clip to the timeline along the bottom of the frame. The timeline is where you can sequence your own scenes or drag clips into other programs like AOL Instant Messenger.
The other buttons under the selected video clip enable you to control it in different ways:
|
|
ADD: pressing the ADD button inserts the current video clip to the timeline along the bottom of the frame.
|
|
PLAY: pressing the PLAY button plays the current video clip.
|
|
STOP: pressing the STOP button stops the current video clip.
|
|
LOOP: pressing the LOOP button will let the current video clip play repeatedly.
|
|
STEP BACK: pressing STEP BACK plays the line of dialog before the current video clip.
|
|
STEP FORWARD: pressing STEP FORWARD plays the line of dialog after the current video clip.
|
|
LOCK: pressing LOCK will prevent changes to this clip's in and out points.
|
|
|
|
|
TRIM: pressing TRIM buttons lets you add/remove a little bit of video from the start or end of the current video clip. TRIM buttons on the left trim the in point, TRIM buttons on the right side trim the out point.
|
You can resequence your clips by dragging and dropping them along the timeline at the bottom of the frame as shown.
By holding down the CTRL key while dragging a clip from the timeline allows you to drag it into other applications, like email applications and AOL Instant Messenger (using Direct Image Connection).
The buttons under the timeline let you control your sequenced clips in different ways:
|
CLEAR: pressing CLEAR deletes all clips in the timeline.
|
|
REMOVE: pressing REMOVE deletes the selected (hilighted) clip in the timeline.
|
|
PLAY MOVIE: pressing PLAY MOVIE plays the clips in the timeine in sequence.
|
|
EXPORT MOVIE: pressing EXPORT MOVIE lets you save sequenced clips in the timeline as one movie file or each clip as a movie file.
|
Here are some other things you can do with talkTV:
- If the timings are off between the closed captions and dialog (common in live broadcasts where someone is typing in real time), then set the correct in point and select /clip/shift time of entire episode/. This should help to put the captions and dialog in better sync. This will not effect clips you have locked with the LOCK button.
- Limit your search to certain television programs by toggling the checkboxes under "/shows/" in the menubar.
- If there is a typo or error in the closed captions, you can edit the captions by selecting "/clip/modify clip/" from the menubar.
- If you want to restore the original closed captions (if you changed them) and the in and out points of a clip, select "/clip/restore original clip/" from the menubar.
- When you change the in or out point of a clip, talkTV will automatically push the in or out point of the next clip in the show backward or forward. If you want to change this option, toggle "/clip/push other clips/".
-
When you make changes (modidying in and out points, or correcting the closed captions), you can record those changes by selecting "/file/save changes/".
videos:
talkTV uses digitized broadcast television. Distributing digitized broadcast television is not easy to do because of copyright infrigements. Here is a guide describing one way to capture and prepare digital video of broadcast television for private use with talkTV.
- Rent/buy a video or dvd with the CC logo on the box. You can also tape a program off of the television - check http://www.tvguide.com to see which programs are closed captioned.
- Capture your video using dv firewire. Make sure you capture using an NTSC default setting.
- Extract the closed captions from the digitized broadcast to a text file. CCaption DV by Leapfrog, Inc. is a good tool for this, see http://www.ccaption.com. Note: you will still need to convert this text file into a talkTV file as described in the next step.
- Launch talkTV and select "/options/convert cc/". Select the text file generated in step 3. This generates a talkTV document with the extension .xml in the same folder.
- Using quicktime, compress the DV movie file using Motion JPEG-A; this is a spatial compression codec allows talkTV to jump to random parts of the video file very quickly. Specifically, compress the movie to 30fps, Motion JPEG-A with one field, medium (50%) quality, 240x180 pixels, 16bit 44.1 WAV audio.
- Put your talkTV .xml file and the compressed quicktime movie into the same folder. Give both files the same name (for example, trek.mov and trek.xml).
Another way to build your video library is to extract digital video from DVDs. It is not straightforward how to convert DVD's into quicktime's Motion JPEG-A format. Nonetheless, here is the rough procedure for doing this on a Windows PC:
- Rent/buy a dvd with the English subtitles on the box -- this is different than CC.
- Extract the movie from the dvd using a program like FlaskMPEG.
- Use a program like SubRipper to extract a .srt text file from the extracted movie. Note: you will still need to convert this text file into a talkTV file as described in the next step.
- Launch talkTV and select "/options/convert cc/". Select the text file generated in step 3. This generates a talkTV document with the extension .xml in the same folder.
- Use FlaskMPEG to convert the ripped movie into a quicktime format (3IVX producd mixed results, cinepak is okay). Motion JPEG-A is highly preferrable for talkTV since it is a spatial compression codec allows talkTV to jump to random parts of the video file very quickly. Specifically, compress the movie to 30fps, Motion JPEG-A with one field, medium (50%) quality, 240x180 pixels, 16bit 44.1 WAV audio.
- Put your talkTV .xml file and the compressed quicktime movie into the same folder. Give both files the same name (for example, trek.mov and trek.xml).
