Step 3 Using the touch panel Operation buttons on each display
When the POWER switch is set to CAMERA PAGE1 PAGE2 SPOT FOCUS, MENU, FOCUS, FADER, END SCH, SPOT METER, EXPOSURE DIG EFFT*, SELFTIMER, LCD BRT, SUPER NS*, VOL, COLOR SLW S*, MEM MIX*
* DCR-TRV22/TRV33 only When the POWER switch is set to VCR PAGE1 PAGE2 PAGE3 MENU, LCD BRT, VOL, END SCH DATA CODE, DIG EFFT*, PB ZOOM* V SPD PLAY, A DUB CTRL, REC CTRL
* DCR-TRV22/TRV33 only When the POWER switch is set to MEMORY (DCR-TRV22/TRV33 only) During memory camera PAGE1 PAGE2 PAGE3 SPOT FOCUS, MENU, FOCUS, PLAY, PB FOLDR, SPOT METER, EXPOSURE SELFTIMER, PLAY, MEM MIX LCD BRT, PLAY, VOL
During memory playback PAGE1 PAGE2 PAGE3 MENU, CAM, PB FOLDR, DEL PB ZOOM, CAM, RESIZE*, DATA CODE LCD BRT, CAM, VOL
* DCR-TRV33 only Notes When using the touch panel, press operation buttons lightly with your finger supporting the LCD panel from the back side of it. Do not press the LCD screen with sharp objects such as a pen. Do not press the LCD screen too hard. Do not touch the LCD screen with wet hands. If FN is not on the LCD screen, touch the LCD screen lightly to make it appear. You can control the display with DSPL/BATT INFO on your camcorder. When executing each item The green bar appears above the item. If the items are not available The color of the items changes to gray. When using the viewfinder You can also operate with the touch panel (p. 70).

Recording Basics

Recording a picture
Perform Step 1 (p. 16) and Step 2 (p. 22) in advance.
(1) Open the lens cap. (2) Prepare the power source (p. 16) and insert the cassette (p. 8). (3) Set the POWER switch to CAMERA while pressing the small green button. This sets your camcorder to standby. (4) Press OPEN to open the LCD panel. The picture appears on the screen. (5) Press START/STOP. Your camcorder starts recording. The REC indicator appears. The camera recording lamp located on the front of your camcorder lights up. To stop recording, press START/STOP again.

3 LOCK switch

Camera recording lamp

REC 0:00:01

Microphone
Notes Position the lens cap so as not to cover the microphone and the infrared emitter. Avoid touching the microphone when you are recording. Note on recording Your camcorder records and plays back in the SP (standard play) mode and in the LP (long play) mode. Select SP or LP in in the menu settings (p. 182). In the LP mode, you can record 1.5 times as long as in the SP mode. When you record the tape in the LP mode on your camcorder, we recommend playing back the tape on your camcorder. To enable smooth transition Transition between the last scene you recorded and the next scene is smooth as long as you do not eject the cassette even if you turn off your camcorder. However, check the following: Do not mix recordings in the SP and LP modes on one tape. When you change the battery pack, set the POWER switch to OFF (CHG). If you leave your camcorder during standby for 5 minutes while the cassette is inserted Your camcorder automatically turns off. This is to save battery power and to prevent battery pack and tape wear. To return to standby, set the POWER switch to OFF (CHG) and then back to CAMERA. However, your camcorder does not turn off automatically while the cassette is not inserted. When you record in the SP and LP modes on one tape or you record in the LP mode The transition between scenes may not be smooth. The playback picture may be distorted or the time code may not be written properly between scenes. Note on the LOCK switch (DCR-TRV22/TRV33 only) When you slide the LOCK switch to the right, the POWER switch can no longer be set to MEMORY accidentally. The LOCK switch is set to the left as the default setting. Self-timer recording You can record images on the tape with the self-timer. See page 50 for details.

TAPE SET I NT. REC I NTERVAL REC T I ME
30SEC TAPE SET I NT. REC I NTERVAL REC T I ME
30SEC 1M I N 5M I N 10M I N TAPE SET I NT. REC I NTERVAL REC T I ME
0. 5SEC 1SEC 1. 5SEC 2SEC
To cancel Interval Recording
Set INT. REC to OFF in the menu settings.
Frame by frame recording Frame recording
You can make a recording with a stop-motion animated effect using frame recording. To create this effect, alternately move the subject a little and make frame recording. We recommend that you use a tripod, and operate your camcorder using the Remote Commander to avoid camera-shake. The POWER switch should be set to Operate by touching the panel.
(1) Press FN to display PAGE1. (2) Press MENU to display the menu. , then press EXEC (p. 182). (3) Select FRAME REC in
TAPE SET REC MODE AUDIO MODE REMAIN FRAME REC INT. REC
(4) Select ON, then press EXEC. (5) Press EXIT to return to FN. The FRAME REC indicator lights up. (6) Press START/STOP to start frame recording. Your camcorder makes a recording for about 6 frames, and returns to recording standby. (7) Move the subject, and repeat step 6.
To cancel frame recording
Set FRAME REC to OFF in the menu settings. Note The proper remaining tape time is not indicated if you use frame recording continuously. When using frame recording The last recorded frame is longer than other frames.

Using the viewfinder

You can operate camera brightness and fader on the touch panel when looking at the viewfinder. Turn the LCD panel over and move it back to your camcorder body with the LCD screen facing out to do this operation. The POWER switch should be set to TRV33 only). Operate by touching the panel. or (DCR-TRV22/
(1) Turn the LCD panel over and move it back to your camcorder body with the LCD screen facing out, then extend the viewfinder. OFF. The PANEL OFF indicator appears on the screen. (2) Press (3) Press OK. The LCD screen is turned off. OK, ON and (4) Using the viewfinder, press the LCD screen. EXPOSURE, FADER (only when the POWER switch is set to CAMERA) appear. OK. (5) Press the LCD screen and select a desired item, then press EXPOSURE: Adjust the exposure by pressing /+. FADER: Press FADER until a desired fader mode appears. The indicator changes as follows: FADER t FADER t BOUNCE t MONOTONE M. (no indicator) T DOT* T WIPE* T OVERLAP*

* DCR-TRV22/TRV33 only

The LCD screen lights up.
4 When the POWER switch is set to CAMERA

EXPO SURE OK

Viewfinder
When the POWER switch is set to MEMORY (DCR-TRV22/TRV33 only)
To make the buttons on the LCD screen disappear

Press OK.

Note When the POWER switch is set to VCR or during memory playback (DCR-TRV22/ TRV33 only), you cannot operate with the touch panel using the viewfinder. To operate the items that do not appear Set the LCD panel and the viewfinder back to the previous position. Operate the items using the LCD screen. Available recording time This is the time when you record pictures using the viewfinder (p. 19).

RETURN EXECUTE ENGAGE REC PAUSE V I DEO ED I T ED I T SET CONTROL ADJ TEST CUT - I N CUT -OUT I R SETUP PAUSEMODE I R TEST
0 V I DEO ED I T ED I T SET CONTROL ADJ TEST CUT - I N CUT -OUT I R SETUP PAUSEMODE I R TEST
Notes When you complete Step 3: Adjusting the synchronization of the VCR, the image to adjust the synchronization is recorded for about 50 seconds. If you start recording from the very beginning of the tape, the first few seconds of the tape may not be recorded properly. Be sure to allow about 10 seconds lead before starting the recording. When the recording unit cannot be operated properly by the i.LINK cable, leave the connection as it is, and make the A/V connecting cable settings (p. 85). Video and audio are sent by digital signals.
Dubbing only desired scenes Digital program editing (on tape) Operation 1: Making a Program
(1) Insert the cassette for playback into your camcorder, and insert the tape for recording into the VCR. (2) Press FN to display PAGE1. (3) Press MENU to display the menu. , then press EXEC (p. 185). (4) Select VIDEO EDIT in (5) Select TAPE, then press EXEC (DCR-TRV22/TRV33 only). (6) Search for the beginning of the first scene you want to insert using the video control buttons on the screen, then pause playback. You can fine-adjust one /. frame at a time with (7) Press MARK IN on the screen. The IN point of the first program is set, and the upper part of the program mark changes to light blue. (8) Search for the end of the first scene you want to insert using the video control buttons on the screen, then pause playback. You can fine-adjust one frame at /. a time with (9) Press MARK OUT on the screen. The OUT point of the first program is set, then the lower part of the program mark changes to light blue. (10) Repeat steps 6 to 9. When one program is set, the program mark changes to light blue. You can set 20 programs in maximum.
V I DEO ED I T MARK OUT START 0:08:55:06 ED I T SET UNDO TOTAL 0:00:00:00 SCENE 0
V I DEO ED I T MARK IN START 0:09:07:06 ED I T SET UNDO TOTAL 0:00:13:00 SCENE 1
V I DEO ED I T MARK IN START 0:10:01:23 ED I T SET UNDO TOTAL 0:00:47:12 SCENE 3
To finish making programs
Press END. The program is stored in memory until the tape is ejected.
Note You cannot operate recording during Digital program editing. On a blank portion of the tape You cannot set IN or OUT. If there is a blank portion between the IN point and the OUT point on the tape The total time may not be displayed correctly.

Selecting image quality and size
Selecting still image quality
The default setting is FINE. The POWER switch should be set to Operate by touching the panel. or.
Press FN to display PAGE1. Press MENU to display the menu. , then press EXEC (p. 179). Select STILL SET in Select QUALITY, then press EXEC. Select a desired image quality, then press EXEC.
MEM SET 1 STILL SET BURST QUALITY IMAGESIZE
OFF MEM SET 1 STILL SET BURST QUALITY IMAGESIZE

FINE STANDARD

REMAIN 12

EXEC RET. EXEC RET.

Image quality settings
Setting FINE (FINE) STANDARD (STD) Meaning Use this mode when you want to record high quality images. Fine images are compressed to about 1/4. This is the standard image quality. Standard images are compressed to about 1/10.
Selecting image quality and size Selecting the still image size (DCR-TRV33 only)
You can select image size or (When the POWER switch is set to CAMERA or VCR, the image size is automatically set to 640 480.) The default setting is 1152 864. The POWER switch should be set to Operate by touching the panel.
Press FN to display PAGE1. Press MENU to display the menu. , then press EXEC (p. 179). Select STILL SET in Select IMAGESIZE, then press EXEC. Select a desired image size, then press EXEC. The indicator changes as follows:
Memory capacity of still images
Recorded images are compressed in JPEG format before being stored in memory. The memory capacity of still images varies depending on the selected image quality mode and image size. Details are shown in the table below. image size (DCR-TRV33 only) Image quality FINE STANDARD image size Image quality FINE STANDARD Memory capacity About 150 KB About 60 KB Memory capacity About 500 KB About 200 KB
Selecting image quality and size Selecting the moving picture size
You can select picture size or 160 112. The default setting is 320 240. The POWER switch should be set to Operate by touching the panel. or.
Press FN to display PAGE1. Press MENU to display the menu. , then press EXEC (p. 179). Select MOVIE SET in Select IMAGESIZE, then press EXEC. Select a desired image size, then press EXEC. The indicator changes as follows:

320 160

EXIT MEM SET 1 MOVIE SET IMAGESIZE REMAIN
MEM SET 1 EXIT 320 MOVIE SET IMAGESIZE REMAIN REMAIN 30sec

REMAIN 30sec

Selecting image quality and size Approximate number of still images and time of moving pictures that can be recorded on one Memory Stick
The number of still images and the time of moving pictures you can record vary depending on which image quality mode you select and the complexity of the subject.

Still images

FINE Type of 1152 864* Memory Stick 1152 864*
(unit: image) STANDARD 640 480
8MB (supplied) 16MB (optional) 32MB (optional) 64MB (optional) 128MB (optional) * DCR-TRV33 only 245

Moving pictures

Image size Type of Memory Stick 8MB (supplied) 16MB (optional) 32MB (optional) 64MB (optional) 128MB (optional) 160 112
1 min. 20 sec. 2 min. 40 sec. 5 min. 20 sec. 10 min. 40 sec. 21 min. 20 sec.
5 min. 20 sec. 10 min. 40 sec. 21 min. 20 sec. 42 min. 40 sec. 1 hr. 25 min. 20 sec.
The table shows approximate number of still images and time of moving pictures that can be recorded on the Memory Stick formatted with your camcorder.
Recording still images on Memory Stick Memory Photo recording
(1) Keep pressing PHOTO lightly and check the image. The green z indicator stops flashing, then lights up. The brightness of the image and the focus are adjusted and fixed while your camcorder targets at the center of the image. Recording does not start yet. (2) Press PHOTO deeply. Recording is complete when the bar scroll indicator disappears. The image displayed on the screen when you pressed PHOTO deeply will be recorded on the Memory Stick.
[a] Approximate number of images that can be recorded on the Memory Stick [b]Folder name to be recorded in Note The recordable number of the images indicated on the screen depends on the setting of the image quality and the condition of the subject being recorded. Therefore, the recordable number may not change even after recording has been done. You can record still images on the Memory Stick during tape recording or tape recording standby See page 46 for details.
When the POWER switch is set to MEMORY The following functions do not work: Digital zoom Super NightShot Color Slow Shutter Wide mode Fader Picture effect Digital effect SPORTS of PROGRAM AE (The indicator flashes.) Title SteadyShot While you are recording a still image You cannot turn off the power or press PHOTO. When you press PHOTO on the Remote Commander Your camcorder immediately records the image that is on the screen when you press the button. When you press PHOTO lightly in step 1 The image momentarily flickers. This is not a malfunction. Recording data The recording data (date/time or various settings when recorded) do not appear during recording. However, they are recorded automatically onto the Memory Stick. To display the recording data, press DATA CODE during playback. You can also use the Remote Commander for this operation (p. 40). If you record still images when the POWER switch is set to MEMORY (DCR-TRV33 only) The angle of view is slightly larger compared with the angle of view when the POWER switch is set to CAMERA.
Recording images continuously (DCR-TRV33 only)
You can record still images continuously. Before recording, select a desired mode described below in the menu settings.
) NORMAL [a] ( Your camcorder shoots up to 4 still images in size or 13 still images in size at about 0.5 sec intervals. EXP BRKTG [b] ( ) Your camcorder automatically shoots 3 images at about 0.5 sec intervals at different exposures. [a] [b]

Viewing a still image Memory Photo playback
You can view the still images recorded in the Memory Stick. Further, you can view 6 images including moving pictures at a time arranged in the same order of recording in the Memory Stick by selecting the index screen. The POWER switch should be set to Operate by touching the panel.
(1) Press PLAY. The last recorded image is displayed. (2) Press (to see the previous image)/+ (to see the next image) on your camcorder to select a desired still image.
To cancel Memory Photo playback
Press CAM. When no file is in the folder The NO FILE AVAILABLE indicator appears.
Screen indicators during still image playback
To make screen indicators disappear, press DSPL/BATT INFO. Image size

8/MEMORY PLAY

1010008
Image number/Total number of recorded images in the current playback folder Folder name Print mark Protect

CAM INDEX FN

Recording date/time/Various settings
Data file name Recording data To display the recording data (date/time or various settings when recorded), press DATA CODE during playback. You can also use the Remote Commander for this operation (p. 40). Notes on the file name The folder number may not appear and only the file name may appear if the folder structure does not conform to the DCF standard. The file name flashes on the screen if the file is corrupted or the file is unreadable. When the Memory Stick contains multiple folders The following icons appear on the screen on the first or last images in a folder.
: You can move to the previous folder. : You can move to the next folder. : You can move to both the previous and the next folders.
Viewing a still image Memory Photo playback Playing back 6 recorded images at a time (index screen)
This function is especially useful when searching for a particular image. Press INDEX to display the index screen. A red B mark appears above the image that was being displayed before the change to the index screen mode.
T: To display the previous 6 images t: To display the next 6 images

B mark

-MARK MARK DEL EXIT
This number indicates the recorded order of the image in the folder.
To return to the normal playback screen (single screen)
Press the image you want to display.
Viewing a moving picture MPEG MOVIE playback
You can view the moving pictures recorded in the Memory Sitck. The POWER switch should be set to Operate by touching the panel.

For Windows 2000 and Windows XP users If you are prompted to confirm the digital signature, select Yes (Windows 2000) or Continue Anyway (Windows XP).
(1) Insert the Memory Stick into your camcorder. (2) Connect the AC Adaptor, then set the POWER switch to. (3) Connect the (USB) jack on your camcorder to the USB port on the computer using the supplied USB cable. The USB MODE indicator appears on the LCD screen of your camcorder. The computer recognizes your camcorder, and the Add Hardware Wizard starts.
(4) Follow the on-screen messages so that the Add Hardware Wizard recognizes that the USB drivers have been installed. The Add Hardware Wizard starts twice because 2 different USB drivers are installed. Be sure to allow the installation to complete without interrupting it.
You cannot install the USB driver if the Memory Stick is not in your camcorder Be sure to insert the Memory Stick into your camcorder before installing the USB driver.
Connecting your camcorder to a computer using the USB cable (For Windows users) If you cannot install the USB driver
The USB driver has been registered incorrectly as the computer was connected to your camcorder before installation of the USB driver was complete. Follow the procedure below to correctly install the USB driver.
Step 1: Uninstall the incorrect USB driver
1 Turn on the computer and allow Windows to load. 2 Connect the AC Adaptor and set the POWER switch to 3 Connect the USB port on the computer to the (USB) jack on your camcorder using the USB cable supplied. 4 Open the computers Device Manager. Windows XP: Select Start t Control Panel t System t Hardware, and click the Device Manager button. If there is no System inside Pick a category after clicking Control Panel, click Switch to classic view instead. Windows 2000: Select My Computer t Control Panel t System t Hardware tab, and click the Device Manager button. Windows 98SE/Windows Me: Select My Computer t Control Panel t System, and click Device Manager. 5 Select and delete the underlined devices below.

Windows 98SE

Windows Me

Windows 2000 Windows XP

6 Set the POWER switch to OFF (CHG) on your camcorder, then disconnect the USB cable. 7 Restart the computer.
Step 2: Install the USB driver on the supplied CD-ROM Follow the entire procedure in Installing the USB driver on page 153.
Step1: Uninstall the incorrect USB driver
Turn on the computer and allow Windows to load. Insert the Memory Stick into your camcorder. Connect the AC Adaptor and set the POWER switch to Connect the USB port on the computer to the (USB) jack on your camcorder using the USB cable supplied. 5 Open the computers Device Manager. Windows XP: Select Start t Control Panel t System t Hardware tab and click the Device Manager button. If there is no System inside Pick a category after clicking Control Panel, click Switch to classic view instead. Windows 2000: Select My Computer t Control Panel t System t Hardware, and click the Device Manager button. Other OS: Select My Computer t Control Panel t System, and click Device Manager. 6 Select Other devices. Select the device prefixed with the ? mark and delete. Ex: (?)Sony Handycam 7 Set the POWER switch to OFF (CHG) on your camcorder, then disconnect the USB cable. 8 Restart the computer.

Remove the cassette before the following operation. If a cassette is inserted, the power automatically goes out after about 5 minutes.
Follow steps 1 and 2 on page 162. Set the POWER switch to on your camcorder. Follow steps 4 to 8 on page 161. Follow steps 3 to 7 in Capturing images recorded on a tape onto a CD-R.
Notes A computer equipped with a CD-R drive is required. Do not press the button on your camcorder while creating a Video CD. If you press , only images up to that point are captured onto the Video CD and Easy Vide CD is closed. Once a disc has been created, you cannot add images to the disc. Select a hard disc with sufficient memory (at least 6 GB) from the Option screen as location for the "Location of work folder.
Playing back the Video CDs
You can use a DVD player or a computer with a DVD drive to play back the Video CDs you created. To play back the Video CDs on a computer, you need to have Video CD software installed. You can play back the Video CDs with Windows Media Player. However, you cannot use menu functions such as capturing function. You also may not be able to play back the Video CDs in certain computer environment such as OS or hardware.
(1) Start up Windows Media Player. Windows XP: Select Start t All Programs t Accessory t Entertainment, and click Windows Media Player. Other OS: Select Start t Programs t Accessory t Entertainment, and click Windows Media Player. (2) Select My Computer t CD-R, and click MPEGAV folder, then drag and drop the sss*.DAT moving picture file onto the Media Player screen. The moving picture will be played back. * The file name is displayed in sss.
Created Video CDs The maximum recording time is about l hour. Capturing is divided into segments of about 10 minutes (approximately 4 GB) due to the specifications of the AVI capturing format. As a result, recording includes segment links about every 10 minutes, at which point images are skipped for several seconds.
Seeing the on-line help (operating instructions) of ImageMixer
An ImageMixer Ver.1.5 for Sony on-line help site is available where you can find the detailed operating method of ImageMixer Ver.1.5 for Sony.

sss stands for any number (within the range) from 101 to 999. Copying a file to the hard disk of the computer before viewing it is recommended. If you play back the file directly from the Memory Stick, the image and sound may break off.
Viewing images recorded on Memory Stick on a computer (For Windows users) (DCR-TRV22/TRV33 only) Image file storage destinations and image files
Image files recorded with your camcorder are grouped according to the folders in the Memory Stick. The meanings of the file names are as follows. ssss stands for any number within the range from 0001 to 9999.
Example: For Windows Me users (The drive recognizing your camcorder is [F:].)
Folder containing image files recorded using other camcorders (for playback only)
Folder containing image files recorded with your camcorder When no new folders have been created, this is only 101MSDCF. Folder containing moving picture data recorded using other camcorders (for playback only)
Folder 101MSDCF (up to 999MSDCF)
File DSC0ssss.JPG MOV0ssss.MPG
Meaning Still image file Moving picture file
When finished, disconnect the USB cable and remove the Memory Stick or set the POWER switch to OFF (CHG)
For Windows 2000, Windows Me, Windows XP users
(1) Move the cursor to the Unplug or Eject Hardware on the Task Tray and click to cancel the applicable drive. (2) After the Safe to remove message appears, disconnect the USB cable and eject the Memory Stick or set the POWER switch to OFF (CHG).
Connecting your camcorder to a computer using the USB cable (For Macintosh users) (DCR-TRV22/TRV33 only)
When connecting to a computer with the USB cable
You must install a USB driver onto a computer in order to connect your camcorder to the computers USB port. The USB driver can be found on the CD-ROM supplied, along with the application software required for viewing images.
Recommended Macintosh environment
Mac OS 8.5.1/8.6/9.0/9.1/9.2 or Mac OS X (v10.0/v10.1/v10.2) standard installation is required. However, note that the update to Mac OS 9.0/9.1 should be used for the following models. iMac with the Mac OS 8.6 standard installation and a slot loading type CD-ROM drive iBook or Power Mac G4 with the Mac OS 8.6 standard installation The USB port must be provided as standard. QuickTime 3.0 or later must be installed to play back moving pictures. Notes Operations are not guaranteed for the Macintosh environment if you connect two or more USB equipment to a single computer at the same time, or when using a hub. Some equipment may not operate depending on the type of USB equipment that is used simultaneously. Operations are not guaranteed for all the recommended computer environments mentioned above. Macintosh, iMac, iBook, Power Mac and Mac OS, QuickTime are trademarks of Apple Computer Inc. All other product names mentioned herein may be the trademarks or registered trademarks of their respective companies. Furthermore, TM and are not mentioned in each case in this manual.

Symptom Cause and/or Corrective Actions A vertical band appears when you The contrast between the subject and background is too shoot a subject such as lights or a high. This is not a malfunction. candle flame against a dark background. A vertical band appears when you This is not a malfunction. shoot a very bright subject. Some tiny spots in white, red, blue SLOW SHTR, Super NightShot* or Color Slow Shutter* is or green appear on the screen. activated. This is not a malfunction. An unknown picture appears on If 10 minutes elapse after you set the POWER switch to the screen. CAMERA or DEMO MODE is set to ON in the menu settings without a cassette and the Memory Stick* inserted, your camcorder automatically starts the demonstration. c Insert a cassette/a Memory Stick* or press the LCD screen. The demonstration stops. You can also set DEMO MODE to OFF in the menu settings (p. 183). The picture is recorded in incorrect NIGHTSHOT is slid to ON. or unnatural colors. c Slide it to OFF (p. 34). Picture appears too bright, and the NIGHTSHOT is slid to ON in a bright place. subject does not appear on the c Slide it to OFF (p. 34). screen. The back light function is active. c Deactivate it (p. 33). The click of the shutter does not BEEP is set to OFF in the menu settings. sound. c Set it to MELODY or NORMAL (p. 184). Black bands appear when you c Set STEADYSHOT to OFF in the menu settings (p. 177). record a TV screen or computer screen. An external flash (optional) does The power of the external flash (optional) is off or the not work. power source is not installed. c Turn on the external flash (optional) or install the power source. 2 or more external flashes (optional) are attached. c Only 1 external flash (optional) can be attached. Flickering or changes in color PORTRAIT or SPORTS of PROGRAM AE is on when occurs. recording in light from a discharge tube such as a fluorescent lamp, sodium lamp or mercury lamp. c Set PROGRAM AE to AUTO in the menu settings. * DCR-TRV22/TRV33 only (continued on the following page)
Types of trouble and how to correct trouble During playback
Symptom Playback is not possible. Cause and/or Corrective Actions The tape has run out. c Rewind the tape (p. 39). There are horizontal lines on the The video head may be dirty. picture or the playback picture is c Clean the head using the cleaning cassette (optional) (p. not clear or does not appear. 206). No sound or only a low sound is The stereo tape is played back with HiFi SOUND set to 2 heard when playing back the tape. in the menu settings. c Set HiFi SOUND to STEREO (p. 178). Volume is turned to minimum. c Turn up the volume (p. 39). AUDIO MIX is set to the ST2 side in the menu settings. c Adjust AUDIO MIX (p. 178). The sound breaks off. The video head may be dirty. c Clean the head using the cleaning cassette (optional) (p. 206). Displaying the record date, date The cassette has no Cassette Memory. search does not work. c Use a cassette with Cassette Memory (p. 77). CM SEARCH is set to OFF in the menu settings. c Set it to ON (p. 181). The tape has a blank portion in the recorded portion (p. 78). Title search does not work. The cassette has no Cassette Memory. c Use a cassette with Cassette Memory (p. 76). CM SEARCH is set to OFF in the menu settings. c Set it to ON (p. 181). There is no title in the tape. c Superimpose the titles (p. 101). The tape has a blank portion in the recorded portion (p. 76). New sound added to the recorded AUDIO MIX is set to the ST1 side in the menu settings. tape is not heard. c Adjust AUDIO MIX (p. 178). The title is not displayed. TITLE DSPL is set to OFF in the menu settings. c Set it to ON (p. 181). Picture is distorted and PAL The TV color system recorded on the tape is different from appears on the screen. that of the TV.

- The image is protected1) 2)
Slow flashing: The image is protected (p. 142). Warning indicator as to the flash (optional) Slow flashing: During charging Fast flashing: The self-diagnosis display function is activated (p. 194).1) There is something wrong with the external flash (optional). Warning indicator as to still image recording Slow flashing: The still image cannot be recorded on the Memory Stick (p. 47).2) The still image cannot be recorded on the tape (p. 48).3) You hear the melody or beep sound. DCR-TRV22/TRV33 only 3) DCR-TRV19 only

Warning messages

If messages appear on the screen, check the following. See the page in parentheses ( ) for details. CLOCK SET FOR InfoLITHIUM BATTERY ONLY CLEANING CASSETTE Set the date and time (p. 22). Use an InfoLITHIUM battery pack (p. 17). The video heads are dirty (p. 206). The x indicator and CLEANING CASSETTE message appear one after another on the screen. You tried to record a picture that has a copyright control signal (p. 197).1)
The Cassette Memory is full (p. 102, 104, 105).1) AUDIO MODE is set to 16BIT.1) You cannot dub new sound (p. 99, 182). REC MODE is set to LP.1) The tape was recorded in different color system. You cannot dub new sound (p. 99, 182). There is no recorded portion on the tape.1) You cannot dub new sound (p. 99). i.LINK cable is connected.1) You cannot dub new sound (p. 99). The Memory Stick is full (p. 117).1) The write-protect tab on the Memory Stick is set to LOCK (p. 106).1) No image is recorded or recognized on the Memory Stick.1)

COPY INHIBIT

FULL 16BIT REC MODE TAPE i.LINK CABLE FULL3)
NO FILE3) NO MEMORY STICK AUDIO ERROR
No Memory Stick is inserted.1) You are trying to record an image with sound that cannot be recorded by your camcorder on the Memory Stick (p. 128).1) The Memory Stick is not recognized (p. 109).1) Check the format. The image is distorted and cannot be played back.1) 2) Turn the power off once, then on again.1) The Memory Stick inserted is not compatible with your camcorder.1) The tape has reached the end of the tape.1) Insert a cassette.1) You press PHOTO on your camcorder while deleting data in the Memory Stick.1) You press PHOTO on your camcorder while formatting the Memory Stick.1) Charging an external flash (optional) does not work correctly.1) You have reached the maximum number of folders that can be created.1)
MEMORY STICK ERROR3) The Memory Stick data is corrupted (p. 109).1) FORMAT ERROR3) PLAY ERROR3) REC ERROR3) INCOMPATIBLE MEMORY STICK3)
READ-ONLY MEMORY STICK3) A read-only Memory Stick is inserted.1) Q Z TAPE END Q NO TAPE DELETING
FORMATTING3) NOW CHARGING

FOLDER NO. FULL

USB STREAMING ON GOING3) USB streaming is being performed.
You hear the melody or beep sound. Reinsert the Memory Stick and play back. 3) DCR-TRV22/TRV33 only

If the video heads get dirtier, the entire screen becomes blue [c]. Note on the video head The video head suffers from wear after long use. If you cannot obtain a clear image even after using a cleaning cassette, it might be because the video head is worn. Please contact your Sony dealer or local authorized Sony service facility to have the video head replaced.

Cleaning the LCD screen

If fingerprints or dust make the LCD screen dirty, we recommend using a cleaning cloth (supplied) to clean the LCD screen. When you use the LCD Cleaning Kit (optional), do not apply the cleaning liquid directly to the LCD screen. Clean the LCD screen with cleaning paper moistened with the liquid.
Removing dust from inside the viewfinder
(1) Pull out the viewfinder in the direction of the arrow 1. Remove the 2 screws in the direction of the arrow 2 with a screwdriver (optional). Remove the eyecup in the direction of the arrow 3.
(2) Remove dust from inside the eyecup and viewfinder with a commercially available blower. (3) In the reverse way of (1), attach the eyecup.
Caution Do not remove any other screws. You may remove only the screw to remove the eyecup. Notes Do not drop or bend the eyecup shaft. Be careful when handling the eyecup.
Charging the built-in rechargeable battery
Your camcorder has a built-in rechargeable battery so that the date, time and other settings are retained even when the POWER switch is set to OFF (CHG). The built-in rechargeable battery is always charged as long as you are using your camcorder. The battery, however, will get discharged gradually if you do not use your camcorder. It will be completely discharged in about 4 months if you do not use your camcorder at all. Even if the built-in rechargeable battery is not charged, it will not affect your camcorder operation. To retain the date and time, etc., charge the battery if the battery is discharged. Charging the built-in rechargeable battery Connect your camcorder to the house current using the AC Adaptor supplied with your camcorder, and leave your camcorder with the POWER switch set to OFF (CHG) for more than 24 hours. Or install the fully charged rechargeable battery pack on your camcorder, and leave your camcorder with the POWER switch set to OFF (CHG) for more than 24 hours.

To prepare the Remote Commander
Insert 2 size AA (R6) batteries by matching the + and on the batteries to the + and in the battery compartment.
Notes on the Remote Commander Point the remote sensor away from strong light sources such as direct sunlight or overhead lighting. Otherwise, the Remote Commander may not function properly. Your camcorder works in the Commander mode VTR 2. Commander modes 1, 2 and 3 are used to distinguish your camcorder from other Sony VCRs to avoid erroneous remote control operation. If you use another Sony VCR in the Commander mode VTR 2, we recommend changing the Commander mode or covering the sensor of the VCR with black paper.
Identifying parts and controls Operation indicators LCD screen and Viewfinder
50min STILL 16:9WIDE NEG. ART

STBY +

0:12:34 48min ZERO SET MEMORY END SEARCH
qd qf qg qh qj qk ql w; wa ws wd wf wg wh wj

AUTO 60 AWB F1.8 9dB

DV IN 16BIT 12:05:56PM
1 Cassette Memory (p. 27) 2 Remaining battery time (p. 27) 3 Zoom (p. 32)/Exposure (p. 64)/ Data file name* (p. 135) 4 Digital effect* (p. 59, 73)/MEMORY MIX* (p. 120)/Fader (p. 55) 5 16:9WIDE (p. 53) 6 Picture effect (p. 58, 72) 7 Data code (p. 40) 8 Volume (p. 39)/Date (p. 27) 9 PROGRAM AE (p. 62) 0 Back light (p. 33) qa SteadyShot off (p. 177) qs Manual focus (p. 66) qd Self-timer (p. 50, 127) qf Recording mode (p. 182) qg Camera recording lamp*** (p. 48) qh STBY/REC (p. 25)/Video control mode (p. 39)/Image size** (p. 111)/Image quality* (p. 110) qj NIGHTSHOT (p. 34)/SUPER NIGHTSHOT* (p. 34)/COLOR SLOW SHUTTER* (p. 35)
qk Warning (p. 195) ql Time code (p. 27)/Tape counter (p. 27)/ Self-diagnosis (p. 194)/Photo recording (p. 114)/Image number* (p. 135) /Playback folder* (p. 138) /Tape photo*** (p. 48) w; Remaining tape (p. 27)/Memory playback* (p. 135) wa ZERO SET MEMORY (p. 75)/Interval Recording (p. 68)/Interval Photo Recording (p. 124)/Frame recording (p. 69) ws END SEARCH (p. 37) wd A/V t DV* (p. 173)/DV IN*** (p. 94) wf Audio mode (p. 182)/Recording folder* (p. 134) wg Time (p. 27) wh Flash (p. 176) This indicator appears only when the flash (optional) is in use. wj Continuous photo recording** (p. 116)
* DCR-TRV22/TRV33 only ** DCR-TRV33 only *** DCR-TRV19 only
AC Adaptor.. 17, 21 Adjusting the viewfinder. 31 Audio dubbing. 97 AUDIO MIX. 178 AUDIO MODE.. 182 AUTO SHTR. 176 A/V connecting cable. 44, 81, 93 BACK LIGHT. 33 Battery Info.. 20 Battery pack.. 16 BEEP.. 184 BOUNCE.. 55

I, J, K, L

i.LINK.. 202 i.LINK cable.. 81, 93 Image protection. 142 Image quality. 110 Image size.. 111, 112 Index screen.. 136 InfoLITHIUM battery pack.. 16, 200 Infrared rays emitter. 34, 86 Insert editing.. 95 Intelligent accessory shoe.. 98, 215 Interval Recording. 68 Interval Photo Recording... 124 JPEG.. 106 Labeling a cassette. 104 LANC jack.. 216 LUMINANCE KEY. 59

DIY Eye Tracking

Visible Spectrum

Match Moving

Ellipse Fitting

NIR Spectrum

Summary

Lecture: Eye Camera Settings and Calibration

Andrew T. Duchowski

School of Computing, Clemson University
` Univeristat Autonoma de Barcelona, 2009, 25 November, Barcelona, Spain

1 / 62

To fully understand how it works, build it yourself Babcock and Pelz (2004) were one of the rst to describe a DIY wearable analog video tracker
an expensive video multiplexer was needed to synchronize eye and scene cameras system relied on somewhat dated (by todays standards) video recorder (a Sony DCR-TRV19 DVR) nevertheless, fostered nascent open-source movement

2 / 62

General Motion Capture
An eye tracker is a specialized kind of motion capture (mocap) device, delineated within four classes: Electromechanical: skin sensors around the eye cavity (electro-oculography, or EOG) Electromagnetic: a metallic stalk is xed to contact lens Video-oculography, or VOG: analysis (historically manual) of video frames Video-based with IR reection: eye trackers benet from one (or more) of four types of IR reections, two from the front and rear surface of the cornea, two from the front and rear surface of the crystalline lens; all four reections are known as the Purkinje images (Crane, 1994) For comparison with video mocap, see Menache (2000)

3 / 62

Two Approaches to Video-Based Eye Tracking
1 Based on a 3D eyeball model (e.g., Gullstrands), compute the ray emanating from a central (nodal) point within the eye, then compute ray/plane intersections within the scene, p = c + t L, where p is the 3D Point Of Gaze, or POG, c is the center of the eye (or cornea, modeled by a sphere), t is the parametric distance along the ray L coinciding with the optical or visual axis 2 Based on traditional video-oculography, estimate the limbus center (x, y )the POG in the scene image is obtained via a mapping of (x, y) to scene coordinates (sx , sy ) given known calibration point coordinators See Hennessey (2008) for a detailed review

4 / 62

Building Our Own
For years I refused to build my own With digital cameras becoming affordable and easier to use, it became feasible (and fairly easy) Derrick Parkhurst suggested a pair of cameras to use My graduate student, Wayne J. Ryan, took up the project (most of these notes are based on his wok)

5 / 62

Off-line Visible Spectrum Wearable Tracker

6 / 62

Visible Spectrum Wearable Schematic

7 / 62

Parts List
Vendor AOSafety AOSafety Jose Ann Fabric Home Depot Amazon.com DejaView
Description X-Factor XF503 safety glasses I-Pilot 90714 nose piece Polyester braided elastic Screws Aluminum pod 1/4 dia. 18 long Two Camwear Model 200 cameras Total
Price $10.00 $10.00 $1.50 $0.75 $5.00 $500.00 $527.25

8 / 62

Visible Spectrum Design
Design based to a large extent on that of Li et al. (2006) We rst explored tracking in the visible spectrum, following Li and Parkhurst (2006) Gaze tracking relies on tracking of the limbus, or the iris/sclera boundary No IR simplies design but complicates image analysis Headgear must be stable to offer sufciently stable reference point for gaze point estimation

9 / 62

Software Architecture

10 / 62

Basic Steps
1 Video preprocessing: convert to some known format from which you can extract frames
we had problem with.asf format; turns out it was proprietary by virtue of 4 unknown bytes in le header, easily xed once known nding good open source codec library was a problem; latest version of ffmpeg is excellent
2 Synchronization: line up eye and scene video feeds via ash screen (as suggested by Li and Parkhurst (2006)) 3 Calibration: track pupil center and calibration dot 4 Visualization/Analysis: detect xations in raw gaze point data stream

11 / 62

Pupil Tracking
Major problem is pupil trackinga good starting point is the Starburst algorithm (Li et al., 2006) Idea is to shoot out rays from within pupil to locate limbic boundary feature points along those rays A seed point is needed from which to shoot rays Our initial approach was based on template matching:

Cannys (1986) edge detector Borgefors (1988) chamfer image pixel pattern

(a) image frame

(b) Canny edges

(c) chamfer image

(d) pattern

12 / 62

Searchmap
A gradient image is also use to guide the algorithm for nding Starbursts initial seed point
MakeSearchMap(Gradient image G, Chamfer image C, Template image T , Template gradient GT ) { for all template locations x, y { sum = 0; for all locations dx, dy T = 0 sum += G(x + dx, y + dy) GT (dx, dy) C(x + dx, y + dy); M(x, y) = sum; // M is the search map; } return maximum pixel in M; }

14 / 62

Feature Point Detection (Starburst)
FeatureDetect(Gradient image G, Threshold T , Start point S) { for from 0 to 2 { // phase 1 L = S; // L is the current location x = cos(); y = sin(); while G(L) < T { L(x) += x ; L(y) += y ; } push L on P; // P is a point stack } for from 2 to 0 { // phase 2 pop L off P; add L to P ; // P is a point set for from ( + ) to ( ) while GL < T { L(x) += x ; L(y) += y ; } add L to P ; } return P ; }

16 / 62

Starburst returns a set of feature points Next step is to t ellipses to points Starburst used RANSAC (Random Sample Consensus) Our visible spectrum approach t multiple ellipses and distinguished between those that t the pupil, those that t the limbus, and those that spanned both (erroneous) Ellipse is used to estimate pupil center (x, y )

17 / 62

Calibration
Calibration maps pupil center (x, y) in eye frames to calibration points (sx , sy ) in the scene frames Calibration dots are displayed on a computer monitor viewed by the scene camera, tracked by a 5-step algorithm

(a) scene frame

(b) low threshold

(c) get largest region

(d) ood ll holes

(e) high threshold

(f) nal result

18 / 62

Computing the Calibration Homography
With calibration screen coordinates (sx , sy ) and pupil center (x, y), the mapping is expressed as a second order polynomial (C. H. Morimoto & Mimica, 2005): sx sy = a0 + a1 x + a2 y + a3 xy + a4 x 2 + a5 y 2 = b0 + b1 x + b2 y + b3 xy + b4 x 2 + b5 y 2
The parameters {a0 ,. , a5 } and {b0 ,. , b5 } can be computed via Lagranges method of least squares (e.g, see Lancaster and Salkauskas (1986))

19 / 62

Reformulate the above equations by writing: 1 x1 y1 xy1 x1 y1 s1x s1y s2x s2y 1 x2 y2 xy2 x 2 y 2 . . . . . . . . six siy = 1 xi yi xyi x 2 y 2 i i . . . . . . snx sny 1 xn yn xyn xn yn or in matrix notation, Y = XB An estimate of B is obtained by left-multiplying by (XT X)1 B = (XT X)1 XT Y Note that this is done for n calibration points, each sampled multiple times

a0 a1 a2 a3 a4 a5

b0 b1 b2 b3 b4 b5

20 / 62

Preliminary Results

21 / 62

Natural Tasks
Eventually wed like to investigate natural tasks Count xations falling over Areas Of Interest (AOIs)

22 / 62

Tracking Renements
Cast ray R from origin O Terminate ray if it exits a dark region (check gradient magnitude collinear with ray) max signies feature point Constrain search by exploiting temporal locality
assume points are near E identied in previous frame if P is intersection of R and E constrain search via max (O + (P O) : 0.8 < < 1.2)

23 / 62

Interactive Manual Intervention
Manual intervention via GUI to synchronize video streams (nd ashes in image frames) Seed Starburst via manually overlayed template Similarly, use crosshair for initial location of calibration dot

24 / 62

AOI Tracking

m xn reference window

A p x q search field
Need to track AOIs to count xations falling within Similar to match-moving trackbox (Paolini, 2006) Bright spot is found where I(x, y) is maximum For efciency, a summed area table S is precomputed (Crow, 1984; Viola & Jones, 2004) S(x, y) =

I(i, j),

0 < i < x,

0<j <y

(S(A) + S(B)) (S(C) + S(D)) pq

25 / 62

Applied Example
Technique worked fairly well in applied setting

26 / 62

OpenCV
We are exploring OpenCV for feature tracking

27 / 62

Revisiting Ellipse Fitting
Ellipse tting is an essential eye tracking component RANSAC ts many ellipses, one winnerslow Given n points {xi , yi }, t the quadratic such that ax 2 + bxy + cy 2 + dx + ey + f = 0 We can rewrite this as x 2 xy y2 x y 1 a b c d e f
Then rewrite in matrix notation XA = B And solve for A

28 / 62

Ellipse denition
Alternate representation of ellipse is (x h)2 (y k)2 + =1 r2 s2 With center (h, k ) and axes (r, s) of length r = r and s = s respectively, coefcients are expressed as:
a b c d e f = s2 M 2 + r 2 N 2 = 2MN(s2 r 2 ) = s2 N 2 + r 2 M 2 = 2h(s2 M 2 + r 2 N 2 ) 2MNk(s2 r 2 ) = 2k (s2 N 2 + r 2 M 2 ) 2MNh(s2 r 2 ) = M 2 (s2 h2 + r 2 k 2 ) + N 2 (r 2 h2 + s2 k 2 ) + 2MNhk(s2 r 2 ) r 2 s2
With M = cos (), N = sin () for an ellipse rotated about its center by angle = tan1 (ry /rx )

29 / 62

Method of Least-Squares
We can set f = 1 (or any non-zero arbitrary constant), ax 2 + bxy + cy 2 + dx + ey = 1 Then rewrite as x 2 xy y2 x y a b c d e

XA = B And solve using the pseudo-inverse of X A = (XT X)1 XT B With X = B = xi2 xi yi yi2 xi yi

11 12. 1n

This works reasonably well, but recovering center (h, k), axes (r, s), and angle is problematic

30 / 62

Fitzgibbon et al. (1996)
Most eye tracking papers refer to Fitzgibbon et al. (1996) Method exploits ellipse-specic constraint b2 4ac < 0 The tting problem is reformulated as min Da

subject to aT Ca = 1

where the design matrix D of size n 6, x1 x1 y1 y1 x1 y. . .x x D = i xi yi yi yi 1 i . . . 2 x y 2 x xn n n yn n yn 1 represents the least squares minimization

31 / 62

The constraint matrix C of size 6 6, expresses the constraint b2 4ac < 0, C = The minimization problem (1) is solved by a quadratically constrained least squares minimization Apply the Lagrange multipliers for the optimal solution a Sa = Ca where S is the scatter matrix of size 6 6, S = DT D

32 / 62

The scatter matrix S Sx 4 Sx 3 y S S = x y S 3 x S 2 x y Sx 2
Sx 3 y Sx 2 y 2 Sxy 3 Sx 2 y Sxy 2 Sxy
Sx 2 y 2 Sxy 3 Sy 4 Sxy 2 Sy 3 Sy 2
Sx 3 Sx 2 y Sxy 2 Sx 2 Sxy Sx
Sx 2 y Sxy 2 Sy 3 Sxy Sy 2 Sy

Sx 2 Sxy Sy 2 Sx Sy S1

represents the least squares minimization in which the operator S denotes the sum

Sx a y b =

xia yib
The system (2) is solved by using generalized eigenvectors

33 / 62

Fitzgibbon et al. (1996) give a 7-line MATLAB program Hal and Flusser (1998) identify several problems: r
the matrix C is singular the matrix S is also nearly singular (and is singular if all data points lie exactly on an ellipse) the computation of eigenvectors (2) is numerically unstable and can produce wrong results

34 / 62

Halr and Flusser (1998)
Hal and Flusser (1998) break up the design matrix D into r its quadratic and linear components D = (D1 | D2 ) where
x1 x1 y1 y1 . . .2 D1 = xi xi yi yi2 . . xn xn yn yn
x1 y. . . and D2 = xi yi 1 . . . xn yn 1

35 / 62

Next, the scatter matrix S is split as follows S= S1 ST 2 S2 S3 with S1 = DT DS2 = DT DS3 = DT D2 2
The constraint matrix C is expressed similarly 2 CC= where C1 = The vector of coefcients is split into a d a1 a= where a1 = b and a2 = e a2 c f

36 / 62

The optimal solution (2) is rewritten as S1 ST 2 S2 S3 a1 a2 = Ca1 a2
which is equivalent to the following two equations S1 a1 + S2 a2 = C1 a1 ST aFrom (4), solve for a2 a2 = S1 ST a3 and plug (5) into (3) to yield S1 S2 S1 ST a1 = C1 a3 (6)

37 / 62

(3) (4)

+ S3 a2 = 0

Because C1 is regular, (6) can be rewritten as C1 S1 S2 S1 ST a1 = awhere M = C1 S1 S2 S1 ST 3 matrix of size is the reduced scatter (7)
For the optimal t, it is enough to nd the appropriate eigenvector a1 of matrix M Halr and Flusser (1998) also provide MATLAB code

38 / 62

Further enhancements
For improved numerical stability, Paul and Zsombor-Murray (2004) suggest rst translating all data points then scaling isotropically to ensure 2 root mean square distance of the points to the origin, i.e., using the transformation matrix T s 0 sx T = 0 s sy 0 where s=

n 2 (xi x)2 + (yi y)2

39 / 62

Further Enhancements

After tting, Paul and Zsombor-Murray (2004) suggest transforming data back to the original position and scaling via K = TT KT where K is the conic matrix a b/2 d/2 e/2 K = b/2 c d/2 e/2 f Paul and Zsombor-Murray also provide MATLAB code

40 / 62

Harker et al. (2008) suggest a similar approach with scaling factor m yielding so-called mean-free coordinates m= 2n x)2 + (yi y )2

n i=1 (xi

They, however, do not provide MATLAB code

41 / 62

Implementation Notes
Eventual solution obtained ofcially from the IEEE PAMI paper by Fitzgibbon et al. (1999), but practically speaking from their web page:
http://research.microsoft.com/en-us/um/people/awf/ellipse/
containing, you guessed it, MATLAB code They also normalize the points by shifting to the centroid and scaling, i.e., 1/sx 0 x 1/sy y T= where s=
max (xi )min (xi ) 2 max (yi )min (yi ) 2

42 / 62

Fitzgibbon et al. (1999)
Whats really useful about Fitzgibbon et al.s MATLAB code is that it includes the steps to unnormalize the tted ellipse and to recover the ellipse parameters (h, k), (r , s), and Only problem is this line:
[evec x, eval x] = eig(inv(tmpD) * (tmpA - tmpB*tmpE));
which is the general solution to the eigensystem represented by M = C1 S1 S2 S1 ST 3 which happens to be non-symmetric I can see why MATLAB is popular.

43 / 62

Obtaining eigenvectors from non-symmetric matrix
My go-to book for this sort of thing states: We consider the problem of nding all eigenvectors of a nonsymmetric matrix as lying beyond the scope of this book. (Numerical Recipes in C, 2nd ed., Press et al., 1992) Whats involved?
balancing the matrix reducing to Hessenberg form applying the QR method for a non-singular square matrix solving for eigenvectors via inverse iteration
Which was really more than what I wanted to know; sometimes you just want to call a math library function

44 / 62

TNT and JAMA
Luckily, we have Google! It came up with the Template Numerical Toolkit, made freely available by NIST here: http://math.nist.gov/tnt/ This has both TNT, templated C++ arrays, as well as JAMA JAMA/C++ is a translation of the Java Matrix Library, developed by the Mathworks and NIST, into C++ classical matrix linear algebra, and includes:

LU QR SVD Cholesky Eigenvalue solver
Java Matrix Library? Did I read that correctly?

45 / 62

The last piece of the puzzle I needed was the C++ equivalent to the eig() MATLAB call: JAMA::Eigenvalue<double> eigen(TNT M); eigen.getV(TNT V); eigen.getRealEigenvalues(TNT E);

46 / 62

47 / 62
On-line NIR Spectrum Wearable Tracker

48 / 62

Vendor Point Grey Point Grey Point Grey Point Grey Point Grey Point Grey Point Grey UKA Optics QuickParts Various
Part No. FL2-03S2C FL2-03S2M FWB-EC-2PORT HF12.5HA-1B DF6HA-1B ACC-01-2012 FW-HUB-5PORT IR83M25.5 Custom Order
Desc. Scene Camera (c) Eye Cameras (b/w) 1394 ExpressCard Eye Camera Lenses Scene Camera Lens 1394b Cables 1394b Hub IR lter Plastic Mounts LED, misc. parts

Quant. 1 2

Unit Cost $695 $695 $80 $170 $190 $150 $139 $185 $476 $50 $4180
Courtesy of Lafayette Instruments, Inc.

49 / 62

Current Features
On-line, real-time processing IEEE 1394b (libraw1394, libdc1394)
DMA access to cameras Format-7 allows specication of hardware ROI
Video encoding/decoding (ffmpeg) Direct ellipse tting (Fitzgibbon et al., 1999)

50 / 62

Why NIR?

IR lter no IR lter

NIR LED

no NIR LED

51 / 62
NIR illumination: 940 nm LED, radiant power output 16 mW Simplies pupil detection (thresholding) Allows frame-differencing method as proposed by Ebisawa and Satoh (1993), popularized by C. Morimoto et al. (2000)

52 / 62

Calibration relies on image frame origin (we should really be using corneal reection) Using laptop screen to display calibration dots

53 / 62

Keep in mind importance of IR light Watch for spurious reections (mascara, spectacles, etc.) As the late Prof. Stark said, Calibrate, calibrate, calibrate!
technology has improved since Larrys time, but. still relevant if you want to measure slippage

54 / 62

To DIY or not to DIY?
If you want to build it yourself, use good cameras, e.g.,
IEEE 1394b Digital Cameras support 80+ fps with hardware ROI (format7) e.g., Flea2 cameras from Point Grey Research be careful about built-in IR lterb/w Flea2s dont have one, color Flea2s do choose correct lens FOV, e.g., we have
6mm (wide angle for scene) 12.5mm (medium range) 25mm (macro-like for close range)
IR lens lters seem difcult to obtain, but not impossible PGR cameras also come with powered pigtail that let you easily connect IR LEDs

55 / 62

Comments, Questions, Discussion?

56 / 62

Appendix

References

Selected References I
Babcock, J. S., & Pelz, J. B. (2004). Building a Lightweight Eyetracking Headgear. In ETRA 04: Proceedings of the 2004 Symposium on Eye Tracking Research & Applications (pp. 109114). San Antonio, TX. Borgefors, G. (1988). Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(6), 849865. Canny, J. (1986). A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8, 679-698.

57 / 62

Selected References II
Crane, H. D. (1994). The Purkinje Image Eyetracker, Image Stabilization, and Related Forms of Stimulus Manipulation. In D. H. Kelly (Ed.), Visual Science and Engineering: Models and Applications (p. 13-89). New York, NY: Marcel Dekker, Inc. Crow, F. C. (1984). Summed-area tables for texture mapping. In Siggraph 84: Proceedings of the 11th annual conference on computer graphics and interactive techniques (pp. 207212). New York, NY: ACM. Ebisawa, Y., & Satoh, S. (1993). Effectiveness of pupil area detection technique using two light sources and image difference method. In Proc. 15th IEEE Int. Conf. on Medicine and Biology Society (pp. 12681269).

58 / 62

Selected References III
Fitzgibbon, A. W., Pilu, M., & Fisher, R. B. (1996, January). Direct Least Square Fitting of Ellipses (Tech. Rep. No. DAIRP-794). Department of Articial Intelligence, University of Edinburgh. Fitzgibbon, A. W., Pilu, M., & Fisher, R. B. (1999, May). Direct Least Square Fitting of Ellipses. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 21(5), 476480. Hal, R., & Flusser, J. (1998, 9-13 February). Numerically r Stable Direct Least Squares Fitting of Ellipses. Harker, M., Paul, O., & Zsombor-Murray, P. (2008). Direct type-specic conic tting and eigenvalue bias correction. Image and Vision Computing, 26, 372381.

59 / 62

Selected References IV
Hennessey, C. (2008). Point-of-Gaze Estimation in Three Dimensions. Unpublished doctoral dissertation, University of British Columbia, Vancouver, BC, Canada. Lancaster, P., & Salkauskas, K. (1986). Curve and Surface Fitting: An Introduction. San Diego, CA: Academic Press. Li, D., Babcock, J., & Parkhurst, D. J. (2006). openEyes: A Low-Cost Head-Mounted Eye-Tracking Solution. In ETRA 06: Proceedings of the 2006 Symposium on Eye Tracking Research & Applications. San Diego, CA. Li, D., & Parkhurst, D. (2006, 4-5 September). Open-Source Software for Real-Time Visible-Spectrum Eye Tracking. In Conference on Communication by Gaze Interaction. Turin, Italy.

60 / 62

Selected References V
Menache, A. (2000). Understanding Motion Capture for Computer Animation and Video Games. San Diego, CA: Morgan Kaufmann, (Academic Press). Morimoto, C., Koons, D., Amir, A., & Flickner, M. (2000). Pupil detection and tracking using multiple light sources. Image and Vision Computing, 18(4), 331336. Morimoto, C. H., & Mimica, M. R. M. (2005, April). Eye Gaze Tracking Techniques for Interactive Applications. Computer Vision and Image Understanding, 98, 424. Paolini, M. (2006). Apple Pro Training Series: Shake 4. Berkeley, CA: Peachpit Press. Paul, O., & Zsombor-Murray, P. (2004, July). Direct and specic least-square tting of hyperboland ellipses. Journal of Electronic Imaging, 13(3), 492503.

61 / 62

Selected References VI
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical Recipes in C: The Art of Scientic Computing (2nd ed.). Cambridge: Cambridge University Press. Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137154.

62 / 62