START/ STOP

10) 11)

12) 13) 14) 15) 16)

Free Style Controller
This controller allows the scene to be shot in various angles, high to low and is also convenient when the Tripod is used. When you do not use this controller, attach the clip to the Grip Belt for convenience. This controller allows the camera to be used more conveniently by a left-handed user. When using the free style controller, plug it in as far as it goes.
Recording Start/Stop Button [REC] Zoom Lever [W/T] Photoshot Button [PHOTO SHOT]
Inserting a Button-type Battery
Before using the Remote Controller, insert the supplied buttontype battery.
1 While pressing the Stopper 1, pull out the

Battery Holder.

2 Insert the button-type battery with the imprint (i)

facing upward.

3 Insert the Battery Holder into the Remote

Controller.

When the button-type battery is weak, replace it with a new CR2025 battery. (A battery is normally expected to last about 1 year. However, it depends on operation frequency.) Make sure to match the poles correctly when inserting the battery....

WARNING

THE LITHIUM BATTERY IS A CRITICAL COMPONENT (TYPE NUMBER CR2025 MANUFACTURED BY PANASONIC). IT MUST NEVER BE SUBJECTED TO EXCESSIVE HEAT OR DISCHARGE. IT MUST THEREFORE ONLY BE FITTED IN EQUIPMENT DESIGNED SPECIFICALLY FOR ITS USE. REPLACEMENT BATTERIES MUST BE OF THE SAME TYPE AND MANUFACTURER. THEY MUST BE FITTED IN THE SAME MANNER AND LOCATION AS THE ORIGINAL BATTERY, WITH THE CORRECT POLARITY CONNECTIONS OBSERVED. DO NOT ATTEMPT TO RE-CHARGE THE OLD BATTERY OR RE-USE IT FOR ANY OTHER PURPOSE. IT SHOULD BE DISPOSED OF IN WASTE PRODUCTS DESTINED FOR BURIAL RATHER THAN INCINERATION....
Using the Remote Controller 1 Direct the Remote Controller to the Remote
Control Sensor of the Movie Camera and press an appropriate button.
Distance from the Movie Camera: Within approximately 5 metres. Angle: Within approximately 15o in the vertical and horizontal directions from the central axis. The above operating ranges are for indoor use. Outdoors or under strong light, the Movie Camera may not operate properly even within the above ranges. For other notes concerning this item, see page 48.

Power Supply

Using the AC Adaptor 1 Connect the DC Input Lead to the [DC/C.C. IN]
Socket on the Movie Camera.
2 Connect the DC Input Lead to the AC Adaptor. 3 Connect the AC Mains Lead to the AC Adaptor

and the AC mains socket.

The AC Mains Leads outlet plug cannot be pushed fully into the AC Adaptor socket. A gap will remain as shown 1. Before connecting or disconnecting the power supply, set the [VCR/OFF/CAMERA/M-CARD] Switch on the Movie Camera to [OFF] and make sure that the [POWER] Lamp is not lit.

Adjusting Brightness and Colour Level
When [LCD/EVF SETUP] on the [CAMERA FUNCTIONS] or [VCR FUNCTIONS] Main-Menu is selected, the following items are displayed. (l 16) LCD Brightness [LCD BRIGHTNESS] It adjusts the brightness of the image on the LCD screen. LCD Colour Level [LCD COLOUR] It adjusts the colour saturation of the image on the LCD screen. Brightness of the Viewfinder [EVF BRIGHTNESS] It adjusts the brightness of the image in the Viewfinder. To Adjust Press the [PUSH] Dial and select the item to be adjusted, and then turn the [PUSH] Dial to raise or lower the number of vertical bars in the Bar Indication. A larger number of vertical bars indicates stronger brightness or colour saturation. For other notes concerning this item, see page 57.

LCD/EVF SETUP

LCD BRIGHTNESS [-]||||----[+] LCD COLOUR [-]||||----[+] EVF BRIGHTNESS [-]||||----[+]

3 180x

PRESS MENU TO RETURN

Using the Menu Screen

SHUTTER / W B / IRIS/ MF / JOG

2, 3, 4, 5

To facilitate the selection of a desired function or setup, this Movie Camera displays various function setups on Menus.
1 Press the [MENU] Button.
The Menu corresponding to the Mode selected by using the [VCR/OFF/CAMERA/M-CARD] Switch 1 is displayed.
2 Turn the [PUSH] Dial to select a desired

Sub-Menu.

Turn the [PUSH] Dial to display the highlightened item.
3 Press the [PUSH] Dial to display the selected
4 Turn the [PUSH] Dial to select the item to be set. 5 Press the [PUSH] Dial to set the selected item to a

desired mode.

With each pressing of the [PUSH] Dial, the cursor [1] moves to the next mode. While a Menu is displayed, you cannot record or play back. Menus can be displayed during playback but not during recording. The above operations can be done using the [MENU] Button, [SET] Button and [ITEM] Button on the Remote Controller. (l 10)

To Exit the Menu Screen

Press the [MENU] Button again. About the Menu Mode Setting The setting selected on the Menu will be retained even when the Movie Camera is turned off. (But, the setups of [DIGITAL EFFECT] are not retained.) However, if the Battery or AC Adaptor is disconnected before turning off the Movie Camera, the selected setup may not be retained.
Menu operation flow is shown in this text by >>.
Image Selection in the Card Playback Mode (NV-DS65 only) (l 42)
An Image Selection Operation may be necessary during Menu Operations. In this case, carry out the following procedure.

Set [D.ZOOM] on the [CAMERA SETUP] Sub-Menu to [OFF]. As the magnification of digital zoom increases, the quality of image may deteriorate. Adjusting white balance manually cannot be set in the digital zoom range.

D.ZOOM

Image Stabilizer Function
If the Movie Camera might be shaken while recording, the camera shake in the image can be corrected. If the Movie Camera shakes too much, this function may not be able to stabilize images.
1 Press the [SIS] Button. The [[] Indication 1 appears.
To Cancel Image Stabilizer
Press the [SIS] Button. For other notes concerning this item, see page 49.

Fade In/Out Functions

COLOUR BACK LIGHT NIGHT VIEW LIGHT FADE
Fade In 1 Fade In brings out images and sounds gradually from a black screen at the beginning of a scene.
1 Keep pressing the [FADE] Button during the

Recording Pause Mode.

The image fades out gradually.
2 When the image is completely gone, press the
Recording Start/Stop Button to start recording.
3 Release the [FADE] Button about 3 seconds after

starting recording.

The image reappears gradually. Fade Out 2 Fade Out causes images and sounds to disappear gradually, leaving a black screen at the end of a scene.
1 Keep pressing the [FADE] Button while recording.
Recording Start/Stop Button to stop recording.
3 Release the [FADE] Button.
The still picture cannot fade in or fade out.

Cinema Function

This function is for recording in a cinema-like wide screen.
>> [CINEMA] >> [ON].
Black bars appear at the top and bottom of the screen. 1
To Cancel the Cinema Mode
Set [CAMERA FUNCTIONS] >> [CAMERA SETUP] >> [CINEMA] >> [OFF]. For other notes concerning this item, see page 49.

CINEMA

Colour Night View Function
The [C.NIGHT VIEW] Indication appears.
This function allows you to record colour images brightly in a dark place without using the Built-in Video Light.
1 Press the [COLOUR NIGHT VIEW] Button.
To Cancel the Colour Night View Function
Press the [COLOUR NIGHT VIEW] Button. For other notes concerning this item, see page 49.

C.NIGHT VIEW

Backlight Compensation Function
This prevents a recording subject from being recorded darker in backlight. (Backlight is the light that shines from behind a recording subject.)

1 Press the [BACK LIGHT] Button. The [] Indication 1 appears.
To Resume Normal Recording
Press the [BACK LIGHT] Button. When the [BACK LIGHT] Button is pressed, the entire screen becomes brighter. By operating the [VCR/OFF/CAMERA/M-CARD] Switch to [OFF], the Backlight Compensation Function is cancelled.
Recording in Special Situations
(Programme AE) You can select optimal automatic exposures under specific recording situations.
1 Press the [AUTO/MANUAL] Button to display the

[MNL] Indication.

2 Set [CAMERA FUNCTIONS] >> [CAMERA SETUP]
>> [PROG.AE] >> a desired mode.
The Indication for the selected mode appears. [5] Sports Mode 1 For recording scenes involving quick movements, such as sports scenes. [7] Portrait Mode 2 For bringing up people clearly from the background. [4] Low Light Mode 3 For recording a dark scene brighter. [] Spotlight Mode 4 For recording a subject under spotlight in a party, theatre, etc. [] Surf & Snow Mode 5 For recording in a glaring surrounding, such as skiing grounds, beaches, etc.
To Cancel the Programme AE Function
Set [CAMERA FUNCTIONS] >> [CAMERA SETUP] >> [PROG.AE] >> [OFF]. For other notes concerning this item, see page 50.
Wind Noise Reduction Function
Set [CAMERA FUNCTIONS] >> [RECORDING SETUP] >> [WIND CUT] >> [ON].
The [WIND CUT] Indication 1 appears.
This function reduces the sound of a wind hitting the microphone for recording.

WIND CUT

Recording in Natural Colours
(White Balance) Depending on the scene or light condition, the Automatic White Balance Adjustment Mode may not be able to bring out natural colours. (l 28, 59) In such a case, the white balance can be adjusted manually.

2 Press the [PUSH] Dial.

The [AWB] Indication appears.
3 Turn the [PUSH] Dial to set a desired White

Balance Mode.

Automatic White Balance Adjustment [AWB] The white balalnce setting that was previously set manually [1] Outdoor Mode [z] Indoor Mode (recording under incandescent lamp) [{]
To Resume Automatic Adjustment
Turn the [PUSH] Dial until the [AWB] Indication appears. Or, press the [AUTO/MANUAL] Button to display the [AUTO] Indication.

Recording from a Cassette 1 Set the [VCR/OFF/CAMERA/M-CARD] Switch to [VCR]. 2 Start playback and set the Movie Camera to Still Playback Mode at the scene you wish to record, and press the [PHOTO SHOT] Button.
Saving a picture takes some time.
1 Remaining Number of Card Photoshot pictures 2 Picture Size

COMPLETED

Maximum number of images recordable on a supplied Card (8MB) [640k480]: approximately 95 pictures This figure varies depending on the subject being photographed.
To Use the Backlight Compensation Function
Press the [BACK LIGHT] Button 6 in the Card Recording Mode. (l 25) For other notes concerning this item, see page 52.

Playing Back a Card

It plays back images recorded on a Card.

[M-CARD].

2 Press the [M-PLAY] Button.
The still pictures recorded on the Card are displayed in Multi-Picture Mode. 1
2 File Number (If the colour of File Number is white, the picture
size is [640k480], if green, the picture size is [320k240].)
3 Turn the [PUSH] Dial and select a desired picture.
The selected picture is underlined. When 7 or more pictures have been recorded, turn the [PUSH] Dial to display the next picture.

4 Press the [PUSH] Dial.

The selected picture is shown on the entire screen.
3 File Number 4 DCF Number (Folder-File Number) 5 Picture Size (l 41) 6 Number of prints set by DPOF (l 45)
To Change the Display on the Screen
Set [CARD SETUP] >> [DISPLAY] or [DCF NO.] >> [OFF]. DISPLAY: Select [OFF] to cancel all displays on the screen. DCF NO.: Select [OFF] to cancel DCF Number on the screen.
Playing Back Another Picture
Turn the [PUSH] Dial and select [BACK] to play back the previous Picture or [NEXT] to play back the next Picture 7. Then press the [PUSH] Dial.

DELETE:M-DEL

Returning to the Multi-Picture Mode
Turn the [PUSH] Dial and select [MULTI] 7, then press the [PUSH] Dial.

1 100-D-000

PICTURE
Returning to the Card Recording Mode
Press the [M-STOP] Button in Multi-Picture Mode. For other notes concerning this item, see page 52.

"BACK MULTI NEXT!

Recording with a Photo Title

1, 2, 3, 4

Card Images can be recorded as the Photo Title in CAMERA Mode.
1 Set [CAMERA FUNCTIONS] >> [PHOTO TITLE] >>
[MODE] >> [WIPE] or [MIX].
2 Set [CAMERA FUNCTIONS] >> [PHOTO TITLE] >>

[TITLE] >> [SET].

A list of Photo Titles is displayed. Turn the [PUSH] Dial to select a desired title. The selected Photo Title is underlined.

To Change the Counter Display Mode
By changing [C.DISPLAY] on the [DISPLAY SETUP] Sub-Menu, you can change the Counter Display Mode to Tape Counter Indication [COUNTER] (0:00.00), Memory Counter Indication [MEMORY] (M0:00.00), and Time Code Indication [TIMECODE] (0h00m00s00f). (l 17) By repeatedly pressing the [COUNTER] Button on the Remote Controller, you can change the Counter Display Mode.
To Display Date/Time Indication
To display Date/Time Indication, set the appropriate Date/Time in [DATE/TIME] on the [DISPLAY SETUP] Sub-Menu. (l 17) You can also press the [DATE/TIME] Button of the Remote Controller repeatedly to display or change the Date/Time Indication.
To Change the Display Mode
By changing [DISPLAY] on the [DISPLAY SETUP] Sub-Menu, you can change the Display Mode to All Function Display [ALL], Partial Display [PARTIAL], and Minimum Display [OFF].
CARD ERROR: This Card is not compatible with this Movie Camera. COPY INHIBITED: Because the medium is protected by a copy guard, images cannot be recorded correctly. CAN NOT CHANGE MODE: The Mode of the Movie Camera can not be changed in the PC connection Mode. DISCONNECT DV CABLE: If the Movie Camera is in Photo Title (CAMERA mode) and the DV Interface Cable is connected, DISCONNECT DV CABLE appears. In this case, remove the DV Interface Cable from the Movie Camera. DISCONNECT USB CABLE: If you are attempting to record from a Cassette by pressing the PHOTO SHOT Button when the USB Cable is connected, DISCONNECT USB CABLE appears. In this case, remove the USB Cable from the Movie Camera. PICTURE (Blink during 3 seconds): When making DV connection at the time of M-CARD mode. WEBCAM (Blink during 3 seconds): When making DV connection at the time of Web Camera mode. STORAGE (Blink during 3 seconds): When making DV connection at the time of USB Device Drive/SD Drive.

Notes and Hints

Selecting Remote Controller Modes
When 2 Movie Cameras are used simultaneously, they can be operated individually by selecting different Remote Controller Modes. If the Remote Controller Mode of the Movie Camera and that of the Remote Controller do not match, [REMOTE] Indication is displayed. Setup on the Movie Camera: Set [REMOTE] on the [OTHER FUNCTIONS] Sub-Menu to the desired Remote Controller Mode. (l 16) Setup on the Remote Controller: [VCR1]: Press the [D] Button and [] Button simultaneously. [VCR2]: Press the [E] Button and [] Button simultaneously. When the battery in the Remote Controller is replaced, the mode is automatically reset to [VCR1] Mode.

Concerning the Power Supply
When the Movie Camera is used for a long time, the Camera body becomes warm, but this is not a malfunction. If the [CHARGE] Lamp does not light up although the Battery is attached to AC Adaptor, detach the Battery and reattach it. Battery CGP-D110 and CGP-D105 cannot be charged with this AC Adaptor.
Inserting/Removing the Cassette
If a new Cassette is inserted, rewind to the beginning of the tape before starting recording. When inserting the cassette, make sure it faces in the right direction and then push it in until it stops.

Alarm Sounds

When [BEEP SOUND] on the [OTHER FUNCTIONS] Sub-Menu on the [CAMERA FUNCTIONS] Main-Menu is set to [ON], confirmation/alarm beeps are issued as follows. 1 Beep When you start recording. When you switch the [VCR/OFF/CAMERA/ M-CARD] Switch from [OFF] to Camera Mode or Card Recording Mode. 2 Beeps When you pause recording. 4 Beeps If you perform a wrong operation before or during recording.

Recording

Before turning on the power, remove the Lens Cap. If the Lens Cap is still attached when the power is turned on, Automatic White Balance Adjustment (l 59) may not function correctly.

Zoom In/Out Functions

When you are zooming on a faraway subject, a sharper focus is achieved if the recording subject is 1.2 metres or more away from the Movie Camera.

Recording Check

For Recording Check, the recording speed mode (SP/LP) must be the same as the mode used for the recording. If different, playback images will be distorted.
Image Stabilizer Function
The Image Stabilizer Function does not operate in a dimly lit place. In this case, the [[] Indication flashes. Under fluorescent lighting, image brightness may change or colours may not look natural. After-images may appear. When a tripod is used, it is recommended that you turn the Image Stabilizer off.
The subject should be within 1.5 metres of the Movie Camera. Using the Video Light reduces battery time. Set the Video Light to off when not in use. Do not look directly into the light. If the Video Light is used with a Conversion lens attached, a slight vignetting (darkening around edges) effect occurs on the screen. Using the Colour Night View at the same time will make conditions even brighter. The Video Light enables to lighten simply the images taken in a dimly-lit place. However we recommend using it in a light place in order to obtain the high quality images.

Cinema Function

If you play back tape recorded in Cinema Mode on a wide-screen (16:9) format TV, playback size is automatically adjusted to fit the TV screen format. Please refer to the TVs operating instructions for details. When images are displayed on a TV screen, the Date/Time Indication may be erased in some cases.
Colour Night View Function
In a bright place, such as the outdoors, the recorded picture may become whitish. The recorded picture is presented in a time-lapse-like manner because of the slow shutter speed of 1/2 second. It may take time to bring the subject into focus. It is recommended to adjust the Focus manually (l 29). White Balance cannot be changed. The shutter speed cannot be adjusted. When this function is set, the Sports Mode, Portrait Mode, Low Light Mode cannot be set simultaneously. Also when the DigitalEffect Mode is set, this function is cancelled. When recording with the Colour Night View Function, it is recommended to use the tripod.

Photoshot

Photoshot Recording results in slightly inferior image quality.
Progressive Photoshot Function
The still pictures are recorded at a slightly rewound point from where you press the [PHOTO SHOT] Button. When you change to the Still Picture Mode, you will hear a click. This is the sound of the iris closing and not an indication of malfunction. If you record still pictures in Progressive Photoshot Mode when your Movie Camera is set to Programme AE (l 26), the brightness of the images may change. When you set the following functions, [PROGRESSIVE] will be set to [OFF]. Digital Effect Modes in [EFFECT] (l 30) Digital Zoom [D.ZOOM] (l 23) Image Stabilizer Function [SIS] (l 24) When [PROGRESSIVE] is set to [ON], the shutter speed cannot be adjusted to 1/750 s or faster. (l 28)

Programme AE

If any of the Programme AE Modes is selected, you cannot adjust the shutter speed (l 28) or iris (l 29). You cannot use Sports Mode, Portrait Mode or Low-Light Mode with Gain-up Mode [GAIN UP] and Colour Night View Function. Sports Mode During normal playback, the image movement may not look smooth. Because the colour and brightness of the playback image may change, avoid recording under fluorescent light, mercury light, or sodium light. If you record a subject illuminated with strong light or a highly reflective subject, vertical lines of light may appear. If the light is insufficient, the [5] Indication flashes. If this mode is used for indoor recording, playback images may flicker. Sports Mode/Portrait Mode If you record a still picture in Progressive Photoshot Mode, the brightness and hue of the recorded still picture may become unstable. Low Light Mode Extremely dark scenes may not be able to be made brighter to a satisfactory degree. Spotlight Mode With this mode, recorded images may turn out to be extremely dark. If the recording subject is extremely bright, its recorded image may turn out to be whitish. Surf & Snow Mode If the recording subject is extremely bright, its recorded image may turn out to be whitish.

Condensation

If the Condensation Indication flashes after the Movie Camera is turned on, condensation has formed inside the Movie Camera. In this case, the Movie Camera power is turned off automatically after about 1 minute. Follow the steps below:

1 Take out the Cassette.

None of the other functions operate. Depending on the amount of condensation, removing the Cassette may even become difficult. If this happens, wait for 2 to 3 hours before taking the Cassette out.
2 Open the Cassette Compartment
Cover and wait for 2 to 3 hours.
The number of hours to wait depends on the amount of condensation and ambient temperature.
to 3 hours later, turn on the Movie
Camera power and check to see if the Condensation Indication is displayed.
Even if the Condensation Indication is not displayed, as a precaution, please wait for another hour before using the Movie Camera.
Watch for Condensation even before the Condensation Indication is displayed. Because condensation takes place gradually, the Condensation Indication may not be displayed during the first 10 to 15 minutes of condensation formation inside the Movie Camera. In extremely cold places, condensation may freeze and form frost. In this case, the frost melts first, thus forming condensation, and then it takes another 2 to 3 hours to eliminate the condensation. When the Lens is Fogged: Set the [VCR/OFF/CAMERA/M-CARD] Switch to [OFF] and leave the Movie Camera in this condition for about 1 hour. When the lens temperature becomes close to the ambient temperature, the fog disappears naturally.
Video Head Clogging and Care
If the heads (the parts that make contact with tape) are dirty, mosaic-pattern noise may appear on the playback image, or the screen as a whole becomes black. If the heads are extremely dirty, recording performance deteriorates, and, in the worst case, the Movie Camera cannot record at all. Causes of Dirty Heads Large quantity of dust in the air High-temperature and high-humidity environment Damaged tape Long operating hours Using Mini-DV Format Digital Video Head Cleaner
Notes: Do not rewind every time you use the Head Cleaner. Rewind only when the tape reaches the end, and then use it again from the beginning in the same manner as before. If the heads become dirty soon after cleaning, the tape may be damaged. In this case, immediately stop using that Cassette. Do not clean the heads excessively. (Excessive cleaning may cause excessive wear of the heads. If the heads are worn, images cannot be played back even after the heads are cleaned.) If the dirty heads cannot be cleaned with the Head Cleaner, the Movie Camera needs to be cleaned at a service centre. Please consult a dealer. Video Head Cleaners can be purchased from service centres. Head cleaning due to dirty heads is not considered a malfunction of the product. It is not covered by the warranty. Periodical Check-up To maintain the highest image quality, we recommend replacement of worn parts, such as heads, etc., after approximately 1000 hours of use. (This, however, depends on operating conditions, such as temperature, humidity, dust, etc.)

Optimal Use of the Battery
Battery Characteristics This Battery is a rechargeable lithium ion battery. Its ability to generate power is based upon the chemical reaction that takes place inside it. This reaction is susceptible to the surrounding temperature and humidity, and, if the temperature is too high or too low, the operating time of the Battery becomes shorter. If the Battery is used in an extremely cold environment, the Battery may operate only for about 5 minutes. If the Battery becomes extremely hot, a protective function may operate, and the Battery may become unusable for a while. Be Sure to Detach the Battery after Use Be sure to detach the Battery from the Movie Camera. (If it is left attached to the Movie Camera, a minute amount of current is consumed even when the Movie Camera power is off.) If the Battery is left attached to the Movie Camera for a long time, over discharge takes place. The Battery may become unusable after it is charged.
1 Insert the Head Cleaner into the Movie
Camera in the same manner as a Video Cassette. 2 Press the [1] Button, and approximately 20 seconds later, press the [] Button. (Do not rewind the tape.) 3 Take out the Head Cleaner. Insert a Video Cassette and start recording. Then, play the tape back to check the recorded image. 4 If the image is not still clear, repeat Steps 1 to 3. (Do not use the Head Cleaner for 3 times or more consecutively.)
Disposing of an Unusable Battery The Battery has a limited life. Do not throw the Battery into fire because it may cause an explosion. Always Keep the Terminals of the Battery Clean Prevent the terminals from getting clogged with dirt, dust, or other substances. If you drop the Battery accidentally, check to see if the Battery body and terminals are deformed. Attaching a deformed Battery to the Movie Camera or AC Adaptor may damage the Movie Camera or AC Adaptor.

Cautions for Storage

Before storing the Movie Camera, take the Cassette out and detach the Battery. Store all the components in a dry place with a relatively stable temperature. (Recommended Temperature: 15 to 25oC, Recommended Humidity: 40 to 60%) Movie Camera Wrap it with a soft cloth to prevent dust from getting into the Camera. Do not leave the Movie Camera in places that expose it to high temperature. Battery Extremely high temperatures or low temperatures will shorten the life of the Battery. If the Battery is kept in smoky or dusty places, the terminal may rust and cause malfunctions. Do not allow the Battery terminals to come in contact with metal objects (such as necklaces, hairpins, etc.). This can result in a short circuit or heat generation and, if you touch the Battery in this condition, you may be badly burned. Store the Battery in a completely discharged state. To store the Battery for a long period of time, we recommend you charge it once every year and store it again after you completely use up the charged capacity.

Before Requesting Repair (Problems and Solutions)
Power 1: The Movie Camera power cannot be turned on. 1: Is the power source connected correctly? (l 12) 2: The Movie Camera power is turned off automatically. 2: If you leave the Movie Camera in Recording Pause Mode for more than 6 minutes, the power is turned off automatically to protect the tape and to save the Battery power. (l 20) 3: The Movie Camera power does not stay on long enough. 3-1: Is the Battery low? Charge the Battery or attach a fully charged Battery. (l 12) 3-2: Has condensation occurred? Wait until the Condensation Indication disappears. (l 54) Battery 1: Battery runs down quickly. 1-1: Is the Battery fully charged? Charge it with the AC Adaptor. (l 12) 1-2: Are you using the Battery in an extremely cold place? In cold places, the operating time of the Battery becomes shorter. (l 55) 1-3: Has the Battery worn out? If the operating time is still too short even after the Battery is fully charged, the Battery has worn out. 2: The Battery cannot be charged. 2: If the DC Input Lead is connected to the AC Adaptor, charging cannot be performed. Disconnect the DC Input Lead. Normal Recording 1: Recording does not start although power is supplied to the Movie Camera and the Cassette is correctly inserted. 1-1: Is the accidental erasure prevention slider on the Cassette open? If it is open (set to [SAVE]), recording cannot be performed. (l 14) 1-2: Is the tape wound to the end? Insert a new Cassette. (l 14) 1-3: Is the [VCR/OFF/CAMERA/M-CARD] Switch set to [CAMERA]? If it is set to the other position, the recording function cannot be used. (l 20) 1-4: Has condensation occurred? Wait until the Condensation Indication disappears. (l 54)

Built-in Video Light

A light emitting diode (LED) is used for the Built-in Video Light.
Other Recording 1: Auto Focus Function does not work. 1-1: Is Manual Focus Mode selected? If Auto Focus Mode is selected, focus is automatically adjusted. (l 29) 1-2: There are some recording subjects and recording surroundings for which the Auto Focus Function does not operate correctly. In this case, use the Manual Focus Mode to adjust the focus. (l 59) Editing 1: Audio dubbing cannot be performed. 1-1: Is the accidental erasure prevention slider on the Cassette open? If it is open (set to [SAVE]), recording cannot be performed. (l 14) 1-2: Are you attempting to edit a tape portion that was recorded in LP Mode? LP Mode does not allow audio dubbing to operate. (l 21) Indications 1: The Time Code becomes inaccurate. 1: The Time Code Indication counter may not be constant in the reverse direction in Slow Motion Playback Mode, but this is not a malfunction. 2: The Remaining Tape Time Indication disappears. 2: When you record still pictures in Photoshot Mode, the Remaining Tape Time Indication may disappear temporarily. However, it reappears when recording normally. 3: The Remaining Tape Time Indication does not match the actual remaining tape time. 3-1: If scenes of less than 15 seconds are continuously recorded, the remaining tape time cannot be displayed correctly. 3-2: In some cases, the Remaining Tape Time Indication may show remaining tape time that is 2 to 3 minutes shorter than the actual remaining tape time. Playback (Images) 1: Images cannot be played back even when the [1] Button is pressed. 1: Is the [VCR/OFF/CAMERA/M-CARD] Switch set to [VCR]? If it is set to the other position, the playback function cannot be used. (l 32) 2: Mosaic-pattern noise appears on images during Cue, Review or Slow Motion Playback. 2: This phenomenon is characteristic of digital video systems. It is not a malfunction. 3: Although the Movie Camera is correctly connected to a TV, playback images cannot be seen.

Dimensions: Weight:

Approx. 66 (W)k87 (H)k123 (D) mm (NV-DS60) Approx. 440 g (without Battery and DV cassette) Approx. 520 g (with CGR-D08R and DVM60) (NV-DS65) Approx. 440 g (without Battery and DV cassette) Approx. 520 g (with CGR-D08R and DVM60) 0oCj40oC 10%j80%
Operating Temperature: Operating Humidity:
Card Memory Functions (NV-DS65 only) Recording Media: MultiMediaCard, SD Memory Card Image Compression: JPEG
AC Adaptor Power Source: Power Consumption: DC Output:
Information for your safety AC 100240 V, 50/60 Hz 20 W DC 7.9 V, 9 W (Movie Camera Operation) DC 8.4 V, 1.2 A (Battery Charging)
Dimensions: 70 (W)k45 (H)k116 (D) mm Weight: Approx. 165 g Weight and dimensions are approximate values. Specifications may change without prior notice.
AC Adaptor.. 12, 54 Audio Dubbing.. 37 Auto Focus.. 59 Automatic White Balance Adjustment. 59 Menu Screen.. 16 Microphone.. 23 Mirror Mode.. 30 Mix Mode... 30 Mosaic Mode.. 30
Backlight Compensation.. 25 Black & White Mode.. 36 Blank Search... 34
Photo Title... 43 Photoshot...22, 41 Picture in Picture Mode.30 Playback Digital Effects.36 Playback Zoom.. 36 Portrait Mode.. 26 Programme AE.. 26
Card Photoshot.. 41 Card Playback.. 42 Charging the Battery.. 12 Cinema Mode.. 25 Colour Night View Function.. 25 Condensation... 54 Continuous Photoshot.. 22 Cue Playback... 33
Recording Check... 20 Recording Pause Mode.20 Recording Speed.. 21 Remaining Tape Time..46 Remote Controller.. 10
Date/Time Indication.. 47 Date/Time Setting.. 19 DCF... 52 Digital Effects... 30 Digital Still Picture.. 22 Digital Zoom... 23 DPOF Setting... 45 Dubbing... 38, 39
Self-Recording.. 21 Sepia Mode.. 36 Shutter Speed.. 28 Slim Mode... 30 Slow Motion Playback..33 Sound Volume.. 32 SP/LP Mode... 21 Sports Mode... 26 Spotlight Mode.. 26 Still Advance Playback.. 34 Stretch Mode.. 30 Strobe Mode... 30 Surf & Snow Mode.. 26
Erasure Prevention Slider.. 14
F Number.. 29 Fade... 24 Formatting... 52 Free Style Controller.. 10
Time Code... 59 Trailing Effect Mode... 30

Gain-up Mode.. 30

Headphones.. 51

USB Connection Kit... 45

Iris... 29
Variable Speed Search.33 Video Head Clogging.. 55 Viewfinder.. 15, 57

A Single-Chip FPGA Implementation of Real-time Adaptive Background Model
Ko Appiah Andrew Hunter Department of Computing and Informatics Faculty of Technology, University of Lincoln Lincoln, LN6 7TS, UK {kappiah, ahunter}@lincoln.ac.uk Abstract
This paper demonstrates the use of a single-chip FPGA for the extraction of highly accurate background models in real-time. The models are based on 24-bit RGB values and 8-bit grayscale intensity values. Three background models are presented, all using a camcorder, single FPGA chip, four blocks of RAM and a display unit. The architectures have been implemented and tested using a Panasonic NVDS60B digital video camera connected to a Celoxica RC300 Prototyping Platform with a Xilinx Virtex II XC2v6000 FPGA and 4 banks of onboard RAM. The novel FPGA architecture presented has the advantages of minimizing latency and the movement of large datasets, by conducting time critical processes on BlockRAM. The systems operate at clock rates ranging from 57MHz to 65MHz and are capable of performing pre-processing functions like temporal low-pass ltering on standard frame size of 640X480 pixels at up to 210 frames per second.
objects at rates much faster than human observers, thus the need to identify imaging functions that allow the computer to behave like a trained human operator[17]. Field Programmable Gate Array, a technology which has recently been made available to researchers, is used as an alternative platform for speedup in computer vision and digital image processing. The potential uses of FPGAs in areas like medical image processing, computational uid dynamics, target recognition, embedded vision systems, gesture recognition and automotive infotainment have been demonstrated in [2] [4] [10] [11] [13]. Digital Image processing or computer vision algorithms can be broken down into three major stages [8]: early processing, implemented by local pixel-level functions; intermediate processing, which includes segmentation, motion estimation and feature extraction; and late processing, including interpretation and using statistical and articial intelligence algorithms. Typically algorithm sophistication is concentrated in the later stages, but processing demands dominate in the early stages. Vanderlei et al [2] used a simplied method in converting Red, Green, and Blue (RGB) values into Hue, Saturation, and Intensity (HSI) components, for extracting their binary region of interest (ROI) for their RAM-based neural network. The simplication was necessary to achieve feasible FPGA implementations. The choice of T1 and T2 (inferior and superior thresholds respectively) as shown in equation 1 is not very clear in their implementation. Elham Ashari [1] used an adaptive thresholding method to separate the foreground and background pixels in gray level images. Hardware implementation results in terms of visual performance, speed, and area consumption of the implementation is given, yet the algorithm is application dependent and requires some training period for perfect segmentation. Jim et al [16] used an experimental colour representation for ltering common colours found in road signs. They used RGB values as opposed to HSV to avoid computationally ex-

1. Introduction

We demonstrate the use of Field Programmable Gate Array (FPGA) for the extraction of accurate background models under variable lighting conditions in real-time. This implementation has a number of uses in embedded systems as well as automated visual surveillance systems. Real-time image processing is difcult to achieve on a serial processor, due to the movement of large data sets and complex operations that need to be performed on the image [9]. Advances in semiconductor technology makes it possible to design complete embedded System-on-Chip (SoC) by combining sensor, signal processing and memory onto a single substrate [13]. New embedded vision systems have emerged as a result of this level of integration and are estimated to increase in the coming years. The aim of computer vision systems is to scan objects and make judgements on those
pensive conversions as outputs of many cameras are in RGB mode. f (x, y) = T1 f (i, j) T2 otherwise (1)
From the above analysis it becomes clear that most object and activity recognition systems require some form of object extraction at their initial stages, a requirement which has not easily been achieved on recongurable platforms such as FPGA. Background subtraction is the commonly used method for extracting moving targets in a scene. Even though it is robust and less computationally expensive, it requires the maintenance of a background model. The maintained model can lead to accumulated errors if not updated over time. This paper presents three different real-time adaptive background models, which have successfully been implemented on FPGA with minimal use of external memory. The paper is organized as follows. Section 2 briey describe some background modelling algorithms used for segmenting moving objects. Their advantages and disadvantages for FPGA implementation are also given. Section 3 gives details of the system showing how the various components and peripherals are connected. This is followed by our background modelling approach in section 4 with details on how it is implemented on FPGA. Section 4 also gives results and analysis of each implementation. Finally, we present a summary of our work and point out future directions.

have an intensity value which has never occurred at that position (a side-effect of the smoothing used in these techniques). Other techniques rely heavily on the assumption that the most frequent intensity value during the training period represents the background. This assumption may well be false, causing the output to have a large error level.

Grimsons Algorithm

Grimson et al [15] introduced a multimodal approach, modelling the values of each pixel as a mixture of Gaussians. The background is modelled with the most persistent intensity values. The algorithm has two variants, colour and gray-scale: in this paper, we concentrate on the gray-scale version. The probability of observing the current pixel value is given as:

P (Xt ) =

i,t (Xt , i,t , i,t )
Where i,t , i,t and i,t are the respective mean, standard deviation and weight parameters of the ith Gaussians component of pixel X at time t. is a Gaussian probability density function (Xt , i,t , i,t ) = i,t e

(Xt i,t )2 i,t

2. Previous Work
The rst stage in processing for many video applications is the segmentation of (usually) moving objects with signicant difference in colour and shape from the background. Where the camera is stationary, a natural approach is to model the background and detect foreground object by differencing the current frame with the background. A wide and increasing variety of techniques for background modelling have been described; a good comparison is given by Gutchess et al [6]. The most popular method is unimodal background modelling, in which a single value is used to represent a pixel, which has been widely used due to its relatively low computational cost and memory requirements [7] [18]. This technique gives a poor results when used in modelling non-stationary background scenarios like waving trees, rain and snow. A more powerful alternative is to use a multimodal background representation, the most common variant of which is a mixture of Gaussians [5] [15]. However, the computational demands make such techniques unpopular for real-time purposes; there are also disadvantages in multimodal techniques [5] [15] [18] including the blending effect, which causes a pixel to
A new pixel value is generally represented by one of the major components of the mixture model and used to update the model. For every new pixel value, Xt , a check is conducted to match it with one of the K Gaussian distributions. A match is found when Xt is within 2.5 standard deviation of a distribution. If none of the K distributions match Xt , the least weighed distribution is replaced with a new distribution having Xt as mean, high variance and very low weight. The update equations are as follows: wi,t = wi,t1 + (mi,t wi,t1 ) where is the learning rate and mi,t =

if there is a match otherwise + (Xt t ) (Xt t )

(5) (6) (7)

t = t1 (Xt t ) = (1
Only the matched distribution will have its mean and variance updated, all others remain unchanged. For = (Xt |t , t ) (8) The rst B distributions (ordered by k ) are used as a model of the background, where

B = argb min(

k > T ).
The threshold T is a measure of the minimum portion of the data that should be accounted for by the background.
Temporal Low-Pass ltering Algorithm
Aleksej [12] introduced a method to avoid false alarms due to illumination, using a temporal lter to update the background model, while a global threshold value T was used to extract target regions. The background update he used is of the form B(k, l, n) = c (p c) B(k, l, n 1) + I(k, l, n) p p
where c is the number of consecutive frames during which a change is observed and is reset to zero each time the new value becomes part of the background; p is the adaptation time or insertion delay constant. The moving target is extracted on a pixel level with the following relation: 1 |I(k, l, n) B(k, l, n)| > L 0 otherwise (10) where f(k,l,n), B(k,l,n) and I(k,l,n) are the respective foreground, background and the grayscale intensity value of pixel (k,l) for the nth frame and L is the global threshold value. The low-pass ltering algorithm is attractive for two reasons. First it is very simple and hence updating the background information is computationally cheap and memory consumption is minimal. The use of single global threshold value as well as a single mode makes it unattractive for scenes with varying lighting intensities. In contrast, Grimsons algorithm [15] is robust to outdoor environments where lighting intensity can suddenly change, and it handles multimodal backgrounds such as moving foliage (cyclical motion) without manual initialisation. Unfortunately, the use of oating-point numbers in all its update parameters makes it computationally expensive, and unsuitable for hardware implementation [1]. f (k, l, n) =
of all available memory banks. Two banks hold camera data, whilst the other two hold background update information. A control unit controls the RAM bank the camera data is written to and which bank the VGA reads from. The two banks reserved for camera data are swapped after every full frame is acquired. Similarly the RAM bank for the display unit is swapped. The architecture is such that, when data from the camera is being written to bank 1 the processing unit reads captured camera data from bank 2 and processes it with stored background data from bank 3. Concurrently, background data is written to bank 4 after update. Figure 1 shows this architecture in detail, which is the same for all three implementations.

foreground write image bank #1 read image bank #2

A B C D E F G H

SELECTED ON-LINE

processor

bank #3 read background data bank #4 write background data background model
Figure 1. The RAM switching architecture

Our Approach

Overview of Setup
The hardware system we present here is composed of a Panasonic NV-DS60B digital video camera, a display unit and an FPGA prototyping board. The camera is interlaced and runs at 50Hz, thus effectively transmitting at 25Hz in 24-bit RGB values of size 768567 in PAL format. The output of the camera is connected directly to the RC300 prototyping board via the S-video input. The RC300 board is packaged with Xilinx Virtex II XC2v6000, 4 banks of ZBT SRAM totalling 32MBytes (thus 4 banks x 2M x 36bits) and two DVI output ports. The outputs of the processed image and the background are displayed on different VGAs for visual inspection. This constrains the available memory resource. The inability to read from and write to a single bank of RAM in a single clock cycle calls for the use
We present here a novel hybrid background modelling algorithm that combines the attractive features of Grimsons algorithm [15] and the temporal lowpass ltering [12], with appropriate modications to improve segmentation of the foreground image, and to allow an efcient implementation on a recongurable hardware platform such as Field Programmable Gate Array (FPGA). Following Grimson [15], we maintain a number of clusters, each with weight wk , where 1 k K, for K clusters. Rather than modelling a Gaussian distribution, we maintain a model with a central value, ck of 11-bits (8 bits integer part and 3 bits fractional part). We use an implied global range, [ck 15, ck + 15], rather than explicitly modelling a range for each pixel based on its variance as in [15]. The weights and central values of all the clusters are initialised to 0. A pixel X = I(i, j) (where X is 11-bit xedpoint) from an image I is said to match a cluster, k, if X ck 15 and X ck + 15. The highest weight matching cluster is updated, if and only if its weight after the update will not exceed the maximum allowed value (i.e. wk 64, given the data width of the weight as 6 bits). The update for the weight is as
follows: wk,t = for the matching cluster otherwise (11) The central values of all the clusters are also updated as follows: ck,t,i,j =
ck,t1,i,j ck,t1,i,j wk,t64 wk,t1

to operate correctly. For these reasons we demonstrate how we can model background scenes in real time on FPGA using a unimodal background. The algorithm used is similar to that described above with K = 1 and no associated weight. The central value is updated on each processed frame and hence the background data can suffer from high deviation.
matching cluster otherwise (12) Where ck,t,i,j is the central value for cluster k at time t for pixel (i, j) If no matching cluster is found, then the least weighted clusters central value, cK is replaced with X; its weight is reset to zero. The way we construct and maintain clusters make our approach gradually incorporate new background objects. This is similar to [12] and hence the insertion delay is 23 = 8 frames in our case. The K distributions are ordered by weight, with the most likely background distribution on top. Similar to [15], the rst B clusters are chosen as the background model, where

+ 1 Xi,j 8

4.1.1. FPGA System Design.

i > T ).

The threshold T is a measure of the minimum portion of the data that should be accounted for by the background. The choice of T is very important, as a small T usually models a unimodal background whiles a higher T models a multi-modal background. We classify a pixel as foreground pixel based on the following two conditions: 1. If the intensity value of the pixel matches none of the K clusters. 2. If the intensity value is assigned to the same cluster for two successive frames, and the intensity values X(t) and X(t 1) are both outside the 40% mid-range [ck 6, ck + 6]. The second condition makes it possible to detect targets with low contrast against the background, while maintaining the concept of multimodal backgrounds. A typical example is a moving object with grayscale intensity close to that of the background, which would be classied as background in [15]. This requires the maintenance of an extra frame, with values representing the recently processed background intensities.
Gray-Scale Background Modelling
It is not always necessary to maintain a multimodal background. Many vision systems for handheld devices may have limited memory and may not require the maintenance of multi-modal backgrounds
The implementation of the unimodal grayscale background modelling algorithm is made up of six distinct processes all running in parallel. The rst of these is the image capture block which acquires pixels in RGB format, at camera rate of 25Hz and converts it to 8-bit grayscale in a single cycle. This block has iterative mechanisms to acquire all pixels from the camera. Thus after acquiring pixel (i, j) this block iterates several times for pixel (i + 1, j). This iteration is necessary and possible as the clock rate of the design is much higher than the camera transmission rate. The successfully acquired pixel is sent to the memory write block. This block takes two cycles to write the camera data to external RAM. The rst cycle is used to enable memory write for either RAM 1 or RAM 2. The data is written to the appropriate RAM, which is dependant on the RAM Control signal (RAM switch) in the second cycle. Figure 2 is a pictorial representation of the memory writing pipeline in two clock cycles. Two display units (VGA) are use for visual inspection to display the background model and moving targets respectively. The two units have the same vertical blanking and hence it is possible to generate different outputs with a single data. The background data read from RAM 3 (or RAM 4 depending on the RAM control) is used to extract the foreground image, which is sent to the foreground display unit (FDU). Simultaneously, the background data is sent to the background display unit (BDU). This has been made possible by reading pixels six cycles ahead of their display time. Out of these six cycle, the rst cycle is used to enable memory read for RAM 2 and 3 (or RAM 1 and 4), camera and its corresponding background data is made available in the second cycle. The 8-bit grayscale intensity value is converted to 11-bit xedpoint value in the third cycle. The fourth cycle is used to build a cluster around the 11-bit central value maintained as the background value. This is followed by the fth and nal stage, which estimates if the 11bit grayscale value falls in this cluster. If the value falls out of range of the cluster, the grayscale value is sent to the foreground display unit else zero (intensity value of zero) is sent. The background value is simultaneously sent to the background display unit. The fth cycle is also used to enable memory write for RAM 4 (or RAM 3). The nal and sixth cycle displays the

values on the display units and writes the updated background value to the external RAM. It should be pointed out that the rst six pixels of the display units are incorrect on reset.

1#MAR 2#MAR

Resource Flip Flops 4 input LUTs Block RAMs bonded IOBs Occupied Slices SSRAM (NTSC) SSRAM (PAL)
Total Used 1,112 out of 67,584 1,762 out of 67,out of out of 824 1,156 out of 33,out of 256 Mbits 11 out of 256 Mbits
Per. 1% 2% 0% 44% 3% 4.3% 6.3%
Figure 2. The the 2 stage memory write pipeline architecture
4.1.2. Results and Analysis.

< = > Q

Many colour segmentation algorithms have been proposed in the past for skin lesion images, intrusion detection and feature extraction. Very few of these have successfully been implemented on hardware platform for speed-up. This is due to a lack of resources to meet the real-time processing needs of these very complex and computationally expensive colour segmentation algorithms. The need for such algorithms running on dedicated systems like FPGAs is mentioned by Neuenhahn [14]. Real-time colour image segmentation could be particularly useful in many applications, especially in biomedical image analysis [3].
Figure 3. The fth stage of the six-stage pipeline
4.2.1. FPGA System Design.
Similar to the grayscale implementation, this architecture has six processes running in parallel. These are: the RAM control; pixel reading from camera; writing camera data to RAM; processing stored data for the output device and updating background data; writing to the output device; and writing the update background

ro tarapmoc tib 11

RGB Colour Background Modelling

< = >

rexelp itl u M

S kcalB

rotarapmoc tib 11
The design described above runs at 64.81MHz as reported by the Place and Route (PAR) tool. At this speed, the design is able to process camera data at more than realtime. The total latency from the time the camera sends the rst pixel to the time the the pixel is available for display on display units is approximately 0.2sec. This is because it takes 5 clock cycles to write a pixel from the camera, which runs at 25Hz. It takes 6 clock cycles to update the background data and extract the region of interest for a given pixel. Thus at a clock speed of 64.81MHz, the latency is approximately 0.2sec. Table 1 summarizes the resource utilization of this implementation. When the pipeline is full the FPGA produces result for a pixel every clock cycle. Thus running at 64.81MHz, the output of a pixel is ready every 15.429ns. At this frequency we can effectively process 210fps of RS-170/NTSC (640480) frame size and 146fps of CCIR/PAL (768576) frame size.

e n il e p i P r e t s i g e r

1# MAR

2# MAR

elb anE etirW

hctiwS MAR
Table 1. Resource utilization of the unimodal grayscale implementation, using XC2v6000, package ff1152 and speed grade -4.
In this section we demonstrate how this challenging task in image processing can be achieved with simplied yet robust algorithms, running at real-time on recongurable computing platforms with minimal resources. The architecture we present here is similar to that in section 4.1, but instead of using an 8bit grayscale intensity value this architecture relies on 24-bit RGB values. Again, the entire system is tted on a single FPGA chip with 4 banks of SRAM.
data back to RAM. These processes take 12 clock cycles in total. The longest process, the background updating and target extraction process, takes 5 cycles. The rst cycle generates memory addresses and reads the corresponding background and camera values from the respective RAM blocks. The length of the camera data is 24bits (8 bits for each RGB channel) and that of the background is 33bits (11bits for each channel). The second cycle makes the addressed data available for processing. This is followed by extending the 38 bit RGB values into 11 bit xed point values in the third cycle. The fourth cycle is used to build a cluster of width 30, each for the Red, Green and Blue colour components. The fth cycle is used to decide if all the colour components of the observed pixels belong to the RGB clusters. The result of this denes the output to the VGA. If all colour components fall in their corresponding clusters, then the pixel is displayed as a background pixel else displayed as a foreground pixel. The last pipeline stage is depicted in gure 3, for the Red colour component, as the other two components have a similar structure.

Code 110 111

Description cluster 0 highest wgt. & data out of range cluster 0 highest wgt. & data in range cluster 0 second wgt. & data out of range cluster 0 second wgt. & data in range cluster 1 highest wgt. & data out of range cluster 1 highest wgt. & in of range cluster 1 second wgt. & out of range cluster 1 second wgt. & in of range
Table 2. Econding scheme for determining which cluster camera data belongs
modal background with n = 2. We demonstrate the success of this implementation and how easily any n can be increased based on the available resources.

4.3.1. FPGA System Design.
4.2.2. Results and Analysis.
This design also runs at an impressive speed of 64.40MHz, about 0.4MHz less than that of the grayscale implementation. The percentage of the resources consumed by this implementation is the same as that of the grayscale implementation, with the exception of 4 input LUTs (approximately 3% in this case). This can be attributed to the increase in the number of pipeline registers. The total latency from the time the camera sends the rst pixel to the time the pixel is available for display on display units is also 0.2sec. The percentages of external RAM consumed by the design for processing a PAL and NTSC frame sizes are 13% and 19% respectively. When the pipeline is full the FPGA produces result for a pixel every clock cycle, thus running at 64.40MHz, the output of a pixel is ready every 15.526ns. At this frequency we can effectively process 209fps of RS170/NTSC (640480 colour) frame size and 145fps of CCIR/PAL (768576 colour) frame size.
Bimodal Gray-Scale Background Modelling
Multimodal background modelling is required to model scenes with non-stationary background objects, so reducing false positive alerts. To successfully implement our algorithm on a Field Programmable Gate Array, access to 17n bits of background data is required every clock cycle, where n is the total number of background clusters and 17 bits is the total number of bits require for the clusters weight (6 bits) and central value (11 bits). Having access to 36 bits in a clock cycle, we can effectively implement a multi-
The design of the bimodal background is common with all the other architectures; it has six different processes running in parallel. The interesting part of the design is the eight stage pipeline for extracting moving targets and updating the background. The rst two stages are used for reading the camera and its corresponding background data from external RAM. The 8-bit grayscale value from the camera is converted into 11-bits xed point value in the third stage. To avoid deeply nested conditions as well as iteration in sorting the weights of the background clusters, different registers are used for the results of the sorted weights. This makes it possible to sort the weights in a single clock cycle in the fourth stage in the pipeline. Clusters are also built on the various central values in this stage. The fth stage is used to determine if the camera data belongs to the highest weighted cluster. We use an encoding scheme to know which cluster the camera data belongs to. Table 2 gives the encoding scheme used for the bimodal implementation and can easily be used for multimodal implementations. The weight and the background values are updates as in equations 11 and 12. If the camera data does not match the highest weighted cluster, the appropriate code is set and the camera data is compared with the second weighted cluster in the sixth stage. The pipeline stages will increase depending on the number of clusters used in the implementation. This causes a delay in terms of setup and propagation time for the pipeline registers but the speed gain as compared to a deeply nested conditional statement is very signicant. In the seventh and nal stage the lowest weighted clusters central value is replaced with the 11bit xed point camera data and its weight set to zero if the data does not belong to any of the clusters. The foreground bit-map for the FDU is extracted in this same stage. For the BDU, a grayscale value is displayed if the

Total Used 1,766 out of 67,584 3,347 out of 67,out of out of 824 2,124 out of 33,out of 256 Mbits 25 out of 256 Mbits
Per. 2% 4% 39% 44% 6% 14.4% 9.7%
Scene In1 Out1 In2 Out2 In3 Out3 In4 Out4
Table 3. Resource utilization of the bimodal grayscale implementation, using XC2v6000, package ff1152 and speed grade -4.
pixel belongs to the highest weighted cluster or none of the clusters, and a red value is displayed if the pixel belongs to the second-weighted cluster. This colour display mode can also be used in a multimodal model. The updated background and weights of the clusters are sent to the background updating processing unit to be written to external RAM. It takes 8 cycles to get the rst correct pixel displayed after reset.
In1 Out1 In2 Out2 In3 Out3 In4 Out4
Our Approach (%) SENS. SPEC. PPV 84.88 98.94 84.01 83.18 99.87 94.79 76.26 97.28 62.23 76.87 99.31 92.50 76.72 96.51 47.82 81.39 99.56 58.03 86.39 89.14 30.62 56.35 99.06 48.53 Grimsons (%) SENS. SPEC. PPV 82.20 99.10 85.68 80.99 99.85 93.50 68.70 97.72 63.93 64.33 99.22 89.75 68.54 98.30 62.68 75.58 99.70 65.40 85.38 91.71 36.38 45.96 99.38 53.91
Table 4. Pixel errors evaluation results The algorithm does produce more false positive errors; this is the side-effect of our approach in detecting targets with low contrast against the background. However, in our target application false positive errors of the type reported are more acceptable than false negative errors, as subsystem tracking stages can discard distracters such as shadows. The evaluation parameters used in table 4 are dened as follows: Sensitivity (SENS.) is the proportion of positives that are correctly classied and its expressed as (T PT P N ) , Specicity (SPEC.) is the +F proportion of negatives that are correctly classied TN and its expressed as (T N +F P ) and Positive Predictive Values (PPV) is the proportion of cases classied as positives that are actually positive and its exP pressed as (T PT+F P ). Figure 4 gives sample images of the outputs generated by the two algorithms. These are output results of seven out of the eight sequences used in table 4. Visual inspection of the images shows where our approach outperforms that of Grimsons. Grimsons algorithm suffers from the foreground aperture problem, but our approach with its frame-level processing suffers minimally from this problem.

4.3.2. Results and Analysis.
This design runs at 57.00MHz, about 7.81MHz less than that of the unimodal implementation. This has only been possible with the use of the encoding scheme, as an earlier implementation could only run at maximum speed of 25MHz due to the use of deeply nested condition for evaluating the cluster that the camera data belongs to. To reduce noise due to the camera jitter, morphological opening is conducted on the foreground extracted. This is entirely conducted on the BlockRAM available on the FPGA. Effectively, this reduces the latency by a factor of 9 per pixel as it takes only one cycle to access data from the dual-port BlockRAM as compared to two cycles for the external RAM. Table 3 gives a summary of the resource utilization in this design. At 57MHz an output is ready every 17.543ns, when the pipeline is full.

5. Experimental Results

We evaluate the performance of our approach against that of [15] using K = 3, thus 3 cluster in our case and 3 distributions in [15]. We use eight randomly selected video sequences, four each from outdoor and indoor scenes. Manually marked frames are used as reference images for the sequences. Our result is based on pixel-wise errors against the reference image, in terms of true positive(T P ), true negative(T N ), false negative(F N ) and false positive(F P ) pixels. There are approximately 50 frames in each sequence. Table 4 clearly shows the superiority of our algorithm against that in [15] in terms of sensitivity.
6. Summary and Conclusions
We have demonstrated how a single chip FPGA can effectively be used in modelling robust multimodal backgrounds in real-time. We have presented architectures for modelling unimodal backgrounds using grayscale intensity value, RGB colour values and bimodal backgrounds with grayscale values. The novelty detector used is motivated by [12] and [15]
Figure 4. Sample outputs of the algorithms: Left shows our approach, middle is the original frame and the right is Grimsons algorithm
with modications to enable the extraction of targets with low contrast. The processing speed and resources used in the implementation make room for other imaging algorithms for tracking and activity interpretation.

7. References

[1] E. Ashari. FPGA Implementation of Real-Time Adaptive Image Thresholding. SPIEThe International Society for Optical Engineering, December, 2004. [2] V. Bonato, A. Sanches, and M. Fernandes. A Real Time Gesture Recognition System for Mobile Robots. International Conference on Informatics in Control, Automation and Robotics, Portugal, August, 2004. [3] R. Cucchiara, C. Grana, S. Seidenari, and G. Pellacani. Exploiting color and topological features for region segmentation with recursive fuzzy C-means, volume 11. 2002.
[4] P. Ekas. Leveraging FPGA coprocessors to optimize automotive infotainment and telematics systems. Embedded Computing Design, Spring, 2004. [5] A. Elgammal, D. Harwood, and L. Davis. Nonparametric Model for Background Subtraction. Proceedings of the 6th European Conference on Computer Vision, Dublin, Ireland, 2000. [6] D. Gutchess, M. Trajkovic, E. Cohen-Solal, D. Lyons, and A. K. Jain. A Background Model Initialization Algorithm for video Surveillance. IEEE, International Conference on Computer Vision, 2001. [7] I. Haritaoglu, D. Harwood, and L. Davis. W 4 : Who? When? Where? What? A real time system for detecting and tracking people. IEEE Third International Conference on Automatic Face and Gesture, 1998. [8] J.Batlle, J. Martin, P. Ridao, and J. Amat. A New FPGA/DSP-Based Parallel Architecture for RealTime Image Processing. Elsevier Science Ltd., 2002. [9] C. T. Johnston, K. T. Gribbon, and D. G. Bailey. Implementing Image Processing Algorithms on FPGAs. Proceedings of the eleventh electronics New Zealand Conference, ENZCON04, Palmerston North, Nov, 2004. [10] M. Leeser, S. Miller, and H. Yu. Smart Camera Based on Recongurable hardware Enables Diverse Real-time Applications. Proceedings of the 12th annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM04), 2004. [11] B. Levine, B. Colonna, T. Oblak, E. Hughes, M. Hoffelder, and H. Schmit. Implementation of a Target Recognition Application Using Pipelined Recongurable Hardware. Military and Aerospace Applications of Programmable Devices and Technologies International Conference, 2003. [12] A. Makarov. Comparison of Background extraction based intrusion detection algorithms. IEEE Int. Conference on Image Processing, 1996. [13] S. McBader and P. Lee. An FPGA Implementation of a Flexible, Parallel Image Processing Architecture Suitable for Embedded Vision Systems. Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS03, 2003. [14] M. Neuenhahn, H. Blume, and T. G. Noll. Pareto Optimal Design of an FPGA-based Real-Time Watershed Image Segmentation. 15th Annual Workshop on circuits systems on signal processing, November, 2004. [15] C. Stauffer and W. E. L. Grimson. Adaptive background mixture models for real-time tracking. IEEE Conference on Computer Vision and Pattern Recognition, 1999. [16] J. Torresen, J. W. Bakke, and L. Sekanina. Efcient Image Filtering and Information Reduction in Recongurable Logic. Proceeding of the 22nd NORCHIP Conference, Norway, November, 2004. [17] R. Williams. Increase Image Processing System Performance with FPGAs. Xcell Journal, Summer, 2004. [18] C. Wren, A. Azarbayejani, T. Darrel, and A. Pentland. Pnder: Real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997.