4-249-341-04(1)

XR-MJ1

2003 Sony Corporation

CD/MD/ MD1 MD MDLP MD / 4

.. 6.. 10.. 12

... 14 CD. 17 CD MD... 22 MD.... 25... 27.. 29

CD CD CD

. 33. 33.. 34.. 36

.. 42.. 44

CD CD-MD CD CD-MD 1 CD/ /
. 53... 55.. 56.. 58. 48. 50. 52

MD MD MD

.. 37. 37.. 38.. 40.. 41

.. 60.. 98. 99

MD NAME.. 61 NAME.. 64 CREATE.. 65 RELEASE. 67 ERASE.. 68 MOVE.. DIVIDE. COMBINE. 74 UNDO. 75 S.F EDIT.. 76

.. 100

MD HCMS 3

.. 102.. 103.. 105

.. 114.. 118. 121.. 121.. 122.. 127

CD CD-TAPE CD/MD/

.. 79. 81
. 82. 84.. 86.. 87. 88.. 89... 90. 91.. 93.. 93. 95

Tel. 03-5353-0336

ENTER/YES

MENU/NO MODE

3,4,5, 6,7
3,4,5, 6,7 3,4,5, 6,7 3,4, 5,6

0:00PM

0:00AM

MODE ECO MODE

MENU/NO

CLOCK 2 ENTER/YES 3

MODE CLOCK SETTING? ENTER/YES 3 7

MULTI JOG

FM AM 32

t t t t t t

Auto Set Complete!
TUNER/BAND FM AM MODE AUTO PRESET?

118 120

1 MODE
> 2. MONO 3 ENTER/YES 5 MULTI JOG

FM MODE TUNER

CLEAR PRESET +. PRESET

1 PRESET CLEAR

2 ENTER/YES

CLEAR?

TUNER/BAND FM AM
> 2. MONO 3 ENTER/YES MULTI JOG

STEREO

2 MHz 3 95.75 101.75 107.75

FUNCTION VOLUME

MUTING
l/ L MODE ENTER/ MULTI JOG

CD CDNX

CD CD CDNX CD

2 CDNX

10 >10/0 MULTI JOG

l L l L

CD OVER MUTING MUTING VOLUME CD

CD CD FUNCTION CD CD

VOLUME
1 MODE 2 MULTI JOG 3 ENTER/YES 2

RESUME CDBON

CDBOFF

CD MD MD

MD 68 CD 48 CD

CD 50 55

CD SYNC REC SEL CDBMD X1 REC START/REC PAUSE

Press START MD MD REC CD

C11 No Disc

Protected

MD 113 115

MD TOC TOC MD

Press START REC START/REC PAUSE

TOC TOC TOC Writing

TYPE I CD 79 CD

ZPUSH EJECT

CD SYNC REC SEL CDBTAPE REC START/REC PAUSE

Press START TAPE REC CD

TAPE No Tape TAPE No Tab

REC MUTING CD 10

CD SYNC REC SEL CDBMD/TAPE 6
C11 Protected No Disc TAPE No Tape TAPE No Tab

MD 117

MD MDNX

MD MDNX MD

FUNCTION MD MDNX

2 MDNX

MUTING MUTING VOLUME MD

TYPE I

TAPEN

FUNCTION TAPE TAPEN

MUTING MUTING

lm/ ML MODE ENTER/YES

TUNER/BAND AM MULTI JOG

TUNER/BAND

CD TUNER/BAND

1 MODE FM MODE

MUTING MUTING VOLUME VOLUME

MONO 3 ENTER/YES

0 >10 10/0

2 m 2 m

AM AM

MD C11 Protected

MD 91 MD TOC TOC MD TOC TOC TOC Writing

REC SEL

MD REC
REC MD REC START/REC PAUSE

REC START/REC PAUSE

REC TAPE REC START/REC PAUSE

TAPE REC

MODE REPEAT

FUNCTION CD

CDBALL Tr CDBONE Tr

MODE PLAY MODE

CDBSHUFFLE

CDBALL Tr 5 CDBONE Tr

CD 1 1

2 CDBOFF

CDBALL Tr MULTI JOG CDBONE Tr
1x 2 MODE MODE > 3. CDBCONTINUE

MULTI JOG CDBSHUFFLE

25 MD 48 79

DISPLAY

CDBCONTINUE

3,5 3,4

CDBPROGRAM ENTER/YES

DISPLAY STEP

2 MODE

PLAY MODE 3.

10 >10/0
CDNX CD 26 CD 31 : MULTI JOG STEP Step Full! 100 CD 31

FUNCTION MD

MDBALL Tr MDBONE Tr

MDBSHUFFLE

MDBALL Tr 5 MDBONE Tr

MD 1 1

2 MDBOFF
MDBALL Tr MULTI JOG MDBONE Tr
1x 2 MODE MODE > 3. MDBCONTINUE

MULTI JOG MDBSHUFFLE

DISPLAY GROUP

MDBCONTINUE

MDBPROGRAM ENTER/YES

2 MODE PLAY MODE 3.

>10 10/0
MDNX MD GROUP 26 STEP Step Full! MD 1,000 : MULTI JOG

37 39

MD MD 1 MDNX

GROUP GROUP

GROUP v

MD 1,000

MD 61 64

, Z =/1

MD TOC CD

TOC Writing MD

TOC TOC MD TOC TOC Writing

T.Mark LSync

MD Protected C11

MD TOC

Table of Contents

CD-MD SYNC NORMAL

S.F EDIT LP2 LP4 S.F EDIT MONO S.F EDIT 4 S.F EDIT 4 S.F EDIT 4 S.F EDIT

PHONES

MD 1 CD GROUP MD MDLP 1 On Off
0 ; Favorites // ; Rock // ; Pops //. 3

Favorites Rock Pops

NAME MD 1 2

GROUP 68

MD MD
REC SEL REC MD REC START/REC PAUSE

New Group MD

ENTER/ YES

CD TAPE FM AUX CD AM 98

4 REC START/REC PAUSE Group Full!
1 MD 99 Group Full! C11 MD 42

MD Protected

3 4,6 ENTER/YES 5

Group Full!

CD ENTER/ YES

MD TOC TOC

TOC MD TOC TOC Writing
MULTI JOG CDBPROGRAM ENTER/YES

9 10,12 3,2

CLEAR GROUP v

GROUP V

CD SYNC REC SEL CDBMD X1

MD CDBMD/TAPE

10 REC START/REC PAUSE

68 CD CD MD

9 CD-MD CD

MD C11 ERASE 68

Protected No Disc MD 113 115

CD SYNC REC SEL CDBMD X4

Press START MD CD
5 REC START/REC PAUSE Retry

CD Retry

MULTI JOG CD HCMS 7 Retry HCMS 103
MD TOC TOC MD TOC TOC Writing

68 CD CD MD

4 CD-MD CD

CD CDNX

CD SYNC REC SEL CDBMD X1 CDBTAPE

CD MD 68 4

MD TOC TOC TOC MD TOC TOC Writing

C11 No Tape MD 117

Protected No Disc TAPE No Tab

113 115

MD FUNCTION CD

CD TAPE FM AUX

CD AM 98
CD New Track CD 1 MD Protected C11 MD 42

REC START/REC PAUSE 122

MDLP EDIT MDLP S.F MD

CD-MD REC MODE

240 MDLP
t MDBSTEREO r MDBLP2 LP2 r MDBLP4 LP4 r MDBMONO

4 SP 2

REC MODE

MD MDLP

C11 MD 42

DIVIDE

MD MDLP LP: MD MD LP: MDLP

L-SYNC

2,5 2,3,4 3,4

MENU/NO Setup Menu

. >

1 2. T 3.

T.Mark Off T.Mark LSync ENTER/YES

T.Mark LSync ENTER/YES

72dB 0dB 2dB ENTER/YES 4 MENU/NO
4 T.Mark Off ENTER/YES L-SYNC

CD On 3

30 Off

On 30 3

S.Space On S.Space Off ENTER/YES

S.Space On ENTER/YES

30 Smart Space Auto Cut 3 MDNX

4 S.Space Off ENTER/YES

S.Space On Off
Combine 1 MD MD MD Undo 74 1

Name

S.F EDIT 76 1

Create

Release 67

Erase Move Divide 1 2

MD Divide

, Z ^/1

MD TOC TOC Writing MD 1 MD 1,700

MENU/NO CLEAR > M

NAME EDIT/SELECT
Selected AB Selected ab Selected Selected 12 Selected AB
&+<>_=;#$%@ ` >10 ,.

1 ABC DEF

1 NAME EDIT/SELECT Selected AB Selected ab 2 10/0

42 abc//def

CLEAR 3 4

Complete! 1

1 MENU/NO > 2. Nm Erase? 3.

ENTER/YES Complete!

1 MD 1,700 800

. MENU/NO ENTER/YES >

GROUP GROUP V 2 GROUP v 1 DISPLAY

GROUP GROUP GROUP v

GROUP GROUP 2 MENU/NO > 3. Erase? GROUP v ENTER/YES Complete!

Nm ENTER/YES

CREATE

MD 42 abc//def //

4,6,7 4,6,7

4,6,7 1

Group Assign None
Gp Create? ENTER/YES MD 6

1 ENTER/YES

Complete!

6. None Complete!

RELEASE

Gp Release? ENTER/YES

REL Gp ??

Erase MD UNDO

3 Track Erase All Erase A-B Erase Track Erase 3 2

Gp All REL? ENTER/YES

All REL??

Complete! MD

2 3,5 3,4

All Erase MD MD

Tr Erase? ENTER/YES

2 3,4 3

Complete! 4

All Erase? ENTER/YES

All Erase?? MENU/NO

75 MENU/NO

A-B Erase? ENTER/YES

A-B Erase 1

1 1/86

Rehearsal A

Point A ok?

1/86 1f

A-B ERASE

M. >

LP2 2f LP4 4f

2 3,4,7, 8,10 3,4,5

A 5 A ENTER/YES

Point B set B

A-B Ers A-B A B 3 2

ENTER/

Move Point B ok?

10 ENTER/YES

Complete! B A-

Impossible MD

3,4,5 1

2 3,4,6 3,4,5

Move? ENTER/YES

Divide

Divide? ENTER/YES

Andante Adagio

Allegro

Rehearsal

Andante Adagio Allegro

Impossible

75 COMBINE 74

COMBINE

Combine 2 1

COMBINE 1 3

Combine? ENTER/YES

75 DIVIDE 72 MD

LP2 LP4

S.F EDIT UNDO

2 Impossible MD

1 3,4 2

S.F EDIT

Name Undo?

1 Group Undo?
1 Erase Undo? Move Undo? Divide Undo? Combine Undo?

3,5,6, 7,8 1

4 3,5,6,7, 9,10 3,5,6, 7,8
MDBCONTINUE ENTER/YES MENU/NO

S.F Edit? ENTER/YES

Tr Level? ENTER/YES

Level 0dB

MDBCONTINUE ENTER/YES

12dB 12dB OVER

Fade In? Out? YES

Fade ENTER/

S.F Edit OK?

Time 5.0s

S.F Edit: %

UNDO

SF Edit: Complete! %

MENU/NO 10 ENTER/YES

CD-TAPE

8 9,3,5

Press START CD

REC MUTING 10

MD 7 CDBMD/TAPE

CD SYNC REC SEL CDBTAPE

TAPE No Tape TAPE No Tab 117

CD/MD/

CD MD CD MD
REC SEL REC TAPE REC START/REC PAUSE

REC MUTING

REC MUTING REC MUTING 4

2 REC START/REC PAUSE

CD MD FM AUX CD MD AM 98

REC MUTING 4

REC MUTING REC MUTING

/,. :!?

MENU/NO CLEAR

2,6,7,8,9 7,8

MODE USER NAME ENTER/YES

CLEAR ENTER/YES

ENTER/ YES ENTER/YES

1 BIRTHDAY? 2 SCHEDULE?

3,4,6,7

m MENU/NO CLEAR

2,3,5,6,7,11 3,4,6,7

MODE SCHEDULE ENTER/YES

1 /,. :!?

CLEAR 8 9

11 ENTER/YES

>10

&+<>_=;#$%@ `

2 CLEAR

3 CLEAR?

ENTER/YES CLEAR?

3 ENTER/YES

DEMO MULTI JOG /

ON ENTER/YES

MODE OFF-LIGHT

OFF-LIGHT 1 OFF-LIGHT 2 OFF-LIGHT 3
OFF-LIGHT 1 OFF-LIGHT 3 OFF-LIGHT 2

OFF LIGHT OFF

11 OFF-LIGHT OFF

MODE DIMMER

DIMMER 1 DIMMER 2

DIMMER OFF

COLOR PRESET?

CLEAR MENU/NO

COLOR AUTO?

3,5 2,4 4

t RAINBOW r

FOREST

NIGHT&DAY 1

MODE COLOR

24HOURS 1
COLOR INITIAL COLOR PRESET? COLOR AUTO? COLOR MANUAL?
COLOR INITIAL COLOR PRESET? 14 COLOR AUTO? 8

EVERY DAY

FUNCTION COLOR MANUAL?

G/B R G B 0 7

COLOR MANUAL?

COLOR INITIAL

COLOR PRESET? COLOR AUTO? COLOR MANUAL?

4 CLEAR

t FLAT r

HIP HOP

TECHNO

EQ i-Bass

, i-Bass

i-Bass 1

i-Bass i-Bass 1

i-Bass 2

i-Bass 2 i-Bass 1

i-Bass

i-Bass OFF

> 1. USER Title? 2 5

TUNER/BAND FM AM PRESET

MODE TITLE?

FM AM TITLE? ENTER/YES

CLEAR 6 7

DISPLAY DISPLAY 1 ABC DEF

CLEAR 3 ENTER/YES

MULTI JOG USER Title?

, SLEEP

t SLEEP OFF

t AUTO T 90 T

3,4,5, 6,7,8
3,4,5, 6,7,9 3,4,5, 6,7,8

T. T 80

SLEEP 1

SLEEP OFF

CD CD

34 MD MD

CD MD 100 CD MD 27 AUX 98 38

TIMER TIMER 1 TIMER 3

TIMER 2

PLAY ENTER/YES

t MD t CD t TUNER AUX T TAPE T

0 00PM 0 00AM

1 TIMER TIMER 1 TIMER 2 TIMER 3 2.
TIMER TIMER 1 TIMER 2 TIMER 3 2.
3,4,5, 6,7,8,9 3,4,5, 6,7,8,9

OFF ENTER/ YES

3,4,5, 6,7,8,9

7 TUNER

TIMER 1 TIMER 2 TIMER 3

REC YES

RECBMD YES

RECBTAPE ENTER/

0:00PM 0:00AM

LP2 YES

40 Welcome VOLUME

FUNCTION AUX

FUNCTION AUX REC SEL REC MD REC TAPE REC MD/TAPE REC START/ REC PAUSE

MD ENTER/YES AUX AUX

t AUX t VIDEO TV T

CD MD CD CD CD CD

CD-R/CD-RW

C-1KN CHK-1

Disc Full!

MD LP4 LP2 8

255 Disc Full!

Group Full!

4 LPLP2 8

MOVE DIVIDE Group Full!

LPLP4 48

CD MD DAT

CD-MD 74

MD DAT

HCMS CD ISRC: International Standard Recording Code 74 74

CD-MD CD-MD

MD HCMS

MD HCMS 74

c VOLUME c c c

c MUTING c c c c c

c c TIMER c c c c c c c

c c ZCD

c c c c

CD c CDNX

c MODE >

PLAY MODE CDBCONTINUE

FM/AM STEREO

FM FM 8 9

c c c MODE
FM MODE AUTO STEREO STEREO

TOC TOC

TOC Writing TOC Writing

MD c MDNX

c GROUP

PLAY MODE MDBCONTINUE

MD c MD
MD MD MD C11 Protected MD MD ERASE 68

c c c 75

LP4 LP4

c c c c c c c c

c MDLP c

MDLP MD

/ C11 / Protected

C13 / REC Error

C13 / Read Error

C14 / TOC Error

MD Erase

All 69

C41 / Cannot Copy

103 E0001 / MEMORY NG

E0101 / LASER NG

E0201 / LOADING NG

Assign None

Auto Cut

MDNX 59

Blank Disc

Cannot Edit

c c MODE >

MD MD PLAY MODE MDBCONTINUE

MDLP c MDLP Cannot REC

Cannot SYNC!

Complete! Disc Full!

Impossible

Combine

MD MD MD MD Incomplete!

Initialize

MD Eject

Name Full!

No Change ENTER/YES

No Disc

Reading

A-B Erase

Divide CD

Retry Retry [HCMS]

c c 75 c

CD-MD CD-MD 48

S.F Edit!

c S.F EDIT

S.F Edit NOW

?/1 c S.F EDIT

Smart Space
59 Step Full! TOC Writing

Track End

Step Full!

TAPE No Tape

TAPE No Tab

OFF TIME NG

FM (TV)
MHz 80.4 82.5 kHz AIR-G NORTH WAVE 1440
MHz 76.1 77.1 80.0 80.4 81.8 82.8 FM Date FM FM FM kHz 1233 1260

IBC YBC ABS RAB TBC

MHz 76.1 76.4 77.1 78.8 77.5 86.5 78.0 78.6 83.0 79.0 79.5 79.7 80.0 81.3 84.7 86.3 Inter FM RadioBerry kHz 1422 1530

YBS AFN TBS SBC BSN IBS

FM bayfm FM-FUJI FM Port NACK5 FM TOKYO FM J-WAVE FM FM
MHz 76.1 77.8 78.9 79.2 79.5 80.5 80.7 82.7 FM ZIP-FM FM K-MIX RADIOi HELLO FIVE FM AICHI FM kHz 1431

KNB FBC MRO SBS GBS

MHz 76.5 77.0 77.6 89.9 80.2 85.1 89.4
FM COCOLO E-Radio Kiss-FM FM802 fm osaka * -STATION kHz AM KOBE ABC KBS MBS WBS
MHz 76.8 77.4 78.8 86.6 78.2 79.2 FM V-air kHz 1350 1494

KRY BSS RCC RSK

HFM FM
MHz 78.6 79.7 80.7 81.6 FM FM FM FM kHz 1269 1449

RKC RNB JRT RNC

MHz 76.1 82.7 77.0 78.7 77.4 77.9 79.5 79.8 80.0 80.7 83.2 87.3 88.0 89.1 LOVE FM761 CROSS FM FM FM SMILE-FM FM FM FM FM FM AFN kHz 1458
AFN RBC MRT OBS MBC RKK NBC RKB

NHK 91

CX-LMJ1
20W + 20W JEITA 6 800mV 47k PHONES 16

SUB WOOFER 1.2 V

2Hz 20kHz

44.1kHz 5Hz 20kHz

TYPE I 60 10,000Hz 3dB
FM 76 108MHz AM 531 1,602kHz

FM 75 AM

SX-LMJ1 SX-LMJ2

2 12cm 3cm 234 226mm

2.3kg 1
AC 100V 50/60Hz 55W 0.45W /CD JEITA
/MD / 299.7mm /CD / 5.3kg FM AM /MD / /

1 ?/1 2

3 MD Z 4 PHONES

qa qs qd

CD SYNC REC SEL

6 FUNCTION

qh qj qk ql w; wa

9 MDNX

0 DEMO

qs TUNER BAND

qd TAPEN

wa MULTI JOG

qf ZPUSH EJECT

MD GROUP v/V

qg i-Bass

PRESET

ws VOLUME

qh l/L

TUNING +/

x qj DISPLAY qk GROUP

ql MODE

1 AUX IN

PLAY MODE

w; MENU/NO

3 FUNCTION

t MD t CD t TUNER

4 SPEAKERS

AUX T TAPE T

5 SUB WOOFER

4 TUNER BAND

SUB WOOFER

5 TIMER

6 NAME EDIT/SELECT

SELECT

7 MENU/NO

PRESET /

./>

1 ?/CLOCK

GROUP v/V

CD MD / 84 92

qs MDNX

CDNX TAPEN

x qd MODE

qd qf qg qh

qf DISPLAY

qg GROUP

qh CLEAR

qj VOLUME +/

9 MUTING

0 SLEEP

25 CD MD 90

26 CD 15 MD CD 33 MD 98

MD CD 33 MD 76

A-B AM CD CD CD CD

19 MD 17 MD 64 76

CD-MD 50 105

CD 36 MD 41
89 CD 36 MD CD 34 MD 38 98

29 MD 82

21 CD-MD CD-TAPE COMBINE DIVIDE ERASE FM MD MD MD MD 22 MD MD MD MOVE NAME S.F EDIT UNDO 1

Printed in China

2007 IEEE International Conference on Robotics and Automation Roma, Italy, 10-14 April 2007

FrE7.3

Particle-based Sensor Modeling for 3D-Vision SLAM
Daniele Marzorati ,Matteo Matteucci ,Domenico G. Sorrenti

Universit` a Politecnico

di Milano - Bicocca, Milano, Italy di Milano, Milano, Italy
Abstract Self localization and mapping with vision is still an open research eld. Since redundancy in the sensing suite is too expensive for consumer-level robots, we base on vision as the main sensing system for SLAM. We approach the problem with 3D data from a trinocular vision system. Past experience shows that problems arise as a consequence of inaccurate modeling of uncertainties; interestingly enough, we found that accuracy in modeling the robot pose uncertainty is much less relevant than for the uncertainty on the sensed data. To overcome the severe limitation of linear and Gaussian approximations, we applied a particle-based description of the inherently nonnormal probability density distribution of the sensed data; the aim is to increase the success rate of data association, which we see as the most important problem. The increase in correct data associations reduces the uncertainty in the model and, consequently, in the robot pose, respectively estimated with a hierarchical map decomposition and a six degree of freedom extended Kalman lter. In this paper, we present approaches for particle-based sensor modeling and data association, with a comparative experimental evaluation on real 3D vision data.
w.r.t 3D LRFs, is impressively large in favor of multi-camera systems; moreover, it is fast-reducing. In this work we show that, for vision-based SLAM, the accurate modeling of the uncertainty on the sensed data allows to reach excellent results, without any particular attention to the uncertainty on the robot pose. Section II briey introduces the state of the art in SLAM while the issues in uncertainty modeling of 3D data from a stereo-vision systems are discussed in Section III. Section IV describes a novel approach to data association exploiting a particle-based description of data uncertainty. The effectiveness of such idea is presented, on real data, in Section V. Conclusions and future work are presented in the last section. II. S TATE OF THE ART A non-linear function of normally distributed variables is usually approximated with a gaussian model; this dates back to the work of [4] and has been widely used in the literature. EKF can be seen as a similar approach, as it also bases on linearization. EKF is also state-of-the-art in SLAM. In the recent years we could appreciate the afrmation of other Bayesian information processing, from Unscented Kalman Filter [5] and (Sparse Extended) Information Filter [6], to Particle Filter [7]. The state of the art in uncertainty modeling for SLAM is to use the normal distribution for the sensed data and a more accurate, particle-based, approximation for the robot pose [2]. For each value of the robot pose a reconstructed world is computed, basing on a normal distribution of the sensed data, given that pose. In these approaches we have a gaussian distribution, to approximate the noise on the observations of the features, and a particle lter to model the robot pose. In this way the different hypotheses on dataassociation (and therefore on world reconstruction) can be kept under consideration as long as they could turn to be correct (a not greedy approach). In grid-based approaches we have a discretization of the state, the robot location, into cells (histogram lter) and a gaussian distribution is used to model the features [8] This lter represents the cumulative posterior, for each region, with a single probability value. III. 3D S ENSOR DATA FROM VISION The perception system we used follows a consolidated approach, i.e. trinocular stereo, and reconstructs the scene in terms of 3D segments [9]. This makes the robot able to capture part of the 3D nature of the environment. Such systems are quite widespread in the computer vision and robotics communities. In order to give out 3D segments

I. I NTRODUCTION In many real-world applications, e.g. autonomous world model building by a mobile robot (a.k.a. the SLAM problem), modeling the uncertainty of the data is of uttermost importance, as it heavily affects the system performance; see e.g. [1] for applications that were successful because of the appropriate modeling of uncertainties. In this problem, the observer mobility implies the integration of data that, for its nature, is very noisy. Every perception system suffers for its own, usually non-gaussian, noise; the gaussian/normal distribution is often used to model it and often this turns out to be a limitation. In the SLAM problem, it is nowadays usual ([1], [2]) to model the non-normal robot pose with the particle approach. This has been proven successful in robots based on Laser Range Finders as the main sensing system. In many mobile robots applications (outdoor navigation, indoor navigation with ramps and/or obstacles not perceivable at the LRF height, door opening with door handle, etc.) such sensing system cannot be the only complex sensing system of the robot. On the other hand, a vision-based sensing system, which could leverage on the huge amount of computer vision algorithms, could be the main sensing system of a mobile robot. This is also pushed by economical constraints: extensive market analyses show that a complex sensing system for a mobile robot cannot cost more that 10US$, for a consumer-level robot [3]. Today a multi-camera system still does not match such target on cost, but the difference,
1-4244-0602-1/07/$20.IEEE.
Authorized licensed use limited to: Politecnico di Milano. Downloaded on January 27, 2010 at 18:46 from IEEE Xplore. Restrictions apply.

Fig. 1.

3D segment-based reconstruction for a trinocular stereo-system.
the system has to deal with segments since the very rst (image) processing steps. The segments are represented by the 3D coordinates of their extrema. The trinocular approach exploits the trinocular epipolar constraint to speed up the search for triplets corresponding 2D segments in the 3 images of the system. Our implementation differs from the original only in the use of the Fast Line Finder [10], in the polygonal approximation phase. The last step, the most interesting for this work, is the computation of the parameters of the 3D segment, represented by the 3D coordinates of the endpoints. In Fig. 1, D is the 3D scene segment, Ci and di are respectively the projection center and the projection of D on image i. The cameras are calibrated with a standard DLT technique, which models the uncertainty on the camera projection matrix as normally distributed. The 2D segments are also measured in the image space as normally distributed. The system therefore produces the 3D extrema of each scene segments as well as an associated covariance matrix, intended to represent the measurement uncertainty of the 3D data as a normal probability distribution. As making evident the limitations of modeling the data uncertainty with a normal distribution is the aim of this work, we developed a particle approximation for the uncertainty of the 3D segments. The normal distributions (expected value and covariance matrix) of the independent variables, i.e. the projection matrixes and the 2D segments on the image plane, are used to generate the samples to which we apply the triangulation process. 3D reconstruction systems based on stereo-vision generate data affected by uncertainties that strongly depend on the observer-to-feature distance. This is due to the triangulation relationship. This uncertainty is relevant in order to correctly associate different measures of the same world feature. Lets consider, for the sake of simplicity, the binocular stereosystem in the plane as in Fig. 2; two cameras are placed at known positions (baseline distance = b). A point N projects in the images; we measure its projections at xN and xN. L R xN and xN are affected by noise, due to many sources, L R e.g. space and intensity image quantization, etc. The noise on xN and xN , in turn, cause the location of N to be L R reconstructed with error. The uncertainty induced by the triangulation, i.e. the intersection of the two interpretation lines of xN and xN , is not a simple scalar function of L R the distance to N, it is also skewed and oriented. Nearby

Fig. 2.

Uncertainty diamonds for points at different distances
(to the observer) points are affected by a fairly compact uncertainty distribution, whereas farther points are affected by an elongated uncertainty distribution, that is roughly aligned with the line of sight to the point. We want to evaluate and represent this uncertainty, in order to take it into account in any further reasoning, e.g. in a SLAM system. A. Gaussian Modeling The image coordinates and the projection parameters are here considered as corrupted by a normally distributed noise. The parameters of the distribution have to be determined. As triangulation is a non-linear operation, the distribution is nongaussian, but in the most used approach it is approximated to a normal. This makes much easier dealing with the errors in the subsequent phases, e.g. data association and integration of the different measures. This approach dates back to [4] and to the work of O. Faugerass group at INRIA. Let L = [xL , yL ]T and R = [xR , yR ]T be the projections in the left and right images, respectively. Their coordinates are normally distributed random vectors with mean L and R , and covariance matrices VL and VR , respectively X = b(xL + xR )/(2 (xL xR )) Y = b(yL + yR )/(2 (xL xR )) N = f tri (L, R) = Z = b/(xL xR )
(1) where b is the baseline, and a unit focal length is assumed. Although f tri is nonlinear the approach stipulates that N can be approximated by a normal distribution with average N = f tri (L, R) and covariance computed basing on J, the Jacobian of f tri. VN = J|L,R Vl Vr J|T L,R (2)
B. Particle-based Modeling We sample the normally distributed independent variables (projection parameters and 2D features) with n particles. We

Map View

Fig. 4.
The moving window problem.
Fig. 3. Shape of the distributions (normal in dark and the unknown in light grey) w.r.t the position of N; baseline is from 0 to 500
then propagate the (2D) particles through the triangulation process as if they were 2D image points. We can exploit (1) for each sample. The results are shown in Fig. 3. Note that the differences, between the gaussian approximations and the sampled ones, increase as N moves away from the observer. IV. DATA A SSOCIATION The key point of the SLAM process, from a robustness perspective, is data association. The goal of such activity is to map uncertain information coming from sensors to uncertain information in the map. In our case the data association algorithm has to map 3D segments in the current view with 3D segments in the current submap. A correct association is obtained when the algorithm is capable to map a pair of segments (one in the view and one in the map) corresponding to the same real segment, perceived from different points of view. The two different approaches in uncertainty modeling (i.e., gaussian approximation and our proposal of particle-based modeling), can be applied. We claim our proposal increases the data-association success rate. In both approaches the association is based on three geometric criteria, for nding matches between view and map. These criteria use a Mahalanobis distance to evaluate the matches, and has been described with more detail in [11]. A. Gaussian Data Association Method In the following we introduce a simple formalization of the data association, to be used in the rest of the paper. Suppose we measured a set of features V = {vi } from a certain robot position. Each feature is represented by its coordinates and its uncertainty. We have also the set of features M = {mj } observed and integrated in the map up to that moment. The purpose of a data association algorithm is to nd a proper hypothesis of correspondences between them (list of pairs vi and mj ) Hk = {< v1 , mj1 >, < v2 , mj2 >,. , < vV , mjV >}. To generate this hypothesis we must explore an interpretation tree, where each node represents a pair of corresponding features < vi , mj >. In [11] we discussed

some algorithms for data association and the importance of using proper criteria to match features in the view with features in the map. Usually the point-to-point (extremum-toextremum) distance is considered as an appropriate criterium for single segment matching and much of the effort is devoted in nding a good association strategy for dealing with the exponential complexity of nding the best match for the whole view. In that paper we showed how better criteria for 3D segment matching result in a better data association, almost independent from the algorithm for interpretation tree traversal (i.e. data association). The approach we proposed is based on a multi-criteria evaluation for associating segments in the View with segments belonging to the Map. The reason for the poor performance of the point-to-point criterium is mainly due to the problem of moving-eld-of-view in the sensing system, which turns in a moving window on the world feature, see Fig. 4. The segments extrema could be due to the reduced eld of view and not always related to real extrema in the world; this can easily become a problem for the classical point-to-point distance1. The vi segment is moved into the Map reference frame by means of an estimate of the robot pose, e.g. the one provided by odometry. Such estimate is known altogether with an estimate of its uncertainty, which is propagated to the segment in the Map reference frame. The data association criteria are based on uncertain Mahalanobis distance. We propose three criteria to validate an hypothesis of match between vi and mj : 1) 1st criterium: Mahalanobis distance between the support line of the map segment and the extrema of the view segment; 2) 2nd criterium: Angle between the support lines of the two segments: this constraint complements the previous criterium and it is also based on a statistical test that considers the uncertainty in the angle computation; 3) 3rd criterium: this is not a probabilistic one; it is related to the projections of the segment extrema on the other segment; one of the following two conditions has to hold to get an association: a) at least one of the extrema of the view segment has to project onto the map segment; b) both the extrema of the view segment project outside the map segment, but the extrema of the
1 Our proposal is of interest also for 2D-3DoF SLAM systems which groups data points into 2D lines, because this moving-eld-of-view issue applies there too.
map segment project inside the view segment. At the end of the process we have a list of possible matches between vi and mj. We consider only one possible match (mj ) for each vi segment and use two algorithms to perform this task: Multi-Criteria Nearest Neighbor and Multi-Criteria Joint Compatibility. Multi-Criteria Nearest Neighbor is a variation on the nearest neighbor data association algorithm that bases on the classic ICP (Iterative Closest Point) algorithm [12] and the multi-criteria approach. Moreover, it orders the list of all potential matches, after each association decision. We call this approach Full-Order (FO), and the whole NNFO. The Multi-Criteria Joint Compatibility (JCBB) integrates the three criteria in the joint compatibility branch and bound technique [13]. To reduce the computational time required to explore one branch of the interpretation tree, we rst compute the 3rd criterium on the couple < vi , mji > because it is very selective and fast. We then compute the 2nd criterium on the same couple, to decrease the possibility to compute in vain the next step. Finally, we use the 1st criterium, in the joint compatibility test. B. Particle-based Association Method Our claim in this paper is that a further improvement in data association can be obtained using a proper uncertainty modeling; to support such claim, we modied the data association algorithm to consider a particle-based model of the uncertainty in the sensed data. Particles represent hypothesis for the real position of 3D extrema of reconstructed segments; each particle, in this approach, represents a single possible extremum of the segment and it is known precisely. We sample a particle from the two distributions, of the two k extrema. This pair of samples is the vi sample for segment k vi. We move then vi into the map reference frame (as for the previous method) by means of an estimate of the robot pose, e.g. odometry. The uncertainty on the particle depends only on the error on the robot pose, which we model with a gaussian distribution. This uncertainty is computed with a classic error propagation. This approach contrasts stateof-the-art SLAM methods, that use particles to model pose uncertainty. We now apply the three criteria to each particle-segment k k vi , w.r.t. all the map segments. If vi passes the tests with a map segment mj , then a specic counter for mj is k incremented. If the number of particle-segments vi for some mj is larger than a threshold, then the match < vi , mj > is admissible. The process is repeated for all view segments. This approach can be seen as the empirical computation of the matching probability integral using particles distribution. At the end a full-order algorithm is executed to extract a list of valid matches: 1) for all mj from Mj and vi from Vi a) sample a set of K particles for each extrema of k vi and move these segments vi s into the Map reference frame k b) compute the number of vi that pass the 3 tests; this count is then compared with a threshold (), relative to the total number of particles for vi ; if larger, then vi can be associated to mj order the matches < vi , mj > in decreasing order select as match the st pair < vi , mj > erase any other pair that involves vi or mj from the list, and order the list repeat from the selection of the best match until no match is possible

2) 3) 4) 5)

The method (PFO) is therefore depending on n, the number of particles, and the threshold (PFOn V. E XPERIMENTAL ACTIVITY For the experimental activity we used a mobile robot from Robosoft, which computes odometry as a 3DoF pose [x, y, ]T ; this datum reaches a PC via serial line. On the PC we have an Eltec frame grabber capable to grab three 704x558x8 pixels images at the same time. Each channel of the frame grabber is connected to a Sony XC75CE camera. Cameras have been calibrated with a standard DLT approach. The robot has been moved inside the 4th oor of building U7, Univ. Milano - Bicocca, Milano, Italy. The robot travelled about 100m, activating its perception system every few centimeters, for a total of 1000 views (activations), with about 15 segments per view. To generate the 3D particles (segments extrema) we sampled 100 or 1000 (depending on the experiment) particles from the normal uncertainty of each 2D extremum. This is repeated for the 6 extrema (3 segments, 1 per image); we also sample the projection matrix from the 11-variate normal given out by our calibration tool. Next, we compute the 3D segment extrema with the triangulation process. This procedure is repeated 100 or 1000 times, for each extremum of each 3D segment. Subsequently, we use this information to reconstruct the scene with a SLAM process. We need to clarify what happens currently, when a view segment vi is associated to a map segment mj , in order to integrate, i.e. fuse, the 2 measures. If both vi and mj are modeled with normal distributions, then the usual fusion applies. On the other hand, when vi is represented by a particle distribution, we use the st two moments of it to update the normal distribution of mj. Table I shows the comparison between the different dataassociation methods. The leftmost (NNFO) is a frequently used approach, based on normal distributions to describe the extrema of the segments. The second is the state-of-theart JCBB, again with normal distributions. The others are based on the new method, with distributions approximated by particles. PFO100 stands for 100 particles per extremum, PFO1000 for 1000 particles. The percentage is the threshold mentioned before (), i.e. the minimum number of particles, k w.r.t. the total number of vi , which have to match with mj , in order to associate vi to mj. Being un-available the ground truth of the explored environment, we performed the comparison looking at the number of matches generated during the view-integration phase, before any loop-closure. Also the number of submaps,

TABLE I 1000 VIEWS , NO LOOP CLOSURE. # matches # submaps time (min) NNFO (norm) 30 JCBB (norm) 62 PFO100 50% 201 PFO100 10% 189 PFO1000 50% 1740 PFO1000 10% 1980
created during this exploration, is reported because it contributes to shed light on the amount of correct associations performed by each approach. It has to be stressed that by not caring of data-association problems the overall map results into larger number of false segments, all apparently independent, but actually referring to the same scene features. On the other hand, approaching the issue by simply increasing the normally modeled uncertainty turns into a large number of wrong associations. We are very excited about the results obtained with our proposal and do not think the processing time to be really relevant at the moment, given that current techniques were indeed unable to reach the goal of building a good map. Nevertheless, we reported the time taken for the processing on a P4 laptop, starting from images on les, and including stereo-matching, just for giving an idea on this aspect of our preliminary results. We also note that particle approaches can be easily and effectively implemented on parallel architectures.

Fig. 9.

Two reconstructed scene segments

Fig. 10.

Segment 1, extremum 2; see text for details.

Fig. 5.

A view of the map obtained with normal modeling of uncertainty
Fig. 6. A view of the map obtained with particle modeling of uncertainty
By looking at the table we can observe that our proposal allows about 34% more matches w.r.t. NNFO (norm) and about 14% w.r.t. JCBB (norm). If looking at the number of submaps (size limit was set at 50 features for all approaches) the reduction appears even larger: about 26% w.r.t. NNFO (norm) and about 36% w.r.t. JCBB (norm).
The effectiveness of the approach can be further appreciated by looking at Fig. 5 and 6; they show a similar view on the NNFO (norm) and PFO1000 10%, respectively, 3D reconstructions. Notice the large reduction in the number of un-matched segments when going from the normal to the particle approach. A view from the top of the two maps is presented in Fig. 7 and 8. In order to support our claim about the need for accurate modeling of uncertainties in the sensed data we present a couple of real extrema; in Fig. 9 notice 2 scene segments: the green endpoints have coordinates [2.9, 8.6, 2.8], [1.6, 9.2, 2.9], while the extrema of the red are [1.4, 9.0, 2.8], [0.8, 9.0, 2.8]. Fig. 10a shows the uncertainty (in the x,y plane) for an extremum of the green segment, using the particle model, while the Fig. 10b shows the comparison between this uncertainty model and the uncertainty computed using the Jacobian method, along the line connecting the extrema of the uncertainty diamond. The direction of the arrow, in the left gure, corresponds to direction of the abscissa in the gure on the right. Fig. 11a and b presents the same for the rst extrema of the red segment. Notice the differences between the shape of the normal and the sampled distributions.

Fig. 7.

A view from the top of the map (normal modeling of uncertainty)

Fig. 8.

A view from the top of the map (particle modeling of uncertainty)

R EFERENCES

[1] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics. MIT Press, 2005. [2] M. Montemerlo, S. Thrun, D. Koller, and B. Wegbreit, Fastslam 2.0: An improved particles ltering algorithm for simultaneous localization and mapping that provably converges, in Proceedings of the Sixteenth International Joint Conference on Articial Intelligence, 2003. [3] C. Angle, Inv. talk at euron meeting, Amsterdam, March 2004, the author is CEO of iRobot inc. [4] L. Matthies and S. Shafer, Error modeling in stereo navigation, IEEE Journal of Robotics and Automation, vol. RA-3, no. 3, pp. 239248, June 1987. [5] S. Julier and J. Uhlmann, A new extension of the kalman lter to non-linear system, in Proceedings of International Symposium on Aerospace/Defence Sensing, Simulate and Controls, 1997. [6] P. Newman and H. Durrant-Whyte, A new solution to the simultaneous localization and map building (slam) problem, in Proceedings of SPIE, 2001. [7] D. Blackwell, Conditional expectation and unbiased sequential estimation, Annals of Mathematical Statistic, vol. 18, pp. 105110, 1947. [8] J. Guivant, E. Nebot, J. Nieto, and F. Masson, Navigation and mapping in large unstructured environments, International Journal of Robotics Research, vol. 23, 2004. [9] N. Ayache, Articial vision for mobile robots. MIT Press, 1991. [10] P. Kahn, L. Kitchen, and E. Riseman, A fast line nder for visionguided robot navigation, IEEE Transaction on PAMI, vol. 12, no. 11, November 1990. [11] D. Marzorati, D. G. Sorrenti, and M. Matteucci, Multi-criteria data association in 3d-6dof hierarchical slam with 3d segments, in Proc. of ASER06 Conf., 2006. [12] P. J. Besl and N. D. McKay, A method for registration of 3-d shapes, IEEE Trans. Pattern Anal. Machine Intell., vol. 14, pp. 239256, Feb. 1992. [13] J. Neira and J. D. Tard s, Data association in stochastic mapping o using the joint compatibility test, IEEE Trans. R&A, vol. 17, no. 6, pp. 890897, 2001.

Fig. 11.

Segment 2, extremum 1; see text for details.

VI. C ONCLUSIONS In this paper we presented a particle-based approach to uncertainty modeling of 3D data sensed through a stereo vision system. Our claim in doing this is that it is possible to obtain a better data association exploiting such modeling, with respect to the classical gaussian approximation. Impressive results have been obtained on real data, conrming that a proper uncertainty model is a key factor in data association and in SLAM in general. It is worth noticing that this approach apparently contrasts with the state of the art in SLAM algorithms, which usually base on a non gaussian (particle-based) estimate of the robot pose, while perceptions are treated as normally distributed. We are currently working to reduce the time required to compute a particle-based representation and to improve the speed of the method.