Full Story - Voice Recognition Software Consumer Report
Source: http://www.consumersearch.com/www/software/voice-recognition-software/fullstory.html Dated: October 2005 NB: Prices listed are quoted in US Dollars (USD) in this article.
Review The best reviews of voice-recognition software compare the two market leaders, ScanSoft Dragon NaturallySpeaking 8 and IBM ViaVoice 10 (now also marketed by ScanSoft). Reviews from Federal Computer Week, PC Magazine and The Boston Globe top our All Reviews chart as the most credible reviews we could nd in this category. These publications get picky with each title, testing for accuracy, speed and ease of use. This is no small consideration when it takes up to thirty minutes just to get voice-recognition software up and running -- more if you want to improve accuracy. All three reviews say Dragon NaturallySpeaking easily trumps IBM ViaVoice in both areas. Most reviews agree that voice-recognition software (VRS) is initially time consuming, but accuracy and ease of-use have now improved so much that it may nally be worth the investment of time and money. If you have to choose one program, reviews agree ScanSofts Dragon NaturallySpeaking 8 (*est. $90 to $700 USD, depending on version) is best. Its accurate, intuitive, easy to use and consumes less of your computing resources in comparison to its competitors. According to a review by Cade Metz in PC Magazine, Speech recognition has never been this good. The fact that Dragon NaturallySpeaking wins out over IBMs ViaVoice is especially ironic since both products had early development in IBMs Yorktown Heights, NY, research facility. IBM let some employees out of the tent, and they formed Dragon to eventually beat the giant in a quality battle. Until the introduction of MacSpeechs iListen (*est. $85 to $130 USD, depending on version), IBM ViaVoice for Mac (*est. $115 USD) was the only voice-recognition software available to Macintosh users. In the reviews we found, testers prefer iListen to IBM ViaVoice for Mac. Charles W. Moore, contributing editor to Applelinks.com, says that iListen 1.6.8 bests ViaVoice because its easier to use and install. Moore says iListen is better for entering text in any application -- not just a word processor -- than ViaVoice. In a review for Macworld, Dennis Sellers also likes this feature (called TalkAnywhere), which lets you dictate anywhere youd normally type. iListen comes in two versions. The pricier version of iListen 1.6.8 (*est. $130 USD) comes with a headset, and the standard iListen 1.6.8 (*est. $85 USD) is just the software. The latest version revision works with OS X Panther Dragon NaturallySpeaking is easily the best voice-recognition software Voice recognition software (VRS), also known as speech-recognition, automatic speech recognition, ASR or natural language recognition software, converts your voice to text on a computer. In essence, what this means is that you can create text les without typing. When you speak through a microphone (most voice-recognition software includes this accessory), the software translates the sounds into written words. Many rst-generation voice-recognition packages used discrete speech technology, which meant you had to pause between words in order for the computer to understand them. The latest generation uses continuous speech technology, which allows you to speak more naturally. All require an enrollment process, during which you sit at the computer and read sample text out loud to help train the speechrecognition software to understand your voice patterns. While most voice-recognition software users want a large vocabulary in their software package, the larger the vocabulary, the more time-consuming
the training process. Thats because many words sound similar, and you need to train the computer to understand how you pronounce them. Dragon NaturallySpeaking claims a 99% accuracy rate in optimal conditions, though reviews say 95% is probably closer to reality. In a review at PC Magazine, IBM ViaVoice reached only a 70% accuracy rate, and in tests at The Boston Globe, reviewer Naomi Aoki simply didnt get the same comic misinterpretations with NaturallySpeaking as she did with ViaVoice. With Dragon NaturallySpeaking 8, you can use natural language strings (e.g., print le, open browser) to edit documents and work around your desktop. Reviews say Dragon integrates easily with leading Windows applications, including Microsoft Word, Outlook Express, Internet Explorer and AOL Instant Messenger. Dragon NaturallySpeaking 8 is available in ve different versions, all of which contain a microphone in addition to the software. The two least expensive versions are suitable for most users. Dragon NaturallySpeaking 8 Standard (*est. $90 USD) is designed for personal use, and you can transcribe into browsers, AOL Instant Messenger, Outlook Express, MS Word and WordPerfect word processors. Dragon NaturallySpeaking 8 Preferred (*est. $170 USD) adds compatibility with Microsoft Excel, as well as support for digital audio recorders. With a compatible device, you can dictate into a small recorder, and download the transcribed text le into your computer. Very limited macro creation tools are included within Preferred; macros let you create combined functions that can be executed by a single command. Dragon NaturallySpeaking 8 Professional (*est. $700 USD) adds support for MS Outlook, PowerPoint and Lotus Notes. The Professional version also provides complete hands-free use of a PC, including mouse control by voice. Dragon NaturallySpeaking 8 Medical (*est. $950 USD) and Dragon NaturallySpeaking 8 Legal (*est. $950 USD) are each designed specically for the medical and legal worlds. They allow professionals to streamline the transcription process, thereby saving time and money. Each comes with expanded vocabularies specic to those professions. The medical version can be used with any of thirteen different specialties. IBMs ViaVoice 10 offers three different levels of voice-recognition ability. The Standard version (*est. $45 USD) provides dictation and basic command and control features, along with simple Internet navigation. The Advanced version (*est. $70 USD) provides support for some PDAs, dictation and command and control for many applications, including the Ofce XP suite. Macro voice shortcuts are introduced in the Advanced version along with natural language commands and a voice mouse. The top of the line ViaVoice Professional (*est. $160 USD) offers more customized templates and macro voice shortcuts. But according to reviews, IBMs ViaVoice lags behind Dragon NaturallySpeaking in overall accuracy and breadth of functions. Interestingly, ScanSoft bought the rights to ViaVoice, so Dragons parent company now owns the competition. Well be interested to see if ScanSoft continues to market ViaVoice alongside Dragon NaturallySpeaking. Experts say theres just no doubt that Dragon is the better product, at least for Windows users. Charles W. Moore reports in Applelinks.com that the IBM ViaVoice development team was ofcially disbanded in April 2003. If you blanch at paying so much for Dragon NaturallySpeaking voice-recognition software, you should know that if you have Windows Ofce XP, your computer already has some speech-recognition capacity of its own. Microsoft is careful when describing its speech recognition function, saying its most likely to benet users in China and Japan who need to create documents and e-mails in English, since writing in another alphabet normally involves employing a dizzying array of menus and character lists. For Englishspeaking users, Microsoft notes, Most skilled typists will nd that dictation for U.S. English is unlikely to be better than typing, and Ofce XP does not address some of the more advanced requirements of the accessibility community. While it doesnt sound like Microsoft is lining itself up to compete with NaturallySpeaking or ViaVoice, if you have MS Ofce XP installed on your computer, youll get to try some voice recognition for free. Ofce XP speech recognition works pretty much like other voice-recognition software; you need to put in some training time with a microphone for optimal performance. For the assisted technology market, a product from Commodio, Inc. shows great promise and provides interesting competition to the current voice-recognition market leaders. QPointer Voice (*est. $180 USD) isnt so much a voice-recognition product as it is a voice-operated navigation tool. Even Yizhar Hon,

SPECTRONICS ~ PO Box 88, Rochedale QLD AUSTRALIA 4123 Ph: (07) Fax: (07) Email: mail@spectronicsinoz.com Web: www.spectronicsinoz.com
Director of Sales and Marketing at Commodio admits that Dragon NaturallySpeaking gets much better dictation accuracy. QPointer Voice differs from standard voice-recognition technology because the software itself analyzes what is displayed on the monitor and intuitively identies those objects. The user can then navigate through the screen by verbally pointing at those words, graphics and controls. QPointer Voice also allows the user to use any combination of the mouse, keyboard and voice controls. This means users can Web surf, click on links with voice commands, and navigate the Internet hands-free -- applications that are not the primary strengths of Dragon NaturallySpeaking or ViaVoice software. Ultimately, the creators of QPointer hope that their software can be integrated with existing speechrecognition software to create both accurate dictation and hands-free Web surng. But right now, this product is in its infancy. If you are more interested in hands-free Web navigation and Internet functionality, QPointer is an innovative product. If you are mainly interested in dictating documents to avoid typing, however, Dragon NaturallySpeaking is a better choice. (Note from Spectronics at May 2006 - QPointer Voice is no longer available in Australia) A rising number of shareware packages are also available on the market, including Voice Studio (*est. $30 USD) and Realize Voice Lite 4.0 (*est. $50 USD). Both of these are available for download at CNets Download.com. It remains to be seen whether or not they will offer any signicant competition to the mainstream products, and we did not see any reviews of these packages. Voice recognition for PDAs and phones There are a couple of products on the market that work with PDAs and smartphones. Microsoft Voice Command (*est. $35 USD) lets owners of PocketPC PDAs and smartphones (running Windows Mobile 2003 and Windows Mobile 2003 Phone Edition) to perform many hands-free functions, such as looking up contacts and calendar information, operating MP3 and media-player functions and voice dialing. While owners say it works well enough, Voice Command cant transcribe much of anything. It cant transcribe e-mail or text messages, for example. Reviews at CNet.com say it does work well for its intended functions, however. VoiceSignal VoiceMode is included in a handful of Samsung cell phones, including the Samsung P207 (for Cingular). You cant purchase VoiceMode separately, but if you own (or buy) one of these phones, you can use VoiceMode to transcribe text and e-mail messages in addition to hands-free dialing and other voice commands. Dragon NaturallySpeaking Preferred comes closest to PDA voice-recognition, but you still cant use it to dictate e-mails or text messages. You can use your Tungsten or PocketPC PDAs voice recorder, then later, when you sync with your computer, the voice data will be transcribed. Were still waiting for a voice-recognition product that will let users dictate e-mails or text messages into a PDA or smartphone. Important voice-recognition software features When considering which speech-recognition package is right for you, experts say to take the following into account: Look for a vocabulary of at least 150,000 words, as well as the ability to add at least another 75,000 words of your own. Make sure the package will work with the software programs you use most frequently (Microsoft Word, Corel, etc.). For instance, the Standard version of Dragon NaturallySpeaking wont work with Excel; for that you need the Preferred version. Only the $700 (USD) Professional version works with MS Outlook, PowerPoint and Lotus Notes. Its often easier to use common sense (i.e., natural language) phrases like insert here for editing purposes, rather than more complicated commands. The phrase natural or free indicates this feature. Training the software to recognize the way you speak can be tedious. Look for an easy and quick enrollment (ideally less than 15 minutes) that can be repeated as often as you need to improve accuracy. The more time you spend teaching the software, the higher the accuracy rate.

Make sure the headset is comfortable and the microphone is in a position that works for you rather than for the software. Good microphones take into account a normal level of background noise. Consensus Report Our Consensus Report shows how many times products are top-ranked by reviewers included in our All The Reviews Reviewed chart. # of Picks 1 Model Dragon NaturallySpeaking 8 (Standard, Preferred, Professional, Medical) MacSpeech iListen QPointer Voice* Prices from Shopping.com est. $90 to $950 USD depending on which version est. $85 to $130 USD
*(Note from Spectronics at May 2006 - QPointer Voice is no longer available in Australia) The numbers in our chart point out the obvious: Dragon, the company thats always held the lead in the consumer end of the industry, is still tops in reviews. Manufacturers and Resellers Web sites: Nuance (Dragon NaturallySpeaking and IBM ViaVoice): http://www.nuance.com/ Voice Perfect Systems (Dragon NaturallySpeaking and IBM ViaVoice): http://www.voiceperfect.com.au MacSpeech (iListen): http://macspeech.com/ Microsoft Voice Command: http://www.microsoft.com/windowsmobile/downloads/voicecommand/default.mspx Vista Voice Recognition: http://blogs.technet.com/chenley/archive/2006/02/21/420136.aspx http://www.pro-networks.org/forum/viewstory.php?t=75117 http://www.tau.ac.il/~itamarez/sr/Vista-vs-Xp.htm This article was prepared by ConsumerSearch - Who are they? ConsumerSearch is a Web-based publishing company that helps consumers nd answers about which products are top-rated or best bets in their class. We try to provide consumers fast answers on top-rated products and provide an intelligent search engine to the best articles on any given product or service. Our mission is to reduce to minutes the time it takes consumers to make an intelligent purchase. Our process begins when our editors scour the Internet and print publications for comparative reviews and other information sources relevant to the consumer. We then analyse these reviews and sources and rank them according to how credible they are, based on specic criteria we have developed for this purpose (see below for more on these criteria). Our website is partly a search tool (see our All The Reviews Reviewed page) and partly a consolidator of wisdom and analysis (see our Editors Report and Fast Answers sections). Use whichever of these tools you nd most useful. While we accept advertising to fund our site, our objectivity is built into our core process of reporting what the reviewers say. We make our process transparent to users, who are free to examine our sources. Consumer Reports has popularised the notion that a publisher cannot be objective if it takes advertising. We appreciate this point of view, and there is good reason to be cautious about the inuence advertisers can have on publishers. That is one of the reasons we rigorously evaluate reviews.

University of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 1: Introduction & Decision Rules
Mark Gales mjfg@eng.cam.ac.uk Michaelmas 2010
1. Introduction & Decision Rules

Syllabus

1. Introduction & Bayes Decision Theory (1L) Statistical pattern processing Bayesian decision theory Classication cost & ROC curves 2. Multivariate Gaussians & Decision Boundaries (1L) Decision boundaries for Multivariate Gaussians Maximum likelihood estimation 3. Gaussian Mixture Models (1L) Mixture models Parameter estimation EM for discrete random variables 4. Expectation Maximisation (1L) Latent variables both continuous and discrete Proof of EM 5. Linear Classiers (1L) Single layer perceptron Perceptron learning algorithm 6. Multi-Layer Perceptrons (2L) Basic structure Posterior distribution modelling Gradient descent parameter optimisation
Engineering Part IIB: Module 4F10 Statistical Pattern Processing

Syllabus (cont)

7. Support Vector Machines (2L) Maximum margin classiers Training SVMs Kernel functions & Non-linear SVMs 8. Gaussian Processes (2L) Covariance functions Non-linear regression Gaussian processes for classication Relevance Vector Machine 9. Classication and Regression Trees (1L) Decision trees Query selection Multivariate decision trees 10. Non-Parametric Techniques (1L) Parzen windows Nearest neighbour rule K-nearest neighbours 11. Application: Speaker Verication/Identication (1L) Speaker recognition/verication task GMMs and MAP adaptation SVM-based verication

Overview of Course

Generative Models
Multivariate Gaussian Gaussian Mixture Model
Discriminative Classifiers
Discriminative Functions Perceptron Algorithm Support Vector Machine Discriminative Models Logistic Classification Multilayer Perceptron Gaussian Processes

NonParametric

Decision Trees Parzen Windows Nearest Neighbour

Training Criteria

Maximum Likelihood Expectation Maximisation Maximum Margin

Course Structure

Total of 14L + 2 Examples Classes Lecturer: Mark Gales Web-Page: http://mi.eng.cam.ac.uk/mjfg/4F10/index.html Assessment by exam (1.5h): 3 questions from 5. A number of books cover parts of the course material. C.M.Bishop, Pattern Recognition and Neural Networks OUP, 1995, CUED: NOF 55 R.O.Duda, P.E.Hart & D.G. Stork Pattern Classication, Wiley, 2001, CUED: NOF 64 D.J.C. Mackay, Information Theory, Inference and Learning Algorithms, CUP, 2004. CUED: NO 277 C.M. Bishop, Pattern Recognition and Machine Learning, Springer 2006.
Statistical Pattern Processing
In this world nothing can be said to be certain, except death and taxes. - Benjamin Franklin We make decisions under uncertainty all the time gambling (not recommended) weather forecasting (not very successfully) insurance (risk assessment) stock market Need to formalise intuitive decisions mathematically Basically, how to quantify and manipulate uncertainty. This course will concentrate on classication, however regression and clustering will be briey mentioned. A range of statistical approaches to decision making will be examined: approaches can be trained (hence Machine Learning) wide range of applications, examples are.
Automatic Speech Recognition
ViaVoice is a dictation package converts spoken words into text surprisingly hard to do (worth trying this.)

Information Retrieval

viavoice image - Google Search http://www.google.co.uk/search?q=viavoice+im.

Sign in

Google

Images

MapsNew!

Products

Groups

viavoice image

Search: the web View and from the UK web history pages manage your New! Results 1 - 10 of about 315,000 for viavoice image. (0.23 seconds) Sponsored Links

IBM ViaVoice 10

Sponsored Link www.Nuance.co.uk Dictate, edit and correct text with your voice. Official site.

Picture

Picasa creates amazing pictures! See demo, download free from Google picasa.google.co.uk
Image: IBM ViaVoice 10.0 Standard Edition: ScanSoft
Image: IBM ViaVoice 10.0 Standard Edition: ScanSoft by Viavoice ScanSoft. Save On Viavoice www.amazon.co.uk/ Fast Shipping. IBM-ViaVoice-10-0-Standard-Edition/dp/images/B00006IIP2 Order Online Now. www.dabs.com - 31k - Cached - Similar pages
Buy It Cheap On eBay Low Prices, New and Used Image: IBM ViaVoice 10.0 Pro USB Edition with ebay.co.uk Headset: ScanSoft by ScanSoft. www.amazon.co.uk/ Viavoice at Amazon.co.uk IBM-ViaVoice-10-0-Pro-Headset/dp/images/B00006IIP1 Get your Software at Amazon.co.uk - 31k - Cached - Similar pages Free Delivery on orders over 15 [ More results from www.amazon.co.uk ] www.amazon.co.uk/software Compare Prices on Software! Great Image: IBM VIAVOICE Advanced 10.0. IBMDeals - Buy Advanced 10.0. VIAVOICE & Save Online Today. Close window. www.kelkoo.co.uk/Software www.amazon.ca/ H009A-G00-10-0-IBM-VIAVOICE-Advanced-10-0/dp/images/B0000A58IW - 29k - Cached - Similar pages
Image: IBM ViaVoice 10.0 Pro USB Edition with Headset: ScanSoft

viavoice

Image: IBM VIAVOICE Advanced 10.0

Viavoice

Image: IBM ViaVoice Standard v.10
Image: IBM ViaVoice Standard v.10. IBM ViaVoice Standard v.10. Close window. www.amazon.ca/ H109A-G00-10-0-IBM-ViaVoice-Standard-v-10/dp/images/B0000A58IV - 31k - Cached - Similar pages [ More results from www.amazon.ca ]
Image: IBM ViaVoice Pro USB Edition
Image: IBM ViaVoice Pro USB Edition. IBM ViaVoice Pro USB Edition. www.amazon.com/IBM-ViaVoice-Pro-USB-Edition/dp/images/B0000A58IX - 32k Cached - Similar pages
Search engines are essential

1 of 2

17/09/07 14:54
query to nd previous image viavoice image determine which pages to return what adverts to include (Google needs to make money)

Basic Model

Feature Extraction Pattern e.g. Speech Signal

Classification

Symbol e.g. Word
Feature Vector e.g. Spectral Components
Initial feature extraction produces a vector of features that contain all the information for subsequent processing (such as classication). Ideally, for classication, only the features that contain discriminatory information are used. Often features to measure are determined by an expert, although techniques exist for choosing suitable features. The classier processes the vector of features and chooses a particular class. Normally the classier is trained using a set of data for which there are labelled pairs of feature vectors / class identiers available.

Simple Speech Features

Speech Waveform

0.8 0.6 0.4 0.2

Amplitude

0 0.2 0.4 0.6 0.0

Samples

Power Spectrum

Log Amplitude

Frequency

Features for vowel classication may be the spectral shape or frequencies of peaks (formants)

Simple Vowel Classier

Select two vowels to classify with a linear decision boundary

Linear Classifier

Formant Two

Formant One

Most of the data is correctly classied but classication errors occur pronunciation of vowels vary from speaker to speaker pronunciation vary for a speaker as well!
Vowel Distributions Using Formants
PetersonBarney Vowel Data

4000 3500

Second Formant

First Formant

Vowel classes are reasonably separated (but some overlap!) using these features: could draw decision boundaries It is often useful to calculate the probability of a particular class (rarely is this 1/0!)
Some Basic Probability (Revision!!)
Discrete random variable x takes one value from the set X = 1 ,. , K We can compute a set of probabilities pj = Pr(x = j ), j = 1,. , K
We use a probability mass function P (x), to describe the set of probabilities. The PMF satises P (x) = 1,

P (x) 0

Continuous random variable: scalar x or a vector x. Described by its probability density function (PDF), p(x). The PDF satises

p(x)dx = 1,

p(x) 0
For random variables x, y, z need conditional distribution: p (x|y) = joint distribution p (x, y) marginal distribution p (x) =

p(x, y)dy p(x, y) p(y)

chain rule p(x, y, z) = p(x|y, z) p(y|z) p(z)

Forms of Classiers

General notation used in this course Observations: each observation consists of a d-dimensional feature vector, x. Classes (labels): each observation will belong to a single class, 1 ,. , K We need a classier that given an observation, x, correctly assigns it to a class,. Classiers can be split into three broad classes. In the rst two a mapping from observation to class can be inferred (the decision rule), the third directly estimates a mapping. Generative models: a model of the joint distribution of observations and classes is trained, p(x, ). Discriminative models: a model of the posterior distribution of the class given the observation is trained , P (|x). Discriminant functions: a mapping from an observation x to a class is directly trained. No posterior probability, P (|x), generated just the class label. See Bishop for a discussion of the merits of these.

Forms of training

Irrespective of the form of classier the classier will need to be trained. There are three basic forms of training: Supervised learning: for each of the observations, x, the correct class label, , is available Unsupervised learning: only the observation, x, is available Reinforcement learning: a set of rewards are associated with actions for each observation This course concentrates on supervised learning. The training data occurs in pairs. For a 2-class, binary, problem the training data would be (x1, y1),. , (xn, yn ) where yi =
1 if xi generate by class if xi generate by class 2
The total number of samples from class 1 will be labelled n1 and for class 2 n2.

Decision Rules

A sensible approach to design a decision rules for generative and discriminative models is to minimises the probability of error: P (error) = = P (error, x)dx P (error|x)p(x)dx
For a two class problem, the conditional probability of error, (i.e. the error probability, given a value for the feature vector), can be written as P (error|x) =
P (1|x) if we decide 2 P ( |x) if we decide 2 1
A decision rule that can minimise this conditional probability of error averaged over all samples is required. This leads to Bayes decision rule, which for a two class problem is Decide
Class 1 if P (1|x) > P (2|x); Class 2 Otherwise
Applying Bayes decision rule to multi-classes yields Decide argmax {P (j |x)}
For generative models the joint distribution is estimated. For Bayes decision rule the class posterior is required - this can be obtained using Bayes rule P (j |x) = p(x|j )P (j ) p(x, j ) = p(x) i p(x, i )
Bayes rule here computes the posterior probability of a particular class, P (j |x) using the likelihood of the data from the class conditional density p(x|j ). prior probability of the class j , P (j ) - this is the probability of the class before any data is observed. The denominator, p(x), is sometimes termed the evidence and is the probability density of the data independent of class. Bayes Rule is sometimes remembered as posterior likelihood prior

Why Generative Models?

Modelling the joint distribution p(x, ) is more complicated than estimating the posterior or a decision boundary. However classication tasks such as speech recognition commonly use generative models! Why use generative models all? Prior distributions easy to interprete. Estimating the class priors is normally performed by simply taking ML estimate from the counts e.g. n1/(n1 + n2). Class-conditional PDF easy to interprete. For speech recognition we can extract the portions of speech associated with a particular word and nd the best model for that segment. Parameters easy to interprete. For speech recognition easy to consider how to adapt the model parameters to a particular speaker.

Example

The gures below (from DHS) give hypothetical class-conditional pdfs for two classes: priors are P (1) = 2/3 and P (2) = 1/3, and the posterior distribution. PDFs:

Posteriors:

Probability of Error
For a 2-class problem the decision rule will split the observation space into two regions 1: observation classied as 1 2: observation classied as 2 P (error) = P (x 2, 1) + P (x 1, 2 ) = P (x 2|1)P (1) + P (x 1|2)P (2) = p(x|1 )P (1)dx + p(x|2)P (2)dx
The error regions for a two-class problem are shown below (from DHS). The decision boundary x is set to xB for minimum error.
Generative Model Decision Rule
For the two-class case the Bayes minimum decision rule can be written as P (1|x) > p(x|1 ) > P (2) < 1, < P (2|x) 2 p(x|2 ) 2 P (1) The rst is the ratio of the posteriors, the second the ratio of the likelihoods compared to the ratio of the priors. For multi-class problems, the posteriors of the K classes can be calculated P (1|x), P (2|x),. , P (K |x) and the largest selected, or use Decide argmax {p(x|j )P (j )}
since the RHS denominator of Bayes rule is independent of class and this is a frequent statement of Bayes decision rule for minimum error with generative models.

Cost of Mis-Classication

So far the decision rule has aimed to minimise the average probability of classication error. Recall that for the two-class problem, the Bayes minimum average error decision rule can be written as: 1 P (1|x) > < 1 P (2|x) 2 Sometimes, the cost (or loss) for misclassication is specied (or can be estimated) and different types of classication error may not have equal cost. C12 Cost of choosing 1|x from 2 C21 Cost of choosing 2|x from 1 and Cii is the cost of correct classication. The aim now is to minimise the Bayes Risk which is the expected value of the classication cost.

Minimum Bayes Risk

Again let the decision region associated with class j be denoted j. Consider all the patterns that belong to class 1. The expected cost (or risk) for these patterns R1 is given by R1 =

p(x|1)dx

The overall cost R is found as R = = =

Rj P (j ) Cij

j=1 i=2

p(x|i )dxP (j )

i=1 i j=1

Cij p(x|j )P (j )dx

Minimise integrand at all points, choose 1 so

C1j p(x|j )P (j ) <

C2j p(x|j )P (j )
In the case that C11 = C22 = 0 we obtain 1 C21P (1|x) > < 1 C12P (2|x) 2 Note that decision rule to minimise the Bayes Risk is the minimum error rule when C12 = C21 = 1 and correct classication has zero cost.

ROC curves

In some problems, such as in medical diagnostics, there is a target class that you want to separate from the rest of the population (i.e. it is a detection problem). Four types of outcomes can be identied (let class 2 be positive, 1 be negative) True Positive (Hit) True Negative False Positive (False Alarm) False Negative As the decision threshold is changed the ratio of True Positive to False Positive changes. This trade-off is often plotted in a Receiver Operating Characteristic or ROC curve. The ROC curve is a plot of probability of true positive (hit) against probability of False Positive (false alarm). This allows a designer to see an overview of the characteristics of a system.

ROC curves (Example)

Example 1-d data, equal variances and equal priors: the threshold for minimum error would be (1 + 2)/2.

Error Probabilities

0.2 0.18 0.16 0.14
Receiver Operating Characteristic Curve

True Positive

0.12 0.1 0.08 0.06

90 100

False Positive
Left are the plots of p(x|i) for classes 2 and 1. each value of x gives there is a probability for each outcome. for x = 4 the probabilities are shown Right is the associated ROC curve obtained by varying x (here % rather probability is given on the axis). curves going into the top left corner are good a straight line at 45 degrees is random