Voice of the Customers: Mining Online Customer Reviews for Product Feature-based Ranking
Ramanathan Narayanan Alok Choudhary Electrical Engineering and Computer Science Department Northwestern University 2145 Sheridan road, Evanston, IL, 60208, USA Abstract
Increasingly large numbers of customers are choosing online shopping because of its convenience, reliability, and cost. As the number of products being sold online increases, it is becoming increasingly difcult for customers to make purchasing decisions based on only pictures and short product descriptions. On the other hand, customer reviews, particularly the text describing the features, comparisons and experiences of using a particular product provide a rich source of information to compare products and make purchasing decisions. Online retailers like Amazon.com1 allow customers to add reviews of products they have purchased. These reviews have become a diverse and reliable source to aid other customers. Traditionally, many customers have used expert rankings which rate limited a number of products. Existing automated ranking mechanisms typically rank products based on their overall quality. However, a product usually has multiple product features, each of which plays a different role. Different customers may be interested in different features of a product, and their preferences may vary accordingly. In this paper, we present a feature-based product ranking technique that mines thousands of customer reviews. We rst identify product features within a product category and analyze their frequencies and relative usage. For each feature, we identify subjective and comparative sentences in reviews. We then assign sentiment orientations to these sentences. By using the information obtained from customer reviews, we model the relationships among products by constructing a weighted and directed graph. We then mine this graph to determine the relative quality of products. Experiments on Digital Camera and Television reviews from real-world data on Amazon.com are presented to demonstrate the results of the proposed techniques.

Kunpeng Zhang

1 Introduction
The rapid proliferation of Internet connectivity has led to increasingly large volumes of electronic commerce. A study conducted by Forrester Research[1] predicted that e-commerce and retail sales in the US market during 2008 were expected to reach $204 billion, an increase of 17% over the previous year. More customers are turning towards online shopping because it is convenient, reliable, and fast. It has become extremely difcult for customers to make their purchasing decisions based only on images and (often biased) product descriptions provided by the seller. Online retailers aim to provide consumers a comprehensive shopping experience by allowing them to choose products based on their specic needs like price, manufacturer, and other attributes. They also allow customers to add reviews of products they have bought. Customer reviews of a product are generally considered more honest, unbiased and comprehensive than descriptions provided by the seller. Furthermore, reviews written by other customers describe the usage experience and perspective of (non-expert) customers with similar needs. A study by comScore and Kelsey group[2] showed that online customer reviews have signicant impact on prospective buyers. However, as the number of customer reviews available increases, it is almost impossible for a single user to read all reviews and comprehend them to make informed decisions. Therefore, mining these reviews to extract useful information efciently is an important and challenging problem. The abundance of customer reviews available has led to a number of scholars doing valuable and interesting research related to mining and summarizing customer reviews[3, 4, 5, 6, 7, 8]. There has also been considerable work on sentiment analysis of sentences in reviews, as well as sentiment orientation of a review as a whole[9]. There has been relatively little work on ranking products 1

1 http://www.amazon.com

negative sentiments about the product being reviewed. Comparative sentences contain information comparing competing products in terms of features, price, reliability etc. Fig. 1 depicts a typical customer review highlighting the different kinds of sentences. After developing techniques to identify such sentences, we build a weighted and directed product graph(feature-based) that captures the sentiments expressed by customers in reviews. We perform a ranking of products based on the graph. Particularly, the main contributions in this work are: NLP and dynamic programming techniques to identify subjective/comparative sentences(product feature-based) in reviews and determine their sentiment orientations. Sentence classication techniques to build a weighted and directed graph which reects the inherent quality of products in terms of their features. A ranking algorithm that uses this massive graph to produce a ranking list of products based on each considered product feature. That is, the end result of the algorithm is a ranking list that a potential customer can use to determine the best products based on customers importance of one or more product features. The reminder of this paper is organized as follows. Section 2 contains a summary of related work. We present our techniques in Section 3. Section 4 contains the details of datasets and experimental results followed by the conclusions in Section 5.

Figure 1: A Customer Review from Amazon.com. Product Features and Subjective/Comparative Sentences are Highlighted.
automatically based on customer reviews. Our work is based on the our proposed ranking scheme, where products are ranked based on their overall quality. A product feature is dened as an attribute of a product that is of interest to customers. Even though an overall ranking is an important measure, different product features are important to different customers based on their usage patterns and requirements. And different products may be designed and rank differently based on the feature of interest. For instance, a digital camera that is ranked highly overall may have less-than-stellar battery life. Thus, both overall ranking and more detailed product feature based ranking are important. In this paper, we propose an algorithm that uses customer review text, mines tens of thousands of reviews, and provides ranking of products based on product features. For each product category, product features are dened and extracted in the data preprocessing step. Note that for most products, there is a standard set of features which are considered important and normally are provided with product descriptions. We then label each sentence with the product features described in it. We then identify four different types of sentences in customer reviews that are useful in determining a products rank: positive subjective, negative subjective, positive comparative, negative comparative. Subjective sentences are those sentences in which the reviewer expresses positive or 2

2 Related Work

Our work is partly based on and closely related to opinion mining and sentence sentiment classication. Extensive research has been done on sentiment analysis of review text[9, 10] and subjectivity analysis (determining whether a sentences is subjective or objective[22]). Another related area is feature/topic-based sentiment analysis[23], in which opinions on particular attributes of a product are determined. Most of this work concentrate on nding the sentiment associated with a sentence (and in some cases, the entire review). There has also been some research on automatically extracting product features from review text[5]. Though there has been some work in review summarization[3], and assigning summary scores to products based on customer reviews[14], there has been relatively little work on ranking products using customer reviews. To the best of our knowledge, there has been no focused study on product feature based ranking using customer reviews. The most relevant work is [5], where
the authors introduce a unsupervised information extraction system which mines reviews to build a model of important product features, and incorporate reviewer sentiments to measure the relative quality of products. Our work differs from theirs in the following aspects: (1) We use a keyword matching strategy to identify and tag product features in sentences, (2) We use different strategies to assign sentiment orientation to sentences, and incorporate comparisons between different products in our model and (3) Based on sentence classication, we build a product graph for each product feature. By mining this graph, we are able to comprehensively rank products based on reviews they have received.

Product Ranking Methodology Identifying Product Features
Product features are attributes that provide added functionality to products. In the product description, manufacturers tend to highlight their product features from different (often contradicting) perspectives. The combination of features available in a product inuences the purchasing decision of a customer. Some consumers typically seek an optimized combination of features to satisfy their needs, while others may focus on a single feature. There has been some research on automatically identifying the different features of a product that customers are concerned about[5]. There has been earlier research on identifying product features and feature sentences, which is not the focus of this paper. We assume that product features associated with a product domain are given. We manually gathered product feature sets for two product cateogories: Digital Cameras and Television based on the ofcial consumer reports2. This is a one-time pre-processing overhead that can be done relatively easily by someone being familiar with a product domain. Table 1 shows the different product features and their major associated synonym sets. The use of synonyms is motivated by the fact that customers use different words/spelling variants to describe product features. To test the effectiveness of our feature-nding mechanism, we randomly picked 1000 review sentences from our review pool and manually labeled each sentence with the product features described. This small dataset was used to evaluate the performance of our feature-nding strategy. The precision and the recall of the keyword strategy for digital camera data were 0.853 and 0.807 respectively. Therefore, we are able to nd a major portion of feature sentences using this simple yet effective strategy.
2 http://www.consumerreports.org/
In our review dataset containing 1516001 sentences, we observed that around 16% of sentences describe one or more of these features. To tag each sentence with features, we use a simple strategy: if the sentence contains one of the words/phrases in the synonym set for a product feature, we mark it as describing the feature. Since we have dened these features manually, we describe them in greater detail. For Digital Cameras, ash is an important feature for indoor and low-light photography. Battery life is a sought-after feature that details the kind of batteries used. Focus talks about auto-focus or manual focus capabilities. Lens is a critical factor for professional photographers purchasing high-end cameras. The Optical feature encompasses digital zoom and optical zoom. LCD represents the digital display/screen that lets a user see how a photo will look like. Resolution refers to the sharpness, or detail, of a picture. Burst is used to describe the rapid re and continuous shooting capabilities. Memory determines the number of pictures that can be taken and Compression determines how le size of a photo is shrunk. For the Television segment, the Sound feature is useful for users interested in audio quality (some TVs come with an extra set of speakers to create surround sound). Reectivity/Anti-glare is important for the viewing experience. Size represents the size, height, weight of a television screen. Connections means the number and type of input ports available for hooking up devices to the television. The richness/quality of the images displayed are described in Picture quality. Users are also interested in the remote control device available with the television. Resolution refers to the number of pixels or lines displayed on the screen. Adjustment is the ability/mode that expands or compresses an image to ll the screen better. Picture-in-picture(PIP) feature allows a user to watch two channels at once. FilmMode/CineMotion improves the movie-watching experience, which may be important to some users.

3.2 Sentence Labeling

Customers express their opinions about products in multiple ways. We identify two kinds of sentences that are useful while ranking products: Subjective Sentences and Comparative Sentences. Before doing this task, we use MxTerminator[11] to split reviews into sentences because a typical customer review comprises of several sentences. We formally dene the different types of sentences below: Denition 3.1. Subjective Sentence(SS) A sentence expressing direct praise or deprecation about a product. Ex. This camera has excellent shutter speed. Denition 3.2. Comparative Sentence(CS) A sentence which indirectly express an opinion by performing a 3
Table 1: Keywords Representing 10 Most Important Product Features for Digital Camera and Television Domains Digital Camera TV resolution|pixel|megapixel connection|input|output| component video|composite video|HDMI lens|wide angle|normal range adjustment|stretch|zoom|expand|compress optical|zoom|optical zoom|digital zoom lm-mode|frame|theatrical|3:2|pull-down| motion compensation|CineMotion memory|megabytes|MB pip|picture-in-picture|dual-tuner|pop| picture-outside-picture|two-tuner burst|continuous|shutter|recovery|motion|sport resolution|1080p|1080i|720p battery|batteries|power screen|anti-glare|reectivity|burn-in|shiny| screensaver|pixel-shift focus|exposure|manual|iso picture|image|picture quality|image quality LCD|screen sound|sound quality|speaker|stereo|audio compression|compress|jpeg size|height|width|depth|weight|inch ash|light remoter|remote|gear|universal comparison between two products. Ex. I think the coolpix is better than the canon sd1200. Denition 3.3. Product Comparative Sentence(P CS) A PCS is a comparative sentence and contains at least one product name. Ex. This TV has much better sound quality when compared to the sony bravia. propose a more rened technique to nd comparative sentences specically related to product comparisons in our previous work[21]. We use a dynamic programming technique (longest common subsequence) to identify product-product comparison pairs in a comparative sentence. We use only comparative sentences which contain at least one product name which is different from the product the sentence is describing while building our ranking model. In [21], we have shown that we get a precision of 82% and a recall of 80% approximately.
Identifying Comparative Sentences
There has been some earlier research[16, 17] regarding identication of comparative sentences in text. These techniques use keyword comparison(KW contains 126 words, some of which are explicit(outperform, exceed, compare, superior, etc.) and others are implicit(prefer, choose, like, etc.)), sentence semantics, and sentence structure to identify comparative sentences. To identify part-of-speech tags, CRFTagger[12], a java-based conditional random eld part-of-speech (POS) tagger for English is employed. We build on these techniques and use the following rules for identifying comparative sentences: Check if the sentence contains any comparative keywords in KW ; Recognize any words with POS tags { JJR(comparative adjective), RBR(comparative adverb), JJS(superlative adjective), RBS(superlative adverb) }; Scan if any predened structural patterns are present in the sentence (as <word> as, the same as, similar to, etc.). Note that not all sentences satisfying these rules are comparative sentences in terms of product comparison. For example, the sentence I bought this camera for my son because he got a higher grade in his second statistical exam. does not show any comparative meanings or implications over other camera products. Therefore, we 4

3.4 Identifying Sentence Sentiment Orientation
In this section, we describe how we assign sentiment orientations for a sentence. We only consider positive and negative sentiments in this work. Unfortunately, dictionaries and other sources like WordNet[13] do not include sentiment orientation information for each word. Some researchers[18] have used supervised learning algorithms to infer the sentiment orientation of adjectives and adverbs from constraints on conjunctions. [9] contains a summary of existing sentiment analysis techniques. In this paper, we use a simple yet powerful method by utilizing a positive word set(POS) and a negative word set(NEG) developed in the MQPA project[19]. We also add some words of our own. At the end of this process, we get a list of 1974 words for the positive set and 4605 words for the negative set. We use a simple technique to identify the orientation of a sentence using these words. If the sentence contains a word that is in the positive word set, we label this sentence with a positive tag. Negative sentiment words are handled similarly. However, many customers do not like to express their opinions by using assertive sentences but using some negations in their reviews. In this case, the orientation should be switched. We constructed a set of 28 negation
words manually. It should be mentioned that we determine the sentence orientation for comparative sentences as well using the same list of sentiment words. A precision of 82% and a recall of 86% approximately can be achieved and shown in [21].
Table 2: Product Ranking Results for Gf in Fig. 2 Rank Vertex ID Score 1 B 0.D 0.C 0.A 0.052781 weight is also crucial to the ranking, in addition to the edge weights. The idea can be formalized using the equation below: pRank(P ) = [(1 d) + d pRank(Pi ) Ce (Pi )] Cv (P ), where
Constructing the Product Graph
We use the subjective and comparative sentences found to construct a directed and weighted graph for each product feature that can be mined to reveal the relative quality of products. The product graph for feature f is dened as follows: Gf = (V, E) where V is a set of nodes, V = {pi | each node represents a product, 0 < i < n}, E is a set of node pairs, called arcs or directed edges. An arc e = (pi , pj ) is considered to be directed from pi to pj. E = {ek = (pi , pj ), | Wei is the weight of the edge ei , 0 < i, j < n, 0 < k < m}, where n is the number of products, m is the number of edges. Consider a comparative sentence in the reviews for product Pi describing feature f. If this sentence compares product Pi with product Pj , we add a directed edge from Pj to Pi. The second step is to assign a weight to this edge. A comparative sentence occurring in the reviews for product Pi and comparing it with product Pj is considered a positive comparative(P C(Pi , Pj )) if it implies that Pi is better than Pj. If it implies that Pi is worse that Pj , it is considered a negative comparative(N C(Pi , Pj )). For each edge(Pj , Pi ), we count the number of positive (P C) and negative (N C) comparative sentences associated with the pair (Pi , Pj ) respectively. We assign the ratio P C/N C as the weight of the edge linking Pj to Pi. The weight of a node is used to represent the inherent quality of a product. For a node Pi , we use the ratio of the number of positive(P S) and negative(N S) subjective sentences (P S/N S) as its weight.

1{Pi ,P }

pRank(P ) is the product ranking of product P ; pRank(Pi ) is the product ranking of product Pi and n is the number of incoming links on product P ; 1{Pi ,P } is an indicator function, s.t. 1{Pi ,P } = if there is a link from Pi to P otherwise
W (P ,P Ce (Pi ) = Pm eWei(Pi),Pj ) , where m is the number j=1 of outbound links on product Pi , Pj are the nodes pointed to from Pi and We (Pi , Pj ) is the weight of the edge (Pi , Pj ). It is the edge weight contributor to the ranking of product P ; ) Cv (P ) = Pn Wv (P,P t ,Pt ). It is the node weight t=1 Wv (P contributor to the ranking of product P.
Let us illustrate the ranking process using a simple example. We have four products(A, B, C, D) which we wish to rank according to product feature f. The numbers of positive/negative, subjective/comparative sentences labeled with feature f are listed below. P Sf (A) = 1, P Sf (B) = 2, P Sf (C) = 3, P Sf (D) = 4 N Sf (A) = 3, P Cf (B, A) = 3, P Cf (B, C) = 7 P Cf (B, D) = 3, P Cf (A, C) = 2, N Cf (B, C) = 2 Based on these sentence statistics, we build a product graph Gf (see Fig. 2). Edge weights are determined by comparative sentences, and node weights are determined by subjective sentences. Since the reviews of product C have 7 positive comparative sentences mentioning product B (and feature F ), and 2 negative comparative sentences mentioning B (and feature f ), there is an edge from C to B with weight 3.5. It must be mentioned that to prevent edges with innite length (when the number of negative comparative sentences is 0), we set the minimum value of the denominator to 1 while 5

Ranking Products

We evaluate the relative importance of each product according to feature f by using the pRank algorithm described below. The classic PageRank algorithm[20] treats all edges equally and does not take node weights into account. A node has a higher importance if it is pointed to from relatively important nodes. In pRank, we not only consider the relative importance among products, but also take the importance/quality of the product itself into account. This means that the node
Algorithm 1 pRank Rank Products for Feature f Require: Product Feature(f ), Product Category(Cat). Ensure: The ranking list of products belonging to category Cat for the feature f.
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:
Figure 2: A Simple Ranking Example With a Product Graph Gf Having 4 Products Regarding a Specic Feature f (+ means positive and - means negative) computing edge weights. By using our algorithm, we get the ranking score for each node shown in the Table 2. The ranking order(the smaller, the product better) for this graph is B D C A. From the graph, we clearly see that A, C, D are worse than B because all of them have edges pointing to B. D has more positive subjective sentences than A, C and their comparative weights with B are approximately equal. C has a better ranking than A because (i) two sentences say A is better than C and (ii) reviews for A contain 1 positive/3 negative subjective sentences while reviews for C contain 3 positive subjective sentences. Algorithm 1 below summarizes our ranking methodology.

XML Data = Download(Cat); SENT = Get Sentence(XML Data); LSENT = Label(SENT, F ); {P S, N S, P C, N C} LSEN T ; for each sentence s {P C, N C} do Find all product comparison pairs {pi , pj } using dynamic programming; Pair {pi , pj } + P os or N eg ; end for Count P Spi , N Spi , P Cpi ,pj , N Cpi ,pj related to all products; for each product pi do for each product pj do if i == j then M atrix[i, i] = P Spi /N Spi ; else M atrix[i, j] = P Cpj ,pi /N Cpj ,pi ; end if end for end for Ranking List = Rank(M atrix[]); return Ranking List;

Experiment Results

In this section we evaluate the performance of our ranking algorithm. We conduct our experiments on customer reviews from two different product categories (Digital Camera and Televisions) from Amazon.com. Further details about the datasets and the APIs used to generate this data can be found at Amazon.com3 and BrowseNodes.com4. The Digital Camera dataset contains 83005 reviews (for 1350 products) and Television dataset contains 24495 reviews (for 760 products) collected by August, 2009. Table 3 and Table 4 show the relevant statistics for these two datasets: total number of sentences, frequency of occurrence of different product features, number of subjective and comparative sentences and their sentiment orientations. To evaluate our ranking algorithm, we rst perform product ranking based on the overall quality. To determine the overall rank of a product, we include all comparative and subjective sentences in our database while constructing
3 http://www.amazon.com/ 4 http://www.browsenodes.com/
the product graph. There is no ltering done for product features. We then mine this overall graph Goverall using the ranking algorithm described in Section 3.6. To evaluate the effectiveness of this ranking strategy, we compare our results with a ranking performed by domain experts. The results indicate that our product ranking strategy achieves signicant agreement with evaluations done by subject experts with several years of experience and insight in their respective elds. Approximately, an average overlapping probability of 62% could be achieved for different price bins for cameras and televisions. More details about this evaluation can be found in[21]. In this paper, we focus on the feature-specic ranking obtained by mining the individual product graphs generated for each product feature. Intuitively, the feature-specic ranking should not be dramatically different from the overall ranking. If we have chosen a relevant set of product features that customers are interested in, then the top-ranked products in these lists should not rank badly in the overall list. However, it is quite likely that there are signicant differences in the ranking order of these products, especially at the top. To clarify this intuition, we give the following example: If a product ranks in the top 5 products according to feature f (Ex. lens), then the probability that it ranks in the 6

Table 3: Breakdown of Subjective/Comparative Sentences(Digital Camera) Feature/Overall No. of Sentences No. of Subjective Sentences No. of Comparative Sentences Positive Negative Positive Negative Flash 514 Battery 533 Focus 720 Lens 437 Optical 338 Lcd 216 Resolution 227 Burst 189 Memory 143 Compression 29 Digital Camera 10890 Feature/Overall Sound Screen Size Connection Resolution Picture Quality Remoter Adjustment PIP Film-Mode TV Table 4: Breakdown of Subjective/Comparative Sentences(TV) No. of Sentences No. of Subjective Sentences No. of Comparative Sentences Positive Negative Positive Negative 9162
bottom 5 products overall should be very low. Similarly, if a product has high overall rank, then it should rank Table 5: The Relative Feature Fraction(RF Ff ) for Digital Camera and TV highly according to some features. Another aspect of our feature ranking methodology is the relative importance of different product features. A valid question that can be asked is: Which product features are customers looking for when making their choices? To answer this question we dene two metrics, Relative Feature Fraction(RF F ) and Importance of Feature(IF ). Digital Camera Flash Battery Focus Lens Optical LCD Resolution Burst Memory Compression RF Ff 18.18% 15.96% 15.93% 13.67% 10.77% 9.72% 5.63% 5.40% 4.06% 0.67% TV Sound Screen Size Connection Resolution Picture Quality Remoter Adjustment PIP Film-Mode RF Ff 24.76% 16.10% 12.87% 11.24% 10.98% 8.90% 8.13% 3.04% 2.15% 1.82%
Denition 4.1. Relative Feature Fraction: RF Ff = PNf *100%, where Nf is the number of sentences laf Nf Table 6: Importance of Feature(IFf ) for Digital Camera beled with feature f. and TV Denition 4.2. Importance of Feature: IFf = |XYf | IFf 100, where X = {top 10% of overall ranked Digital Camera Features IFf TV Features |X| Lens 79.9 Size 78.7 products}, and Yf = {top 10% of products according to Resolution 79.8 Film-Mode 72.3 feature f }. Optical 77.5 Picture Quality 70.7 Focus 76.3 Connection 69.1 76.3 PIP 69.1 Table 5 shows the RF F for different features in Dig- Memory Burst 75.2 Sound 67.5 ital Camera and Television categories respectively. We Lcd 74.1 Remoter 67.5 observe that customers are mentioning ash and sound Flash 72.9 Adjustment 64.3 (for Camera and TV) product features in their reviews Battery 71.6 Screen 61.2 most often. However, a higher RF F does not always in- Compression 68.4 Resolution 61.2 dicate greater importance of a feature. The importance of 7

Table 7: Top 10 Rated Digital Cameras for Each Feature and Overall Ranking Lens Resolution Optical Focus
PENTAX K100D SUPER KODAK EASYSHARE ONE CANON POWERSHOT SD500 CANON POWERSHOT SD990IS NIKON COOLPIX L3 NIKON COOLPIX 8400 NIKON COOLPIX L1 KODAK EASYSHARE C653 NIKON COOLPIX P6000 NIKON COOLPIX L4 CANON POWERSHOT SD200 NIKON COOLPIX S4 NIKON COOLPIX L3 NIKON COOLPIX S600 KODAK EASYSHARE C653 NIKON COOLPIX P90 NIKON COOLPIX L4 SONY ALPHA A700K NIKON COOLPIX P50 PENTAX OPTIO 60 CASIO EXILIM PRO KODAK EASYSHARE ONE CANON POWERSHOT SD550 CANON POWERSHOT SD990IS NIKON COOLPIX 3200 NIKON COOLPIX L3 NIKON COOLPIX S500 HP PHOTOSMART M537 NIKON COOLPIX P50 NIKON COOLPIX 7600 CASIO EXILIM PRO KODAK EASYSHARE ONE PANASONIC DMC-FX37A 10.1MP CANON POWERSHOT S60 CANON POWERSHOT SD990IS NIKON COOLPIX 3200 PENTAX OPTIO P70 NIKON COOLPIX L3 NIKON COOLPIX 8400 NIKON COOLPIX L1

Memory

KODAK EASYSHARE ONE CANON POWERSHOT S80 PANASONIC DMC-FX37A 10.1MP PENTAX OPTIO A10 CANON POWERSHOT SD990IS NIKON COOLPIX 3200 OLYMPUS SP-550UZ 7.1MP NIKON COOLPIX 4300 CANON POWERSHOT S410 NIKON COOLPIX 8400
CASIO EXILIM PRO CANON POWERSHOT SD500 PANASONIC DMC-FX37A 10.1MP CANON POWERSHOT S80 CANON POWERSHOT S60 CANON POWERSHOT SD990IS NIKON COOLPIX 3200 NIKON COOLPIX 995 CANON POWERSHOT S100 NIKON COOLPIX L3
KODAK EASYSHARE ONE PANASONIC DMC-FX37A 10.1MP CANON POWERSHOT S80 PENTAX OPTIO A10 CANON POWERSHOT SD990IS NIKON COOLPIX 3200 CANON POWERSHOT S400 CANON POWERSHOT G3 SONY DSCP150 7MP NIKON COOLPIX S500
KODAK EASYSHARE ONE CASIO EXILIM PRO PANASONIC DMC-FX37A 10.1MP NIKON COOLPIX S200 SONY DSCP150 7MP CANON EOS 1D NIKON COOLPIX 8400 NIKON COOLPIX L1 NIKON COOLPIX P90 SONY ALPHA A700K

Battery

CANON POWERSHOT SD990IS NIKON COOLPIX S4 NIKON COOLPIX S500 NIKON COOLPIX L3 KODAK EASYSHARE C653 HP PHOTOSMART M537 NIKON COOLPIX P90 NIKON COOLPIX P6000 NIKON COOLPIX L4 SONY ALPHA A700K

Compression

CANON POWERSHOT A620 CANON POWERSHOT SD300 KODAK EASYSHARE ZD710 CANON POWERSHOT S100 NIKON COOLPIX 8700 NIKON COOLPIX 4300 NIKON COOLPIX L3 NIKON COOLPIX P50 CANON POWERSHOT S230 CANON POWERSHOT A95

Overall Quality

NIKON COOLPIX L1 PANASONIC DMC-FX37A 10.1MP CANON POWERSHOT A990IS NIKON COOLPIX P90 HP PHOTOSMART M537 CANON POWERSHOT A70 NIKON COOLPIX 8800 FUJIFILM FINEPIX A330 KODAK EASYSHARE C653 OLYMPIS STYLUS 550
a feature should reect the role a feature plays in inuencing a customer. The IF metric measures the agreement between the overall ranking and feature-specic ranking. The values for IF for different product features are shown in Table 6. They indicate that our product feature-based ranking is consistent with overall ranking (which is veried previously by comparing with expert ranking [21]). In addition, this ratio should not be close to 1 because overall quality is different from feature quality. Table 6 also tells us that lens (Digital Camera) and size (Television) are the leading factors inuencing the overall quality of a product. Table 7 shows the top 10 cameras according to the overall ranking and featurespecic ranking. These results are consistent with the arguments made earlier.

a methodology and algorithm to rank products based on their features using customer reviews. First, we manually dene a set of product features that are of interest to the customers. We then identify subjective and comparative sentences in reviews using text mining techniques. Using these, we construct a feature-specic product graph that reects the relative quality of products. By mining this graph using a page-rank like algorithm(pRank), we are able to rank products. We implement our ranking methodology on two popular product categories (Digital Camera and Television) using customer reviews from Amazon.com. We believe our ranking methodology is useful for customers who are interested in specic product features, since it summarizes the opinions and experiences of thousands of customers.

Conclusion

6 Acknowledgement
This work was supported in part by NSF grants CNS0551639, HECURA CCF-0621443, OCI 0956311, SDCI OCI-0724599, OCI-0956311, IIS-0905205, and DOE grant DE-FG02-08ER25848. 8
Recent trends have indicated that large numbers of customers are switching to online shopping. Online customer reviews are an unbiased indicator of the quality of a product. However, it is difcult for users to read all reviews and perform a fair comparison. We describe

References

[1] Forrester research, http://www.comscore. com/Press_Events/Press_Releases/2007/ 11/Online_Consumer_Reviews_Impact_ Offline_Purchasing_Behavior. [2] comscore and Kelsey, http://www. shop.org/c/journal_articles/ view_article_content?groupId= 1&articleId702&version=1.0. [3] M. Hu and B. Liu, Mining and Summarizing Customer Reviews, Proceedings of the 10th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD-2004), 8 (2004), pp. 168174. [4] M. Hu and B. Liu, Mining Opinion Features in Customer Reviews, Proceedings of the 19th National Conference on Articial Intelligence., 7 (2004), pp. 755-760. [5] A. Popescu and O. Etzioni, Extracting product features and opinions from reviews, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing., (2005), pp. 339-346. [6] B. Liu, M. Hu, and J. Cheng, Opinion Observer: Analyzing and Comparing Opinions, WWW., 5 (2005), pp. 342351. [7] S. Kim, P. Pantel, T. Chklovski, and M. Pennacchiotti, Automatically Assessing Review Helpfulness, EMNLP., 7 (2006), pp. 423-430. [8] B. He, C. Macdonald, J. He, and I. Ounis, An Effective Statistical Approach to Blog Post Opinion Retrieval, CIKM., 10 (2008), pp. 1063-1069. [9] B. Liu, Sentiment Analysis and Subjectivity, to appear in Handbook of Natural Language Processing, Second Edition, 2010. [10] B. Pang, L. Lee, and S. Vaithyanathan, Thumbs up? Sentiment Classication using Machine Learning Techniques, EMNLP., (2002), pp. 79-86. [11] J. Reynar and A. Ratnaparkhi, A Maximum Entropy Approach to Identifying Sentence Boundaries., In Proceedings of the Fifth Conference on Applied Natural Language Processing , March 31-April 3, 1997. Washington, D.C. [12] X. Phan, CRFTagger: CRF English POS Tagger, http: //crftagger.sourceforge.net/, 2006. [13] G. A. Mille, C. Fellbaum, etc., WordNet: An Electronic Lexical Database, 5 (1998),MIT Press. [14] M. Hu, and B. Liu, Opinion extraction and summarization on the web, AAAI., 7 (2006), pp. 1621-1624. [15] B. Liu, X. Li, W. S. Lee, and P. S.Yu, Text Classication by Labeling Words, AAAI., 2004, pp. 425-430.

[16] N. Jindal, and B. Liu, Identifying Comparative Sentences in Text Documents, SIGIR., 8 (2006), pp. 244-251. [17] N. Jindal, and B. Liu, Mining Comparative Sentences and Relations, AAAI., (2006), pp. 1331-1336. [18] V. Hatzivassiloglou, and K. R. McKeown, Predicting the semantic orientation of adjectives, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, 1997, pp. 174-181. [19] MPQA corpus http://www.cs.pitt.edu/mpqa, 2002. [20] L. Page, S. Brin, R. Motwani, and T. Winograd, The PageRank citation ranking: Bringing order to the Web, Stanford University, 1999. [21] K. Zhang, R. Narayanan, and A. Choudhary, Mining Online Customer Reviews for Ranking Products, Technical Report, EECS department, Northwestern University, 2009. [22] S. Arora, M. Joshi, and C. P. Rose, Identifying Types of Claims in Online Customer Reviews, NAACL HLT., 6 (2009), pp. 37-40. [23] Y. Lu, C. Zhai, and N. Sundaresan, Rated Aspect Summarization of Short Comments, WWW., 4 (2009), pp. 131140.