CWS: A Comparative Web Search System
Jian-Tao Sun , Xuanhui Wang , Dou Shen , Hua-Jun Zeng , Zheng Chen
Microsoft Research Asia, Beijing, P.R.China {jtsun, hjzeng, zhengc}@microsoft.com Department of Computer Science, University of Illinois at Urbana-Champaign xwang20@cs.uiuc.edu
Department of Computer Science, Hong Kong University of Science and Technology dshen@cs.ust.hk

ABSTRACT

In this paper, we dene and study a novel search problem: Comparative Web Search (CWS). The task of CWS is to seek relevant and comparative information from the Web to help users conduct comparisons among a set of topics. A system called CWS is developed to eectively facilitate Web users comparison needs. Given a set of queries, which represent the topics that a user wants to compare, the system is characterized by: (1) automatic retrieval and ranking of Web pages by incorporating both their relevance to the queries and the comparative contents they contain; (2) automatic clustering of the comparative contents into semantically meaningful themes; (3) extraction of representative keyphrases to summarize the commonness and dierences of the comparative contents in each theme. We developed a novel interface which supports two types of view modes: a pair-view which displays the result in the page level, and a cluster-view which organizes the comparative pages into the themes and displays the extracted phrases to facilitate users comparison. Experiment results show the CWS system is eective and ecient.
Categories and Subject Descriptors
H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval-Search Process; H.3.5 [Information Storage and Retrieval]: Online Information Services-Web based services

General Terms

Algorithms, Experimentation, Performance

Keywords

Clustering, Comparative Web Search, Keyphrase Extraction, Search Engine

INTRODUCTION

Nowadays, search engines have become popular tools for users to seek information from the Web. In general, Web
Copyright is held by the International World Wide Web Conference Committee (IW3C2). Distribution of these papers is limited to classroom use, and personal use by others. WWW 2006, May 2326, 2006, Edinburgh, Scotland. ACM 1-59593-323-9/06/0005.
users may have various goals when conducting search. For example, one user may want to nd a picture of British Museum, another user may hope to nd favorite blogs, and some other users may have the need of comparing two products to guide their purchases. In this paper, we dene and study a novel search problem, which we refer to as Comparative Web Search (CWS). CWS is targeted to help users when they wish to make comparisons among a set of topics, e.g., dierent games, cars, or conferences, etc. Its task is to retrieve relevant and comparative information from the Web so as to facilitate Web users comparison needs. Conducting comparisons on the Web is becoming more and more common recently. For example, the emergence of e-commerce makes online shopping very convenient and it is preferred by Web users. To make good purchases, many shoppers indeed rst leverage the Web to nd relevant information as guidance before their purchases. They may want to compare the features of dierent products, the online customers reviews about the products, the stores selling the products, and so on. Other examples include: comparing two related terminologies to understand their dierences; comparing two anti-terrorism wars about their costs, their consequences, and also the opinions of the critics. Apparently, CWS can benet all the above needs. There are several approaches available which can help people make comparisons on the Web. For example, some newly emerged Web sites began to provide comparison shopping services. Shopping.com and Froogle (http://froogle.google.com) have integrated product comparison services to provide comparative information such as price and customer reviews. However, most of these Web sites are specialized in a certain domain (e.g., products) and can only help fulll limited comparison tasks for a certain group of users. Whats more, their services are based on the structured information provided by the database. Another method is to use traditional search engines for comparative search tasks. Unfortunately, this is not eective since Web users have to manipulate several search windows for a comparative view. To make comparisons with respect to dierent aspects, users have to frequently rene the queries appropriately or navigate through the result pages. This obviously is tedious for the users. Thus it is much desired to maintain a general platform on which users can easily retrieve and compare every kind of information they need.

In this paper, we propose a comparative Web search system, CWS, which can help users to nd comparative information easily. The CWS system is dierent from traditional search engines conceptually. In a traditional search scenario, a Web user submits a query describing his/her information need and a search engine returns a list of presumably relevant pages. In contrast, the objective of our CWS system is to facilitate Web users comparison needs. It allows a user to submit a group of comparative queries with each of them describing a concept the user wants to compare. Our system retrieves the relevant information from the Web, aligns the comparative contents, and ranks them by combining both their relevance to the issued queries and the amount of comparative information they share. Moreover, to help the users digest the comparative contents, we cluster them into dierent themes and extract representative keyphrases to summarize each theme. At the user end, we implement a novel interface which supports two types of view modes: a pair-view which displays the result in the page level, and a cluster-view which organizes the comparative pages into the themes and displays the extracted phrases to facilitate users comparison. In summary, the CWS system is characterized by: (1) automatic retrieval and ranking of Web pages based on both their relevance to queries and the comparative contents they contain; (2) automatic clustering the comparative contents into semantically meaningful themes; (3) extraction of representative keyphrases to summarize the commonness and dierences of the comparative contents in each theme. The remainder of this paper is organized as follows. Section 2 provides related works. Section 3 gives a brief introduction to our CWS system and we describe our algorithms in Section 4. Section 5 presents the experimental results and Section 6 oers some concluding remarks and directions for future research.

RELATED WORK

There were few works on comparative Web search. The most related ones are those focusing on comparing specic Web sites or data collections. Liu et al. [10, 11] compares two Web sites, e.g., the sites of two competitive companies. Given two Web sites, all their pages are merged and partitioned into hierarchical clusters. The pages are then displayed in a tree form and visualization techniques are adopted to emphasize the dierences between the two sites. In [12], the authors developed a comparative browser for comparing pages of two Web sites. Their system concurrently presents multiple Web pages thus enabling users to view them at the same time. After a user selects a page from one site, the system retrieves similar contents from the other site. Our system is dierent from the above works since our purpose is to conduct Web search given a set of comparative queries, instead of making comparisons between two Web sites. Recently, Zang and Zhai et al. dene a novel comparative text mining (CTM) problem [21, 18]. Though related, CTM is dierent from comparative Web search: comparative text mining is conducted on a set of comparative text collections to discover latent common themes across all collections as well as the themes specic to each collection. Tao and Zhai [16] conducted mining on the comparable bilingual text corpus to align a word from one language to a word in another language based on their statistical informtion. In contrast, the task of CWS is query-dependent and the ob-

jective is to retrieve comparative information from the Web. Another related work is opinion mining [7, 9]. It is to extract customers opinions on product features based on a collection of customer reviews. Then both customers and manufactures can make comparisons between products. The authors use several natural language processing techniques and data mining approaches to help identify product features and sentiments of customer opinions. Their methods can not be easily used in CWS because they are domaindependent. Moreover, the data used in opinion mining is usually well organized and less noisy. All the above works are based on oine mining while CWS focuses on online search. In this paper, we developed a comparative search system named CWS. Our system can automatically retrieve Web pages containing comparative information and align comparative page pairs. As far as we know, the available search systems have no such kind of functionalities. Another advantage of our system lies in that it is able to organize the comparative Web pages into clusters and extract keyphrases from them to summarize the common contents of a cluster, as well as the dierences between the concepts compared. There are some recent researches on search result clustering [19, 1, 8]. Dierent from them, our objective is to cluster comparative page pairs in order to facilitate Web users comparison purpose. In this paper, we adopted a probabilistic clustering algorithm proposed in [21]. The advantage of this approach is that it provides a method to rank the topic themes of all clusters and can produce representative terms for each cluster. There are also some works on automatic keyphrase extraction from documents [17, 20]. In [17], the authors developed a system named KEA, which uses Naive Bayes algorithm to extract keyphrases. In [20], the authors proposed a simultaneous method for keyphrase extraction and text summarization by modeling text documents as bipartite graphs. In [6], the authors discussed the extraction of important phrases from a text stream (e.g., news) and use it as a query to search relevant pages from the Web. In this paper, we use a keyphrase extraction system, called KEX, developed in our group to extract keyphrases [3]. Furthermore, we also propose an entropy based method to select keyphrases which are unique to the concepts compared by a Web user.

3. SYSTEM OVERVIEW

In this section, we give an overview of our CWS system. Figure 1 illustrates the owchart of our system. For simplicity, our system allows users to give two comparative queries q1 and q2 as input. Both queries are submitted to a search engine to get the ranked list of pages from the Web. Then, we re-organize these two lists to get the comparative page pairs and rank them. This is the pair-view output. To help the users to digest the information, we also adopted one clustering algorithm to group the similar pairs together. The keyphrases are extracted from the clusters to highlight the contents of the clusters. This gives the cluster-view output. Figure 2 gives an example of the CWS system interface. The pair-veiw is illustrated in Figure 2(a) and the clusterview is given in Figure 2(b). In both modes, two text boxes are provided to input the comparative queries. In the pairview mode, after queries are submitted, two lists of Web pages are generated by the system and are displayed in two columns. The left list of pages correspond to the query con-

(a) Pair-view Interface

(b) Cluster-view Interface Figure 2: CWS System Interface

Search Engine

q2 Web

Page List 1

Comparative Page Pair Matching and Ranking

Page List 2

(1) p1 is relevant to q1 ; (2) p2 is relevant to q2 ; (3) If q1 and q2 are removed from p1 and p2 respectively, the remaining contents of p1 and p2 are similar. We use R to denote the relevance of a query to a page, and S to denote the similarity between two text segments. The function below is used to estimate the likeliness that two pages form a comparative pair with regard to the input queries: fq1 ,q2 (p1 , p2 ) = R(p1 , q1 ) + R(p2 , q2 ) + Tq1 ,q2 (p1 , p2 )

Comparative Page List

Clustering and Key Phrase Extraction

Comparative Cluster List

Tq1 ,q2 (p1 , p2 )
= S(url1 , url2 ) + (1 ) S(p1 \q1 , p2 \q2 ) p1 SR1 , p2 SR2

Figure 1: The Flowchart of CWS System.
tained in the left textbox, while the right list corresponds to the right query. For each result page, the information including title, URL, and snippet is displayed. There are two dierences between the pair-view result and that of traditional search engines. (1) The left page and its corresponding right one share comparative information and they two form a page pair. That is, both pages discuss common topics related to the two input queries. (2) The page pairs are ranked based on their relevance to the queries and the amount of comparative information they contain. In the cluster-view mode, result pages are organized into at clusters. Each of them contains pages of similar topics. The keyphrases reecting the common contents of each cluster are extracted and displayed on the left. If a user clicks on these phrases, all the pages of the corresponding cluster will be displayed on the right using the format similar to the pair-view mode. For each of the two page lists in one cluster, the keyphrases unique to this list are also extracted and displayed on the top.

ALGORITHMS

Our CWS system is based on an existent search engine, denoted by SE. Given two queries, SE will return two lists of pages ranked by their relevance to the two input queries respectively. We then re-organize the search result pages to facilitate Web users comparison needs.
4.1 Ranking Comparative Page Pairs
In order to return comparative information for the input queries q1 and q2 , our rst approach is to automatically rerank the search results returned by SE. Assume SR1 and SR2 represent the result pages corresponding to queries q1 and q2 respectively. In a traditional search, these result pages are ranked by their relevance to the query. In contrast, our purpose is to re-rank SR1 and SR2 to display the comparative page pairs. Assume p1 and p2 are two pages from SR1 and SR2 respectively. If p1 , p2 is a good comparative pair, p1 and p2 should contain information about q1 and q2 respectively and both pages should discuss some common aspects about both queries. Our assumption is: if p1 , p2 is a comparative page pair, they should satisfy:
In Equation (1), Tq1 ,q2 (p1 , p2 ) measures the amount of comparative information of p1 and p2 associated with q1 and q2. The function f considers the relevance between pages and their corresponding queries, as well as the comparative information contained in the two pages. Parameters and are set to be equal in order to guarantee the relevance measures corresponding with the two queries are treated equally. is a tradeo parameter, balancing the relevance measure and the comparison measure. When is zero, the above equation is only a linear combination of relevance information. In Equation (2), the comparative information of p1 and p2 is computed based on their contents and URLs, with balancing the two kinds of information. p1 \q1 and p2 \q2 denote the remaining text contents of page p1 and p2 after removing terms contained in their snippet texts respectively. S(url1 , url2 ) denotes the similarity between the URL strings of p1 and p2. The computation of f is straightforward. In traditional search, R is used to rank Web pages. Usually two factors are considered: the rst is the importance of a page, which is usually computed based on the links among Web pages (e.g. PageRank [13]); the second is the similarity between a query and a page, which can be computed by traditional information retrieval models, such as probabilistic model, vector space model, etc, [2]. These models can also be used for the computation of S. It is quite common for a page editor to put some comparative contents about q1 and q2 in one single page. Such kinds of pages will be in both SR1 and SR2. In this paper, we regard these kinds of pages themselves as comparative pages. The ranking of these pages can also be handled by our approach. In this case, Tq1 ,q2 (p1 , p2 ) is maximal because the same contenst are left if q1 and q2 are removed from the original pages and both pages share the same URL. Thus only R(p1 , q1 ) + R(p2 , q2 ) is needed for ranking purpose. Our purpose is to identify the comparative page pairs from the pages of SR1 and SR2. Those pages form a bipartite graph, where the edge weight is computed by f. Although traditional maximum matching algorithms can also be used to for pair matching [14], they are not suitable for the comparative search task for two reasons: 1) The maximum matching algorithms are not ecient, while CWS is an online application. 2) When Web users make comparisons in a search scenario, they are usually interested in the top

k X j=1

4.3 Extracting Keyphrases for Comparative Clusters
After the page pairs are clustered, we extract keyphrases from each cluster in order to facilitate users comparisons. As each cluster consists of pages corresponding with two queries, we extract the phrases reecting the common theme of all these pages in one cluster, as well as those specic to each query. As discussed in Section 4.2, the important words estimated by the clustering algorithm will be used as common keyphrases for each cluster. In this section, we rst describe our approach to extracting keyphrases for each page. Then we discuss our entropy based method to select keyphrases specic to each query from the phrases generated in the previous step.
4.3.1 Keyphrase Extraction Algorithm
We use a phrase extraction package, KEX, implemented in our group to extract keyphrases for each result page [3]. KEX is based on a supervised approach. The training examples of our package are created by three human annotators who manually extract keyphrases from a collection of Web pages. For each candidate phrase in a Web page, a 4-dimensional feature vector x1 , x2 , x3 , x4 is constructed. These phrases are used to train a linear regression model: y = b0 +

4 X i=1

bi x i

P (w|d ) = (1 B )

[d,j P (w|j )] + B P (w|B )
where w is a word, d,j is the document mixing weight asPk sociated with the j-th theme and j=1 d,j = 1, and B is the mixing weight for the background model. To estimate the parameters = {d,j , j |d C, j = 1, , k}, the
If a phrase is keyphrase, y = 1; otherwise, y = 0. The phrase features include: (1) PF : phrase frequency. This feature is calculated in the traditional meaning of term frequency (TF ). Intuitively, frequent phrases are more likely to be better candidates of keyphrases. (2) ATF, average frequency of all terms in the phrase. Sometimes, a keyphrase may have low PF but contain keyterms with high TF. The ATF feature can be used to discover this kind of keyphrases. (3) AIDF, average inverse document frequency (IDF ) of all terms contained in the phrase. Intuitively, if a phrase contains many terms with low IDF, it is less informative. (4) OKA, modied Okapi weighting score. Okapi is a

highly eective document weighting model in information retrieval [5]. The formula is: X

0.9 P@1 0.8 0.7 P@5 P@10

N df (w) + 0.5 ln df (w) + 0.5

(k1 + 1) c(w, d)

|d| k1 ((1 b) + b avdl ) + c(w, d)
0.6 0.5 0.4 0.3 0.2 0.URL Snippet URL&Snippet
(k3 + 1) c(w, d) k3 + c(w, d)
In our system, we adopt this parameter setting: k1 = 1.2, b = 0.25, k3 = 1000 and avdl = 100. We use log(OKA) score as a feature. After the feature vectors are constructed for all the candidate phrases, we train a linear regression model as described in Equation (3), where x1 =PF, x2 =ATF, x3 =AIDF and x4 =log(OKA). Then we apply this model on every page in each cluster c to rank all candidate phrases. Those ranked at top are selected as keyphrases. In our system, all the candidate phrases are extracted from the title and snippet text returned by SE. We do not use the HTML contents to guarantee the eciency of our CWS system, as downloading these pages and parsing them are quite time-consuming.
Figure 3: Precision measures of comparative page pair results
5.1 Results of the Comparative Page Pair Ranking Approach
In this experiment, we evaluate the eectiveness of the comparative page pairs returned by CWS in the pair-view mode. As discussed in Section 4.1, we need to compute R and T to rank the page pairs by f. In this experiment, as we use a search engine to retrieve Web pages, the search engine does not return the relevance score between a query and a page. We have only the rank order of the result pages. A straightforward approach to estimate the relevance between a query q and a page p is: R(q, p) = 1 , r is the rank of the r page in the corresponding search results returned by SE. The cosine similarity is used to compute the function T in Equation (2) [2].
4.3.2 Keyphrase Selection for Clusters
As the query specic keyphrases summarize the contents contained in sub-clusters c1 and c2 respectively. We propose to use the entropy measure to help select them. X Ent(w) = pi log pi
where pi (i=1,2) measures the probability that phrase w occurs in sub-cluster ci (i = 1, 2). For each sub-cluster, all the keyphrases contained in it are ranked by Ent(w) and those with low entropies are regarded as query specic phrases. Intuitively, if a phrase frequently occurs in one sub-cluster and seldom occurs in the other, it has low entropy value and will be regarded as a keyphrase specic to the current sub-cluster.

q2 bird u KFC Adidas Afghanistan war Google map Jiawei Han Canon camera linux Google talk Clinton
formation is used, URL is better than Snippet. The combination of them leads to better comparative ranking results. The conclusions are consistent when the results are evaluated by P@1, P@5 and P@10 respectively. The best P@10 (in URL&Snippet setting) precision is 0.57, which indicates 57% page pairs in the top 10 results returned by our CWS system are meaningful comparative page pairs.

Recent Additions

Shiite Power Struggle Simmers in Najaf Jill Carroll. Christian Science Monitor, 02 November 2005. The Good News from Iraq is Not Fit to Print

Afghanistan

CIA Holds Terror Suspects in Secret Prisons Dana Priest. Washington Post, 02 November 2005. Posted on the MSNBC website. Detainee Policy Sharply Divides Bush Officials Tim Golden and Eric Schmitt. New York Times, 02 November 2005. Posted on the Fairuse website.

5.1.2 Case Studies

In Section 5.1.1, the eectiveness of comparative page pairs are evaluated using precision measure. Here, we also study two cases in order to give intuitive results of our CWS system. In Table 2, we give the results of two query pairs. The rst pair contains two product queries: Canon Sure Shot 130u and Olympus Stylus Epic. The second consists of query Afghanistan War and Iraq War. The titles and URLs of each page pair are given side by side but the snippets are omitted for the limit of space. The two product queries refer to two types of cameras manufactured by Cannon and Sony, respectively. Web users may submit these two queries in order to make comparisons between the two cameras. From the annotation results, we nd that all the three subjects annotate the 10 results as comparative page pairs. As listed in Table 2, for the rst 9 page pairs, both pages of each pair come from a same website. Take the rst pair as an example: DealTime (http://www.dealtime.com/) is an online shopping Web site and the two pages in this pair come from this website. Both pages contain the price information of several shops selling the corresponding cameras. The two pages are automatically discovered by our system and form a comparative pair. As for the second page pair, PhotographReview (http://www.photographyreview.com/) is a site providing information like digital camera and photo equipment reviews. The pages returned by our system are exactly the two containing the customer reviews about the two cameras queried by the user. The next 7 pages are also comparative page pairs of other Web sites. That is, our CWS system can integrate the comparative pages of various Web sites together and present them to end users, which will greatly facilitate Web users comparison needs. As for the 10th pair returned by our system, the two pages come from Shopping.com and DealTime, respectively, and are put together to form a comparative page pair. This indicates the pages from dierent Web sites can also be identied to form a comparative page pair.

Jeff Jacoby. Boston Globe, 02 November 2005. U.S. to Intensify Its Training in Iraq to Battle Insurgents
Eric Schmitt. New York Times, 02 November As Gitmo Hunger Strike Continues, 2005. Posted on the Fairuse website. 'Failure Is Not an Option' Michael Hirsch. Newsweek, 07 November 2005. Posted on 02 November 2005. Lawyers Step Up Fight for Access Saadia Iqbal. New Standard, 02 November 2005.
Figure 4: A comparative page returned for query pair: Afghanistan war and Iraq war.
Table 2 also gives the results for the query pair: Afghanistan war and Iraq war. Web users may submit the two queries in order to make comparisons between the two recent wars. We can nd that the 5th page pair consists of only one page. This page should contain comparative contents relevant with both wars. This is veried after we check this page. It is a war report page which archives articles about the two wars. All the articles are listed side by side, the left corresponding with the Iraq war and the right corresponding with the Afghanistan war. Partial contents of this page are displayed in Figure 4.
5.2 Results of Comparative Page Clustering and Keyphrase Extraction
Traditional document clustering relies on the category information as ground truth for evaluation [15]. However there is no such information for all the pages we clustered. Instead, we evaluate the clustering results by investigating the accuracy of the extracted keyphrases. The KEX package is used to extract keyphrases for each result page [3]. The linear regression model is trained on a set of 300 Web pages which have been manually annotated by three human subjects. This model can achieve a top 10
Table 2: Results Returned by CWS in Pair-view Mode q1 =Canon Sure Shot 130u, q2 =Olympus Stylus Epic Canon Sure Shot 130U 35mm Film Camera - Find, Compare, and Buy at. Olympus Stylus Epic QD 35mm Film Camera - Find, Compare, and Buy. http://www.dealtime.com/xPC-Canon Sure Shot 130U http://www.dealtime.com/xPC-Olympus Stylus Epic QD Canon Sure Shot 130u Reviews Olympus Stylus Epic Reviews http://www.photographyreview.com/cat/cameras/lm-cameras/point-and-. http://www.photographyreview.com/PRD 84048 3108crx.aspx Olympus Stylus Epic QD - Point & Shoot camera - 35mmprices - CNET. Canon Sure Shot 130u - Point & Shoot / Zoom camera - 35mmprices. http://shopper.cnet.com/4014-6503 9-30231950.html?pbrpt=4583 http://shopper.cnet.com/Canon Sure Shot 130u Point Shoot Zoom Canon Sure Shot 130U - Reviews, Best Prices and Product. Olympus Stylus Epic - Reviews, Best Prices and Product Information. http://www.bizrate.com/marketplace/product info/overview/index. http://www.bizrate.com/marketplace/product info/overview/index cat id. Compare Prices and Read Reviews on Canon Sure Shot 130U 35mm Film. Compare Prices and Read Reviews on Olympus Stylus Epic Zoom 170 QD. http://www.epinions.com/pr-Film Cameras Canon Sure Shot 130u Ca. http://www.epinions.com/pr-Film Cameras Olympus Stylus Epic Zoom 170. Canon Sure Shot 130u II 35mm Camera Kit @ Unverse Olympus Stylus Epic Zoom 170 QD Date 35mm Camera @ Unverse http://www.unverse.com/id-Canon+Sure+Shot+130u+II+35mm+Came. http://www.unverse.com/id-Olympus+Stylus+Epic+Zoom+170+QD+Da. Compare Prices and Read Reviews on Canon Sure Shot 130U 35mm Film. Compare Prices and Read Reviews on Olympus Stylus Epic DLX 35mm. http://www.epinions.com/pr-lm cameras canon sure shot 130u caption 35mm p. http://www.epinions.com/elec Cameras-Point And Shoot OlympusStyluss-. Olympus Stylus Epic QD - Point & Shoot camera - 35mm - SLR. Canon Sure Shot 130u - Point & Shoot / Zoom camera - 35mm - SLR. http://www.mysimon.com/Olympus Stylus Epic QD Point Shoot camera. http://www.mysimon.com/Canon Sure Shot 130u Point Shoot Zoom cam. Canon Sure Shot 130u 35mm Camera w/ Zoom @ Unverse Olympus Stylus Epic QD CG Date 35mm Camera @ Unverse http://www.unverse.com/id-Canon+Sure+Shot+130u+35mm+Camera+w+Zoom- http://www.unverse.com/id-Olympus+Stylus+Epic+QD+CG+Date+35mm. B00006K154 Canon Sure Shot 130U 35mm Film Camera - Find, Compare, and Buy at. Olympus Stylus Epic Zoom 170 QD 35mm Film Camera - Find, Compare. http://www.shopping.com/xPC-Canon Sure Shot 130U http://www.dealtime.com/xPC-Olympus Stylus Epic Zoom 170 QD q1 =Afghanistan War, q2 =Iraq War Afghanistan War. The Columbia Encyclopedia, Sixth Edition. 2001-05 Iran- Iraq War. The Columbia Encyclopedia, Sixth Edition. 2001-05 http://www.bartleby.com/65/af/AfghanWar.html http://www.bartleby.com/65/ir/IranIraq.html The Observer Special reports War in Afghanistan Muslims, Islam, and Iraq http://observer.guardian.co.uk/afghanistan/0,1501,573451,00.html http://www.uga.edu/islam/iraq.html Afghanistan Timeline, 21st Century Iraq War Timeline http://www.mapreport.com/countries/afghanistan.html http://www.infoplease.com/ipa/A0908792.html Articles about September attacks on USA and subsquent. Iraq War http://people.pwf.cam.ac.uk/nwm20/usa afghanistan.htm http://webhost.bridgew.edu/jhayesboh/iraq.html War Report - Iraq War and Afghan Aftermath - compiled by the. http://www.comw.org/warreport/ Informed Comment Independent Online Edition > World Politics: http://www.juancole.com/2004/07/preoccupation-with-iraq-slowed-ushttp://news.independent.co.uk/world/politics/article313450.ece uk.html Government Resources VAIW :: Veterans Against The Iraq War http://library.louisville.edu/government/subjects/war/afgwar/afgwar.html http://www.vaiw.org/vet/index.php Iraq War Cartoons events 19691979 crises recovery eec world renewal tensions cartoon. http://www.ena.lu/europe/crisis-recovery/cartoon-murschetz-afghanistan-war.htm http://www.cartoonistgroup.com/bysubject/theiraqcartoons.php Amazon.com: The Lessons of Afghanistan : War Fighting, Intelligence. Amazon.com: The Iraq War : Books http://www.amazon.com/exec/obidos/tg/detail/-/089206417X?v=glance http://www.amazon.com/exec/obidos/tg/detail/-/1400041996?v=glance Afghanistan : War Without End? UNCOVERED: The War on Iraq http://www.pbs.org/newshour/bb/asia/afghanistan/afghan 12-27-85.html http://www.truthuncovered.com/

1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
precision and recall of 0.303 and 0.297 respectively. This result is not bad, because when we evaluate the annotation result of one subject on those of the other two, the average precision at 10 and the recall at 10 is 0.44 and 0.388 respectively. These values indicate that keyphrase extraction is quite subjective and not an easy task. This conclusion is also drawn in previous research works [17]. In this paper, we do not present the evaluation details of our keyphrase extraction algorithm. Table 3 presents the phrases extracted for query pair: ChengXiang Zhai and Jiawei Han. Table 4 corresponds the result of query pair: Canon Sure Shot 130u & Olympus Stylus Epic. For each cluster, the top 3 common keyphrases as well as the top 3 keyphrases specic to each query are given. As we extract the query specic keyphrases, those which are sub phrases of the query are omitted as they do not provide additional information. The result given in Table 3 is very interesting. As both the professors are from University of Illinois at UrbanaChampaign, from the three common phrases we can nd that the rst cluster corresponds with the pages introducing the two professors. The second cluster corresponds with their research works and the third is about their publications. Most query specic phrases also make sense. For example, in the third and fourth clusters, phrases like information retrieval are extracted for the query ChengXiang Zhai and phrases such as data mining are extracted for the query Jiawei Han. This exactly reects the dierent research interests between Professor ChengXiang Zhai and Professor Jiawei Han. As for the results of the two camera queries, the results are also interesting. For the rst cluster, the words date, compact and kit are extracted as common keyphrases. This is because both the cameras are compact. The two terms date and kit also frequently appear in all the result pages corresponding with the two queries. According to the common phrases, we can also nd that clusters 3, 6 and 7 contain pages on consumer reviews and cluster 4 is about price comparisons.
page pairs which are very relevant with the input queries can be identied, they do not make extra contribution to the precison evaluation. At the beginning of the labeling process, we also asked the subjects to rank the comparative page pairs. However, we found ranking them is much more dicult than just identifying whether two pages form a comparative pair or not. Thus we need other approaches to evaluate the ranking order of the comparative page pairs. In the cluster-view mode, our CWS system can automatically cluster the comparative information into dierent themes. The keyphrases are also extracted to summarize the commonness and dierences of each theme. The examples given in Section 5.2 show the comparative information produced by CWS are helpful for making comparisons. However, it is hard to quantitively evaluate the clustering results as well as the extracted keyphrases.

6. CONCLUSION AND FUTURE WORK
In this paper, we proposed and studied a novel search problem, Comparative Web Search. We developed a CWS system to help users seek comparative information from the Web. Human evaluations and some case studies show that our system is quite eective to facilitate users comparative information needs. In the future, we plan to investigate the following issues: (1) The evaluation of the comparative Web search system is challenging and labor intensive. In this paper, our evaluation result of the CWS system is based on a relatively small query sets. It is interesting to adopt other approaches to evaluate the eectiveness of comparative search system. (2) The queries input to the CWS system represent the topics which the users will compare. How to automatically distinguish comparative query pairs is also an interesting problem. (3) In this paper, we combine the contents and the ranking information of Web pages to construct comparative page pairs. We also plan to incorporate the link structure information to our system. (4) Our approaches to the comparative Web search problem are still preliminary and our CWS system only provides very basic comparison functionalities. More advanced functions can be added by leveraging other relevant techniques. In conclusion, the CWS system is challenging but very helpful to satisfy users comparison needs. We expect to conduct more research work on this direction.

5.3 Discussions

Based on the above experiments and case studies, we nd our CWS system is eective. In the pair-view mode, the percentage of meaningful comparative page pairs in the top 1, 5, 10 results is 80%, 69% and 57% respectively. We can also nd the combination of URL and snippet contents is eective in measuring the comparative information of two pages. The case studies also show our comparative page ranking function is able to nd those pages which contain comparison information relevant with both input queries. As Equation (1) indicates, both the comparative and relevance information help decide whether two pages form a meaningful comparative pair. We also did experiments to study which kind of information is more promising. In this experiment, the parameter is xed and , and are varied. The conclusion is: with the increase of , the precison of the pair matching grows steadily. This shows the relevance information between queries and pages has no impact on the pair matching result. The reason is: when the three subjects annotated the 20 queries, they only identied which two pages form a comparative pair. They did not rank the pairs according to their relevance scores with the input queries. When is small, even if those comparative

7. ACKNOWLEDGMENTS

We thank Dr. ChengXiang Zhai for insightful discussions and Liu Xin for his help on organizing the labeling work and implementing the CWS system. We also thank the reviewers for their valuable suggestions on this work.

8. REFERENCES

[1] Vivisimo website. http://vivisimo.com. [2] R. A. Baeza-Yates and B. A. Ribeiro-Neto. Modern Information Retrieval. Addison Wesley, 1999. [3] M. Chen, J.-T. Sun, H.-J. Zeng, and K.-Y. Lam. A practical system of keyphrase extraction for web pages. In CIKM, pages 277278, 2005. [4] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM
Table 3: Keyphrase Extraction Result for Query Pair: q1 =ChengXiang Zhai, q2 =Jiawei Han Common Keyphrases q1 Specic Keyphrases q2 Specic Keyphrases illinois, urbana, champaign mellon university, list, pakdd-2001 1. university, ltering, collaborative (44) tutorials research, system, database beespace, automated, news-gazette mining, participation, concepts 2. (44) online annual, information retrieval, em3. author, title, resource (44) data mining, data, anhai bedding information retrieval, research, an- mining, conference, data 4. author, track, kdd (24) hai 5. usa, tao, award (26) papers, zhai cs hong, zhang fa di, delete, business intelligence
Table 4: Keyphrase Extraction Result for Query Pair: q1 =Canon Sure Shot 130u, q2 =Olympus Stylus Epic Common Keyphrases q1 Specic Keyphrases q2 Specic Keyphrases canon 35mm, ebay canon, canon lm cameras, science stu, dlx 1. date, compact, kit (122) rebel 2. point, shoot, available (42) compare, canon buy, compact zoom, resnick, rambling 3. read, compare, epinion (26) cameras, shot 130u caption, canon dlx, electronic equipment, glorianas 8036a006 court photo, shot 130u 35mm camera, 4. price, bizrat, online (40) digital, save, day photo canon 35mm lm, shot 130u 35mm lm camera-mint, camera, compare 5. compare, nd, shopper (8) camera, cameras review, consumer, internet 6. lm camera, watch, digital video equipment used, rooks archives, cg (28) reviews, 35mm, shoot list- shoot, reviews canon, 35mm com- excite partner, photograph, out7. ings (12) pact door photographer
algorithm. J. of the Royal Statistical Society, Series B, 34:138, 1977. H. Fang, T. Tao, and C. Zhai. A formal study of information retrieval heuristics. In Proceedings of SIGIR 04, pages 4956, 2004. M. R. Henzinger, B.-W. Chang, B. Milch, and S. Brin. Query-free news search. In WWW 03: Proceedings of the 12th International Conference on World Wide Web, pages 110, 2003. M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of KDD 04, pages 168177, 2004. K. Kummamuru, R. Lotlikar, S. Roy, K. Singal, and R. Krishnapuram. A hierarchical monothetic document clustering algorithm for summarization and browsing search results. In Proceedings of WWW 04, pages 658665, 2004. B. Liu, M. Hu, and J. Cheng. Opinion observer: analyzing and comparing opinions on the web. In Proceedings of WWW 05, pages 342351, 2005. B. Liu, Y. Ma, and P. S. Yu. Discovering unexpected information from your competitors web sites. In Proceedings of KDD 01, pages 144153, 2001. B. Liu, K. Zhao, and L. Yi. Visualizing web site comparisons. In Proceedings of WWW 02, pages 693703, 2002. A. Nadamoto and K. Tanaka. A comparative web browser (CWB) for browsing and comparing web pages. In Proceedings of WWW 03, pages 727735, 2003. L. Page, S. Brin, R. Motwani, and T. Winograd. The pagerank citation ranking: Bringing order to the web.

Names of Parts

Selftimer/remote control button Shutter release button Zoom lever Viewfinder Light sensor Selftimer indicator Strap eyelet Lens barrier Remote control sensor Viewfinder Flash indicator Autofocus indicator Quartz Date mode buttons Back cover Diopter adjustment dial Film window Back cover release Battery compartment cover Mid-roll rewind button Tripod socket Lens Shooting mode button LCD panel Flash Autofocus windows Autofocus auxiliary light
Ultra-Compact All-Weather Full-Auto 38170mm Zoom 35mm Camera
Viewfinder Display & LCD Panel
Close-up correction marks Date/time indicator Selftimer Spot mode Autofocus mark Battery check Exposure counter
Flash indicator Autofocus indicator Flash modes Remote control

Main Specifications

Type: Film format: Lens: Shutter: Viewfinder: Focusing: Full automatic 35mm autofocus lens-shutter camera with built-in 38~170mm lens. 35mm standard DX-coded film (24 36mm). Olympus lens 38~170mm F4.8~13, 10 elements in 8 groups. Programmed electronic shutter. Real image zoom viewfinder. Advanced combination autofocus system. Focus lock possible. Focusing range: 2.6 ft.~ (infinity). Exposure Programmed automatic exposure control, 3-zone light metering, switchable to spot control: metering. Auto exposure range: Wide-angle EV2.6 (F4.8, 4sec.)~EV16 (F12.8, 1/400 sec.), Telephoto EV5.4 (F13, 4sec.)~EV17 (F22.9, 1/250 sec.). Selftimer: Electronic selftimer with approx. 12-sec. delay. Remote control (optional): Infrared remote control unit with approx. 3-sec. delay. Film speed Automatic setting with DX-coded film with ISO 50, 100, 200, 400, 800, 1600 and 3200. range: For non DX-coded film and film with less than ISO 50, film speed is set to ISO 100. Film loading: Automatic loading. Film advance: Automatic film winding. Film rewind: Automatic film rewind. Rewind possible at any point with rewind button. Flash: Built-in flash. Recycling time: Approx. 0.5~5.5 sec. (at normal temperature with new battery). Flash working range: Wide-angle 2.6~16.4 ft. Telephoto 2.6~5.9 ft. with ISO 100 color negative film; Wide-angle 2.6~32.8 ft. Telephoto 2.6~11.8 ft. with ISO 400 color negative film. Flash modes: Auto Flash, Red-Eye Reduction Flash, Flash Off, Fill-In Flash, Night Scene Flash and Red-Eye Reduction Night Scene Flash. Data coding: No data, year-month-day, month-day-year, day-month-year and day-hour-minute. Automatic calendar system: Up to year 2035. All-weather: IEC standard publication 529 (protected against water splashed from any direction). Power source: One 3V lithium battery (DL123A/CR123A). Dimensions: 4.8 (W) 2.6 (H) 2.0 (D) in. (without protrusions). Weight: 9.7 oz. (without battery).
Appearance and specifications are subject to change without notice. Example photographs shown in this catalog are included to illustrate camera features and functions.
Internet: http://www.olympus.com Telephone: 1-800-622-6372

Olympus business areas

Medical and health-care area
Imaging and information area
Industrial applications area
Tokyo, New York, Hamburg, London, Paris
C1330-0101D2 Printed in Japan
A new dimension in compact zoom power and easy-to-use photographic design elegance

The Olympus Stylus Epic Zoom 170 brings awesome zoom power to our elegant Stylus camera line-up. Incredibly light and compact, it has a 4.5x lens with 170mm telephoto capability that makes distant subjects seem close enough to touch. Whats more, it boasts an advanced combination autofocus system and an intelligent variable-power flash with six versatile flash modes. Offering full-featured photo creativity in a stylish, all-weather body, the Stylus Epic Zoom 170 is everything a compact camera should be.
Wide-Ranging 38~170mm Zoom Power The powerful 38~170mm 4.5x zoom lens features a number of optical design innovations. An Extra-low Dispersion (ED) glass lens element reduces color aberration, while two aspherical elements provide edge-to-edge sharpness and enhanced image clarity. Whats more, our advanced combination autofocus system, which combines the advantages of active and passive autofocusing, ensures spot-on accuracy with virtually any subject. You can also shoot from as close as 32 inches at any zoom setting.
Extra-low Dispersion (ED) Lens

70mm 110mm

Auto Mode The intelligent variable-power flash fires automatically when needed, providing soft-flash illumination at close range, and auto color-balancing flash in artificial light. Fill-In Mode Fires the flash with every shot, assuring correct exposure in high-contrast lighting conditions. Off Mode For available-light shooting, or when you are in a location where flash photography is prohibited. Red-Eye Reduction Mode Suppresses unsightly red-eye by emitting a series of preflashes that help your subjects eyes adjust to the bright light of the main flash.
*Results may vary depending on subjects eye color and exposure to pre-flash.

With Conventional Flash

FUNCTIONS
All-Weather Construction Special seals and coatings allow worry-free shooting in any weather. Spot Mode Solves tricky lighting problems by limiting exposure readings to a specific area of the frame. Dioptric Correction An adjustments dial allows you set viewfinder focus to suit your preference.
With Red-Eye Reduction Flash Mode
Night Scene / Red-Eye Reduction Night Scene Mode At night, these modes illuminate your subject while ensuring that lights in the background show up beautifully.
Electronic Selftimer A 12-sec. delay gives you plenty of time to get in the picture too.
With Night Scene Flash Mode
Remote Control The optional RC-200 infrared remote control lets you release the shutter from up to 16 feet away!

RC-200