CSCI 373 Information Retrieval and Web Search: Search Engine Case Studies

CSCI 373: Search Engine Case Studies

Search Engine Assignments

Chris, Angela, Rick, & Travis	HotBot
Alan, Stephen, Aida, & Jonathan	www.askjeeves.com
Dave, Elizabeth, Adam, & Troy	www.dmoz.org

Objective

The objective of this assignment is to learn more about existing web search engines from the prospective of someone who is both an informed user and a search engine designer. To that end, each team of 3 or 4 people will select an existing search engine (excluding Google) to study and give a 15 minute class presentation of their findings on March 27th.

To assure that no two groups select the same search engine, each group must submit their choice no later than March 20th. The first group to request a particular engine (excluding Google) gets it. Here is a summary (one of many) of the major search engines available today.

Guidelines

Each case study should evaluate one search engine. The evaluation should include both a research (i.e., reading papers and reviews) and an experimentation (i.e., using the engine) component. Your task is to identify the important design features of the engine you select and then conduct a comparative evaluation of that engine's performance.

Search Engine Features

It is important to provide as much insight as possible into the design and implementation of the engine you select. Examples of design features are:

the types of queries supported
the assignment of relevance to retrieved documents
support for relevance feedback
indexing techniques
interface features
web coverage

Unfortunately, with the exception of Google, few search engines are well documented. Because Google is the current leading search engine and because it is well documented, you should compare your chosen engine to Google whenever possible. To assist you, here is a paper on Google presented at the Seventh International World Web Conference in Brisbane, Australia during April 1998.

Comparative Performance Evaluation

At a minimum, you should evaluate the performance of your search engine relative to Google plus the UNCA library catalog. Evaluate user interface features (i.e., ease of use) as well as search performance. Measures of search performance might include:

total number of documents returned
number of deadlinks returned
overlap in documents returned by different engines
relevance of the documents returned

Your performance evaluation should be conducted using a small number (i.e., less than 4) of very specific queries. Example queries are:

Information on the history of information retrieval.
Information on indexing structures used in information retrieval.
Information on user interface features important in information retrieval.

Grading

Your grade will be based on content and clarity of your 15 minute presentation.