A System for Collaborative Web Resource Categorization and Ranking

Ph.D. Dissertation Proposal

Department of Computer Science,
SUNY at Stony Brook,
Stony Brook, NY 11794, U.S.A.
maxim@cs.sunysb.edu

Adviser: Prof. Tzi-cker Chiueh

October 2001

Abstract

The Web is quickly becoming more and more vast, comprehensive, and used source of various types of information and services for a large fraction of human population. As a result tools --currently exemplified primarily by search engines and classification directories-- that allow web users to navigate the Web to quickly locate high quality information resources and services to satisfy their specific needs are becoming more and more important. At the same time, the demands on the quality of service of such tools in terms of their ability to return unbiased results tailored to the needs of each particular user, as well as in terms of the amount of data and user queries they must handle become more and more exacting. In this proposal we examine the problem of creating an adequate tool for global web navigation in detail. We start with high-level abstract analysis of the problem and associated restrictions on the possible solutions, and continue by discussing and analyzing the features and approaches of existing systems for global web navigation including modern link-based web page ranking methods and collaboratively maintained web resource directories. We then present our approach for building a global web navigation service, which relies on democratic collaborative mechanism to aggregate individual preferences of interested parties about global web navigation into a view that is presented to the end users. More specifically, we propose to let web page authors specify on their pages their desires regarding global web navigation such as what textual fragments best describe a particular web resource, which resource is better with respect to a given category in a classification directory, which subcategories a given category should have, and which other web authors are delegated the power to provide such preferences instead of the original author. Such information can be extracted from explicitly prepared metadata embedded into web pages; some subclass of it can also be derived from linkage and text of regular web pages and from snapshots of a popular web directory. The algorithms we propose can then aggregate all such metadata items to arrive at collaboratively constructed web directory and a set of preferences regarding which web resources are best suited as answers to various keyword based queries. We show that our approach subsumes the ranking model of Google search engine and possesses a set of useful properties that make it a powerful method for building a high quality global web navigation service. On the implementation side we present our experience developing a partial prototype of the proposed navigation service by discussing its architectural requirements and design decisions made to satisfy the requirements. Our prototype is currently roughly comparable in terms of functionality, scalability, and quality of search results to the research prototype of the Google search engine.