COLLOQUIUM
Department of Computer Science and Engineering 
University of South Carolina
 
Clustering, Resource Management, and Replication 
Support for Large-scale Internet Services

Kai Shen
Department of Computer Science
University of California at Santa Barbara 

Date: March 8, 2002 (Friday)
Time:  4:00-5:00PM
Place: 300 Main B110

Abstract

With the increasing demand of providing highly scalable, available and
asy-to-manage services, the deployment of large-scale complex server
clusters has been rapidly emerging in which service components are
usually partitioned, replicated, and aggregated.  In this talk, I will
present my research in building a software infrastructure(Neptune)
that provides clustering, resource management, and replication support
for large-scale internet services.  Neptune has been deployed at
Internet search engines Teoma and Aks Jeeves in a commercial system
serving 6-7 million search queries per day.  Neptune's clustering
architecture is based on the design principles of symmetry,
decentralization, non-scaling overhead, and soft state to achieve high
scalability, availability, and manageability.  I will first talk about
Neptune's overall clustering architecture, followed by a brief
discussion on the load balancing support for fine-grain services.  In
the second part of the talk, I will present an integrated resource
management framework combining resource utilization efficiency,
consumer-perceived service response time, and service differentiation
support.  In the end, I will present my work on service replication.
Replication of persistent service data is crucial to achieving high
availability. My work focused on providing scalable replication
support at the intended consistency level with high availability and
failure recovery support.

Kai Shen is currently a Ph.D. student in the Computer Science
Department at the University of California at Santa Barbara.  His
research interests include parallel and distributed systems, scalable
network services, Internet search, and high performance scientific
computing.  He worked for Microsoft and Internet search engine Teoma
Technologies (acquired by Ask Jeeves).