Subject: Re: Propose some information retrieval protocols for Internet

From: "wang liang" <[email-masked]>
Date: Fri, 26 Dec 2003 11:17:26 +0800

> I believe you are talking about information *indexing* service,
> not "information *retrieval* service"

DRIS will build the information retrieval infrastructure for Internet, but
not the final search engines. Many intelligent search systems can apply DRIS
as their data source and provide high quality of personal search service.

> DRIS is only a design.
> When you say "build", who will do it? How does it get paid?
> (i.e. What is the incentive for anyone to do this?)

I have mentioned in our paper, DRIS will improve the performance of Internet
search engine in recency, coverage and so on, but this can't ensure the
establishment of DRIS.

The architecture of DRIS is organization level-sub country Internet
level-country level-whole Internet level. DRIS will first solve some urgent
problems in the bottom level, then to the top level. Just in our testbed,
CERNET (China education and research network), few universities have the web
search engine for the school network. Further more, most university has many
characteristic information resources such as Ftp, BBS and special databases
in library, but almost no a university has union search system that can
efficient integrate all these resources. To find the comprehensive, we
always have to search in many search interfaces one by one. It's the problem
in organization level. Then there still no an efficient to share these
resources among different universities. It's the problem in sub Internet
level. These all bring the request for creating the underlying structure of
DRIS. Solving some urgent problems of his own and then benefiting others may
be the real guarantee for the success of DRIS.

Who control the DRIS?It's administrated by none of us but every of us. DRIS
is managed by its users and coordinated by a public organization, just like
management method of DNS. Every organization is its customer and also its
builder and manager. It's just the real truth of Internet. DRIS is a public
opening system, which needn't any profits from its users and of course need
not any advertisements and Spam of company.

> I had a quick look of your first paper.
> - It seems to suggest that each DNS domain has a central authority,
>   which may not be the case
> - It is unclear to me DNS domains are the right unit for indexing
>   webs, as opposed to topical areas.

Current search engines all managed by corresponding company. This is the
centralized management method. This method is not suitable to manage the
information on Internet. Now there are billions of web pages, millions of
databases and many other kinds of information resources on Internet. Search
engines will encounter many bottleneck problems when the size of its
database reaches some critical values. In fact, just as a web pages search
engine, it can't continue to index close to the entire Web as it grows. Now
the update interval of most pages database is almost one month. We can also
obtain information from different special databases like IEEE's digital
library, FTP, P2P, etc. Could you image single private company can
efficiently administer all these information resources?

Every search engines try to provide the comprehensive and fresh information
for its users, but none of them would build a database system that can
mirror the whole Internet.

So a distributed management frame may be more appropriate for Internet. As
our experience, decentralized management is much more effective than
absolutely centralized administration in a large-scale system. By this
means, the key issue is how to divide the Internet correctly. We found there
has been an available division method on Internet, domain name system (DNS).
DNS is a hierarchical distributed system. All the web site on Internet is
efficiently managed in this system. The basic architecture of DNS is also an
organization level-sub country internet level-country level. We just apply
its basic idea to DRIS, but not strictly comply it.

>
> bottomline: yes the current google domination is not sustainable, however
> the basis of DRIS design raises its own problems.

Although I can't say DRIS is better than Google at now, but it can surely
meet some demands that Google can't fill. In fact,DRIS will build a system
than can integrate all kinds of resources, but not only web pages. The first
testbed of DRIS on CERNET will be finished in 2004 fall. Practice is the
only principle to judge a theory. More discussion is also very important for
a new system.

---

From: "Stephen Sprunk" <[email-masked]>
Date: Wed, 31 Dec 2003 05:47:35 -0600

Thus spake "wang liang" <[email-masked]>
> as their data source and provide high quality of personal search service.

Why do we need DRIS if we already have HTTP/FTP/etc?  Put another way, how
are HTTP et al insufficient for retrieving information once its location
(i.e. URL) has been determined?

Your original post presents several arguments against the current crop of
search engines and proposes a new distributed search engine system, but this
is an _indexing_ function, not a _retrieval_ function.

> level-country level-whole Internet level.

Can you demonstrate why country-level databases are a technical advantage,
other than providing certain governments the ability to restrict the
information their citizens can access?

> You could find more information about our project in
> http://202.114.9.200/English/main.htm
> More content will be translated in these days.

It appears your site isn't accessible many places outside CERNET/ChinaNet;
can you post your papers somewhere with better reachability?

S

Stephen Sprunk        "Stupid people surround themselves with smart
CCIE #3723           people.  Smart people surround themselves with
K5SSS         smart people who disagree with them."  --Aaron Sorkin