Re: Creating soup packets

From: Allen Scott-Thoennes (sthoenna@efn.org)
Date: Wed, 23 Jul 1997 00:12:57 -0700

In article <NzyzzYhtjH7F092yn@mozart.inet.co.th>,
brianj@mozart.inet.co.th (Brian Jackson) wrote:
>In article <CZnzz4uYOZSK089yn@stack.nl>,
>galactus@stack.nl (Arnoud "Galactus" Engelfriet) wrote:
>>
>>Basically, you need to run your own news *server* (INN, usually)
>>and then get your ISP to "peer" with you. However, this does not
>>allow for filtering before downloading news, since you're then
>>communicating on the "Hey, I have an article, want it?" "Yeah sure,
>>send it" level.
>
>It sounds like this would not be an option for us over here. It seems
>like the best solution will be to gather news from our own server
>(once its upgrade is finished, I expect it will have a more complete
>feed) and maybe one other server to round it out. As I said, we have
>to keep our online time down to a minimum.
>
>Thanks for the info.
>Brian

A project I started working on quite a while ago but then had to shelve
for lack of time involved enhancing uqwk to allow an alternate news server.

The basic plan (if anyone else wants to take this on...) was:

get news exactly as usual from primary news server (NNTPSERVER)
read from a new config file ".uqwkrc" to get a timestamp for the
last date and time that an alternate news server (specified via
new command line switch or environment var) was opened
open the alt news server
get current date and time from server (save it and update timestamp in
.uqwkrc *after* everything completes successfully)
for each group in .newsrc request a list of message ids new since the
stored timestamp
for each message id, query the local server. if and only if the article
is *not* available there, fetch it from the alternate server

The basic assumption here is that the alternate server is presumably
a public one and thus swamped. By getting the message ids and only
fetching articles not available locally, the interaction with the
alternate server should be drastically reduced. The tradeoff is
more of a burden on the local server.

For bonus points:
add ability to specify all current options in .uqwkrc
allow multiple alternate servers
when the local server is down, automatically get all articles from the
alternate server
allow a separate list of newsgroups for the alternate server
allow some groups to be fetched via normal article number methods
abandon 'query the local server' step in favor of a database of
most recent x number of message ids