Re: Duplicates

0 (ten.eerfrot@445sa)
Mon, 25 Nov 1996 20:26:25 +30000

Tue, 26 Nov 1996 16:45:27 -0500, jstanley@gate.net (John A. Stanley) wrote:
>Yeah, Yarn takes care of it, but you _will_ know about it if you see
>the duplicates being eliminated during import. Alas, with my new 200
>MHz Pentium Pro imports scroll by so quickly I can no longer read it.

I run import through a batch file which backs up the soup packet (for
safety) and redirects the output to a file which is then appended to a yarn
folder I call "import.log" (with my Text 2 Folder rexx script) ... which is
kept trimmed to the previous thirty days of imports (with my Foldup rexx
script) for reference in case something goes wrong, or I need to know for
some unknown reason. Much fun.

(how's that for scripting, Phil? <-: )

Now i'm thinking of adding a new script which parses the import outputs
and keep running statistics on the whole thing. If i have too much time on
my hands....

And yes, i feel so sorry about your problem with your Pentium Pro. So sad.
Maybe you could hook an old RLE hard drive up to it to slow it down a bit.
(-;

I have been informed (in great detail!) via email of the many circumstances
in which the Message-ID could be changed on duplicate messages -- so yarn
woudln't recognise them as Dupes. I demure... it could happen. Hasn't
happened to me, but could happen.

I suggested Yarn forget about Message-ID's and do an MD5 hash of the body
of each message coming in and judge dupes that way. <-: And on your pentium
pro you won't notice the slow down. Heh.

Um, what were we talking about again?

-- 
 .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
  Tum Myddleten =-= with love and a banana fish =-= as544 torfree.net
  -=-=-= now out of school, read Hamlet by William Shakespeare =-=-=-
 ~`~`~'~`~`~'~`~`~'~`~`~'~`~`~'~`~`~'~`~'~'~`~'~'~`~'~'~`~'~'~`~'~'~`~
    Yarn/2 Bells & Whistles Page: http://www.io.org/~tm/bells2.html
          * November 25th * International Day of the Big Bang