Subject: [MORG] SEARCH=INTHREAD

From: Timo Sirainen <[email-masked]>
Date: Tue, 13 May 2008 07:49:11 +0300

I'm planning on implementing this pretty soon (I need SEARCH INTHREAD
feature). Two issues though:

1) SEARCH THREADROOT sounds like it should return a list of roots for
all threads (should "For each thread in the message" be "in the mailbox"
instead?). The problem is with threads where the root is a dummy instead
of a message. I guess SEARCH THREADROOT just doesn't return anything
with them? I can't really think of a good alternative for that behavior.

2) THREAD=REFS algorithm. It has 4 differences from THREAD=REFERENCES:

1. In-Reply-To: header is ignored.
2. Date: header is ignored (and I assume INTERNALDATE is too?)
3. Base subject merging isn't done
4. Server is allowed to return "wrong" THREAD replies because of caching

I think 1 shouldn't be done. Using the same base threading algorithm as
THREAD=REFERENCES makes it a lot easier to implement THREAD=REFS as well
as making a better result.

Why 2? With THREAD command it seems useful to sort messages in threads
by their Date: header so they could be read in correct order (but for
root sorting I prefer sorting by newest-INTERNALDATE-in-thread). With
SEARCH INTHREAD command the server is free to ignore Date: header (and
INTERNALDATE) also with THREAD=REFERENCES, because it doesn't affect the
result. Although with SEARCH THREADROOT/THREADLEAF the date may affect
the result with THREAD=REFERENCES because of subject merging.

3 I agree with and 4 I'm not sure.

I was planning on writing a draft about THREAD=REFERENCES2. It basically
sorts thread roots using newest-INTERNALDATE-in-thread and doesn't do
base subject merging. This also means that SEARCH
INTHREAD/THREADROOT/THREADLEAF work correctly without looking up dates.
So I'd think it accomplishes all goals of THREAD=REFS except for 4.

If 4 is really wanted, I'm beginning to think maybe the entire algorithm
should be changed to be described simply as "server-dependent
implementation" where the server is free to use whatever algorithm it
thinks is the best (could even be user-configurable). It could renamed
then to THREAD=SERVER or something.
_______________________________________________
MORG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/morg
Note Well: http://www.ietf.org/maillist.html

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 10:09:02 +0200

Timo Sirainen writes:
> all threads (should "For each thread in the message" be "in the 
> mailbox" instead?). The problem is with threads where the root is a 
> dummy instead of a message. I guess SEARCH THREADROOT just doesn't 
> return anything with them? I can't really think of a good alternative 
> for that behavior.

THREADROOT was difficult. I haven't implemented it. The root of a thread 
seems to be a fuzzy concept.

> 4. Server is allowed to return "wrong" THREAD replies because of caching

No, let's use i-r-t.

I don't use Date, but that's, hm, orthogonal. The code I have doesn't 
make a list of threads, it makes a set of threads, so sorting threads 
by date or anything else is not something I've had to do.

The THREAD command has to sort threads. I don't care how it's done - as 
REFERENCES, as REFERENCES2 or another way. All are fine IMO.

> I think 1 shouldn't be done. Using the same base threading algorithm 
> as THREAD=REFERENCES makes it a lot easier to implement THREAD=REFS 
> as well as making a better result.
>
> Why 2?

It just slipped my mind. I somehow didn't realize that the part I hadn't 
written needed Date.

> With THREAD command it seems useful to sort messages in threads by 
> their Date: header so they could be read in correct order (but for 
> SEARCH INTHREAD command the server is free to ignore Date: header 
> (and INTERNALDATE) also with THREAD=REFERENCES, because it doesn't 
> affect the result. Although with SEARCH THREADROOT/THREADLEAF the 
> date may affect the result with THREAD=REFERENCES because of subject 
> merging.
>
> I was planning on writing a draft about THREAD=REFERENCES2. It 
> basically sorts thread roots using newest-INTERNALDATE-in-thread and 
> doesn't do base subject merging. This also means that SEARCH 
> INTHREAD/THREADROOT/THREADLEAF work correctly without looking up 
> dates. So I'd think it accomplishes all goals of THREAD=REFS except 
> for 4.

I'd like to have only one of THREAD=REFS and THREAD=REFERENCES2, so if 
we can find a way to make one serve both of our needs, that would make 
me happy.

I don't care very much about 4. I assume my server will be noncompliant 
of 4 is dropped, but that doesn't scare me in this case. (I used to 
think there would be differences in practice, but don't think so any 
more.)

If I understand it correctly, then this would satisfy us both:

1. Use References.
2. Use In-Reply-To exactly as THREAD=REFERENCES does.
3. Don't use Subject.
4. Sort the threads by the thread's most recent internaldate.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 10:19:24 +0200

Dave wanted THREADROOT and THREADLEAF, and I agree they make sense. 
However, I'm not sure they carry their load. How about these 
definitions?

THREADROOT matches a message if none of the message's ancestors (as 
listed in References and In-Reply-To) exist in the current mailbox.

THREADLEAF matches a message if no messages in this mailbox list its 
message-id in References or In-Reply-To.

Are there good enough use cases to add these? (They're tricky for me to 
implement.)

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 13 May 2008 11:39:05 +0300

> seems to be a fuzzy concept.

Maybe in case of a dummy root return all its children as roots? Probably
more useful than not returning them at all.

I can think of a couple of use cases for THREADROOT, but is THREADLEAF
actually useful for something? But I guess it would be simple enough for
me to implement, so I don't mind that much.

> we can find a way to make one serve both of our needs, that would make 

That would be better, yes. I also don't care if it's called REFS or
REFERENCES2.

> think there would be differences in practice, but don't think so any 

I'm mainly concerned that such sentence would be a loop hole for a lazy
server developer. "It's not returning a correct reply because it's
caching some data" is an easy reply for all bug reports. :)

> 2. Use In-Reply-To exactly as THREAD=REFERENCES does.

1. also "exactly as THREAD=REFERENCES does".

> 4. Sort the threads by the thread's most recent internaldate.

Yes. I had thought about writing something like:

THREAD=REFERENCES2 changes THREAD=REFERENCES algorithm in two ways:

1) Skip the step "(5) Gather together messages under the root that have
the same base subject text".

2) Replace the step (4) sorting with the following algorithm:

Initialize each thread root's sort-date to non-existing. For each
(non-dummy) message get its INTERNALDATE and find its thread root. If
the thread root's sort-date is non-existing or the INTERNALDATE is
higher than sort-date, update sort-date to INTERNALDATE. Finally sort
all the thread roots by their sort-date. Because all threads have at
least one non-dummy message, there should be no non-existing sort-dates
left when sorting.

Doesn't really seem worth having a separate RFC for that, so if you can
change THREAD=REFS to do something similar that'd be great. :)
_______________________________________________
MORG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/morg
Note Well: http://www.ietf.org/maillist.html

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 13 May 2008 11:45:47 +0300

> message-id in References or In-Reply-To.

I just realized that these don't specify the threading algorithm. I
don't think they're very useful with these kind of static definitions,
since THREAD command may give a completely different kind of a thread
tree.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 11:50:01 +0200

Timo Sirainen writes:
> don't think they're very useful with these kind of static 
> definitions, since THREAD command may give a completely different 
> kind of a thread tree.

Indeed. But how do you specify THREADLEAF in orderedsubject?

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 13 May 2008 13:27:13 +0300

> Indeed. But how do you specify THREADLEAF in orderedsubject?

Each thread has one root and the rest are leaves :) But I was more  
thinking about REFERENCES vs. REFERENCES2.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 12:44:50 +0200

Timo Sirainen writes:
>> I'd like to have only one of THREAD=REFS and THREAD=REFERENCES2, so 
>> if we can find a way to make one serve both of our needs, that would 
>> make me happy.
>

Btw. I care at least a little. I don't like the name THREAD=REFS and 
actively dislike the name THREAD=REFERENCES2. T=R2 reminds me of the 
Windows API, which has many such names. For example, CreateWindow() 
takes maybe a dozen arguments, but they forgot one, so they added 
CreateWindowEx() with n+1. IMNSHO a function with a dozen arguments has 
a problem, but it's not that it lacks a thirteenth.

I know it's irrational and not at all related to IMAP, but I hope you 
forgive me for hating anything that reminds me of that %@#!$@#$!#$#!$.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 13 May 2008 13:48:59 +0300

On May 13, 2008, at 1:32 PM, Arnt Gulbrandsen wrote:

> Timo Sirainen writes:
>> Each thread has one root and the rest are leaves :)
>
> I thought it would be more likely that the thread has one root and  
> one leaf, and the rest are intermediate nodes.

Oh, could be. I've never implemented orderedsubject so I don't  
remember if it was (1 2 3) or (1 (2)(3)). Anyway I don't think it  
should be a special case. It's already defined what the reply thread  
tree looks like, so THREADLEAF lists the leaves from the returned tree.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 12:32:31 +0200

> Each thread has one root and the rest are leaves :)

I thought it would be more likely that the thread has one root and one 
leaf, and the rest are intermediate nodes.

> But I was more   thinking about REFERENCES vs. REFERENCES2.

Yes, the useful stuff is simple, it's specifying the thing so rest isn't 
damaging that's difficult.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 May 2008 12:30:47 +0200

Timo Sirainen writes:
> Maybe in case of a dummy root return all its children as roots? 
> Probably more useful than not returning them at all.

Depends on the use.

> actually useful for something? But I guess it would be simple enough 
> for me to implement, so I don't mind that much.

Dave?

THREADLEAF is basically "messages to which noone has replied", which 
seems interesting, but not necessarily worth standardising.

> Doesn't really seem worth having a separate RFC for that, so if you 
> can change THREAD=REFS to do something similar that'd be great. :)

Will do.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 15 May 2008 04:59:31 +0300

On May 13, 2008, at 1:44 PM, Arnt Gulbrandsen wrote:

>>> I'd like to have only one of THREAD=REFS and THREAD=REFERENCES2,  
>>> so if we can find a way to make one serve both of our needs, that  
>>> would make me happy.
>>
> actively dislike the name THREAD=REFERENCES2.

THREAD=REFSONLY maybe?

BTW. Why don't I see your messages in morg list even though it's Ccd?

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 15 May 2008 05:08:38 +0300

> BTW. Why don't I see your messages in morg list even though it's Ccd?

Oh, duplicate suppression is enabled by default. I hate that feature.