Subject: Re: [MORG] WGLC on draft-ietf-morg-fuzzy-search-02; respond by 10 Feb 2010

From: Barry Leiba <[email-masked]>
Date: Mon, 1 Feb 2010 14:47:07 -0500

Here is a reminder that we are a week into WGLC for the subject
document (see details below), and I have not seen any comment on it
yet.  Please comment on the draft, even if it's just to say that you
think it's ready to go.  WGLC ends on 10 February.

> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the Message Organization Working Group of the IETF.
>
>        Title           : IMAP4 Extension for Fuzzy Search
>        Author(s)       : T. Sirainen
>        Filename        : draft-ietf-morg-fuzzy-search-02.txt
>        Pages           : 6
>        Date            : 2010-01-23
>
> This document describes an IMAP protocol extension enabling server to
> perform searches with inexact matching and assigning relevancy scores
> for matched messages.
...
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-morg-fuzzy-search-02.txt

Barry, MORG secretary

---

From: Mark Crispin <[email-masked]>
Date: Mon, 1 Feb 2010 12:11:13 -0800 (PST)

You want comments?

Here's my comment:

This document, and from what I can tell the other work of MORG, is best 
suited for an April 1 RFC.

It is a solution in search of a problem.

The IMAP Extensions WG was supposed to have ended years ago.  Instead, it 
found a new home in LEMONADE, and now MORG.  I wish that the IESG would 
apply some discipline here!

The ostensible mission of the IETF is to solve problems that need solving. 
It is not to be a sinecure for efforts that have long been overtaken by 
events, and working groups that were long ago supposed to die.

On Mon, 1 Feb 2010, Barry Leiba wrote:
> _______________________________________________
> MORG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/morg
> Note Well: http://www.ietf.org/maillist.html
>

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

---

From: Timo Sirainen <[email-masked]>
Date: Mon, 1 Feb 2010 22:21:23 +0200

On 1.2.2010, at 22.11, Mark Crispin wrote:

> This document, and from what I can tell the other work of MORG, is best suited for an April 1 RFC.
> 

Many IMAP server implementors' solution to this problem is to just do it already and unconditionally, breaking RFC 3501 compatibility. So it is already a problem that some people "solved" without the help of this document. What this document is trying to do is to make it possible for servers that actually care about being RFC 3501 compatible to do what users/clients typically want to do, and what some other non-conforming servers already do in reality (and providing better user experience for typical users).

---

From: Barry Leiba <[email-masked]>
Date: Mon, 1 Feb 2010 16:01:54 -0500

> Many IMAP server implementors' solution to this problem is to just
> do it already and unconditionally, breaking RFC 3501 compatibility.
> So it is already a problem that some people "solved" [...]

[...] in a non-standard way.

To be fair to Mark's comment, it's always been his contention that
"string" searches (as opposed to flag searches, certainly, or date
searches, perhaps) could always be "fuzzy" without violating the
intent of the IMAP spec.  The spec necessarily makes sure that string
searches are well defined, but "violations" of that aren't considered
violations of the protocol.  Lots of reasonable IMAP servers do
creative things with string searches today.

That's a major reason that Mark doesn't see any need for this doc, in
particular.

In general, Mark thinks there are too many extensions that have been
and are being proposed for IMAP, a spec that some think is already too
involved.  He sees little or no value in yet more extensions that'll
be rarely, if ever, implemented.  Mark isn't alone in this opinion.

That's why I've been pushing the question, for each spec that we're
working on, of who will implement it.  We got a good response to that
for fuzzy search, so we're going ahead.  We got a good response for
multimailbox-search, as well.  We haven't had much commitment to
implement some of the other proposals, and I'll continue to ask the
question.

I don't agree with Mark that we should essentially shut off IMAP
extension work.  But I partially agree that we're throwing them out
too readily.  We might think there's no harm in having them, and if
they don't get implemented, "So what?", and we'll just let the market
decide.  The danger with that is that it's like pouring a beachful of
sand on some nice seashells, and then asking collectors (IMAP
implementors) to find the nice shells.  The myriad extensions can get
in the way of clarity.

So, while Mark's being a little abrasive in his manner (he's MARK,
yes?), don't write off his comment out of hand.

Barry, as a participant

---

From: Mark Crispin <[email-masked]>
Date: Mon, 1 Feb 2010 13:06:34 -0800 (PST)

On Mon, 1 Feb 2010, Timo Sirainen wrote:
> Many IMAP server implementors' solution to this problem is to just do it 
> already and unconditionally, breaking RFC 3501 compatibility.

At least one of the servers that does this does does not have any 
capability to do any other type of search, and it would be extremely 
difficult to add substring search.

That's OK.

RFC 3501's purpose in specifying substring search was not to prohibit the 
use of a fuzzy search in IMAP SEARCH, but rather to protect against 
someone claiming in the future that substring search is not compliant.

> So it is 
> already a problem that some people "solved" without the help of this 
> document.

That solution was anticipated from the inception of IMAP.

> What this document is trying to do is to make it possible for 
> servers that actually care about being RFC 3501 compatible to do what 
> users/clients typically want to do, and what some other non-conforming 
> servers already do in reality (and providing better user experience for 
> typical users).

This makes two (in my mind) false assumptions.

The first is the assumption that the majority of servers will offer both 
substring and fuzzy searching; or even that the majority of servers that 
have fuzzy searching will offer substring searching.

The second is that clients desire to control this.

-- Mark --

http://panda.com/mrc

---

From: Timo Sirainen <[email-masked]>
Date: Mon, 1 Feb 2010 23:18:45 +0200

On 1.2.2010, at 23.06, Mark Crispin wrote:

>> Many IMAP server implementors' solution to this problem is to just do it already and unconditionally, breaking RFC 3501 compatibility.
> 
> At least one of the servers that does this does does not have any capability to do any other type of search, and it would be extremely difficult to add substring search.

I know.

> That's OK.
> 
> RFC 3501's purpose in specifying substring search was not to prohibit the use of a fuzzy search in IMAP SEARCH, but rather to protect against someone claiming in the future that substring search is not compliant.

Really? You had exactly the opposite opinion 3 years ago:
http://mailman2.u.washington.edu/pipermail/imap-protocol/2006-December/000328.html

---

From: Barry Leiba <[email-masked]>
Date: Mon, 1 Feb 2010 16:18:55 -0500

> The second is that clients desire to control this.

I don't think anyone thinks clients want to control it; they think
users do.  This user, at least, would certainly like to.  There are
many times when I *do* want to disable the fuzzy search or the
word-bounded search that the server does today.  Very, very much.

But I absolutely agree with Mark's characterization of the first
assumption.  The commercial IMAP server I'm most familiar with, IBM's
Lotus Domino server, does its own style of search that we'd definitely
call "fuzzy".  If you search for "walked", you'll also get matches for
"walk", "walks", and "walking".  And if you search for "tall", you'll
match "taller" but not "tally".  I can not imagine that they will ever
change that, nor that they might offer an alternative.  [I couldn't
speak for them when I worked there, and I certainly can't now, but I
think this is a very safe bet.]

Similarly, I'd be surprised if Google should offer a strict substring
search as an option.  Do we think they plan to?

And, so, however much users might like the option, do we *really*
think they'll get it in practice?

Barry

---

From: Barry Leiba <[email-masked]>
Date: Mon, 1 Feb 2010 16:23:08 -0500

By the way; I meant to do it on my last reply, but forgot to: let's
take the IESG off the CC list for the rest of this discussion.  The
point has been made to them, and they don't need the details of our
further conversation.

Barry

---

From: Mark Crispin <[email-masked]>
Date: Mon, 1 Feb 2010 13:28:25 -0800 (PST)

> be rarely, if ever, implemented.  Mark isn't alone in this opinion.

Apropos to this, I refer you to all to my seminal work RFC 748; and to 
ponder its true meaning.

There were only a handful of important TELNET options.  Opinions vary, but 
I number them as three; and one of these was to disable a mandatory TELNET 
facility that served no purpose unless your terminal was an IBM 2741.

TELNET already had an order of magnitude more options at the time RFC 748 
was written.

The last time that I actively followed the IETF follies, there was a 
working group dedicated to improving TELNET long after ssh replaced it. 
That, along with the recent publication of the USEFOR and NNTP 
specifications, is a stirling example of the IETF solving problems that 
were long overtaken by events.

-- Mark --

http://panda.com/mrc

---

From: Barry Leiba <[email-masked]>
Date: Thu, 18 Feb 2010 20:50:47 -0500

On Mon, Jan 25, 2010 at 8:34 PM, Barry Leiba
<[email-masked]> wrote:
> This message starts working-group last call on this document.  Please
> review this version of the draft, even if you have reviewed a previous
> version, and post comments by 10 Feb.  Post your comments as a reply
> to this message, and please also respond if you think the document is
> ready for publication -- we need confirmation, as well as issues.

The only last-call comment on this was from Mark.  I'm extending the
WGLC until 3 March.  Let's have some comments on this draft, one way
or another.

Otherwise, if we don't have enough participants to have meaningful
output from this working group, perhaps we should stop work and close
it down.

Barry, MORG secretary