Subject: [imapext] IMAP clients

From: Brandon Long <[email-masked]>
Date: Thu, 2 Aug 2012 14:56:28 -0700

On Thu, Aug 2, 2012 at 2:26 PM, Adrien de Croy <[email-masked]> wrote:

>
> ------ Original Message ------
> From: "Ned Freed" <[email-masked]>
>
>> You might want to look at statistics for typical mixes of clients seen by
>> servers then, with an eye to how many old versions are in use.
>>
> most of our customers (and admittedly I'm talking a fairly restricted use
> case) are using 1 or more of
> outlook or outlook express or Thunderbird
> and/or IOS mail client
>
> We really don't see significant use of other clients.  But this is not
> rigorous or general by any means.

We try to keep track of what clients are using our IMAP service.  If you're
willing to log all FETCH requests, its not that hard to at least bucket by
client (yeah for a protocol so general that every client implements it
differently).

Doesn't help overly much with versions, though.  We've been trying to
encourage client authors we deal with to use the ID extension so we can do
a better job of understanding when things break, but obviously that's
mostly new clients (iOS, Android Email, WinPhone, Apple Mail and I think
Thunderbird now all report with ID).

Obviously our list of clients is skewed compared to what most servers
probably see, but most of our usage is mobile clients, followed by the most
popular desktop clients (in order), Apple Mail, Thunderbird, Outlook,
Outlook Express... then much further down we have Pine, Pegasus Mail, and
Opera.

But agreed, with the top 4 mobile (iOS, Android Email, WinPhone and
Blackberry/BIS (not sure if they speak IMAP to anyone but Gmail, though))
plus Apple Mail, Thunderbird, Outlook and Outlook Express, you're probably
talking about the top 99% of all IMAP clients.

I'm happy to share the FETCH matches we do for about 25 clients.  Its not
exact, and probably unnecessary since most of those are just in the noise.

Brandon

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 3 Aug 2012 01:04:13 +0300

On 3.8.2012, at 0.56, Brandon Long wrote:

> Obviously our list of clients is skewed compared to what most servers probably see, but most of our usage is mobile clients, followed by the most popular desktop clients (in order), Apple Mail, Thunderbird, Outlook, Outlook Express... then much further down we have Pine, Pegasus Mail, and Opera.

I'd be interested in percentages also if those aren't a secret. :)

> But agreed, with the top 4 mobile (iOS, Android Email, WinPhone and Blackberry/BIS (not sure if they speak IMAP to anyone but Gmail, though)) plus Apple Mail, Thunderbird, Outlook and Outlook Express, you're probably talking about the top 99% of all IMAP clients.

BIS talks IMAP to everyone.

> I'm happy to share the FETCH matches we do for about 25 clients.  Its not exact, and probably unnecessary since most of those are just in the noise.

I'm interested of those. Maybe at some point I'll make a Dovecot plugin doing similar statistics. There are probably also other things you can use to detect clients.

---

From: Brandon Long <[email-masked]>
Date: Thu, 2 Aug 2012 15:48:19 -0700

Ok, attached is the list of unique-ish FETCH commands that we use to match
on.  They probably don't match what the client sends exactly (our logging
is based on our parsing of the fetch, not on the direct client text, and
then we further munge to generize the line some more, hence <chunk.chunk>
and <0.chunk>).

Its also focused more on the state of clients ~3-4 years ago, with the
uptick in usages of the ID command and the overwhelming numbers of a small
set of clients, we haven't really kept it up to date.

> I'd be interested in percentages also if those aren't a secret. :)

I don't know if they're secret.  I can certainly imagine some ways that
bloggers could spin the data in various ways, though, and I wouldn't like
to put PR through the fun of handling that.

The vast majority is mobile, and the majority of that is iOS.  We also see
more than one device per user, on average (ie, I good portion of iPad users
also have another iDevice).  Note that our Android numbers are higher than
expected because the Gmail app doesn't speak IMAP, it speaks its own
protocol.  And our Outlook numbers may be low because Google Apps Sync
speaks its own protocol.  But Apple Mail is about the same as both Outlook
and Outlook Express combined, and Thunderbird comes in the middle of that.

> BIS talks IMAP to everyone.

Ok, I knew they used to use POP with us, and upgraded some of their users
to IMAP, at least when they use the Gmail Blackberry extension, wasn't sure
if that was general or not.

> > I'm happy to share the FETCH matches we do for about 25 clients.  Its
> not exact, and probably unnecessary since most of those are just in the
> noise.
>
> I'm interested of those. Maybe at some point I'll make a Dovecot plugin
> doing similar statistics. There are probably also other things you can use
> to detect clients.
>

Definitely.  My solution was based on our logs analysis, which basically
means individual commands with no sequence, and we didn't originally bother
to store the tag.

If you look at the way the tag is constructed, whether the tag is numeric
and in order, whether it has sub sections (1.2) or all lower case letters,
etc and the sequence of commands, you can probably do an even better job in
real-time, especially if you want to tell the difference between versions
of clients.  Certain bugs also stand out, like Outlook which likes to do an
IDLE without having a folder selected.

Brandon

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 10 Aug 2012 15:17:01 +0100

On 02/08/2012 15:48, Brandon Long wrote:
> Certain bugs also stand out, like Outlook which likes to do an IDLE 
> without having a folder selected. 

As a side note: as much as I like complaining about Outlook and its use 
of IMAP, IMAP IDLE command is actually allowed when no folder is 
selected. The behavior is undocumented (most servers wouldn't send any 
updates in such case), but the command is still valid in such state.

---

From: Barry Leiba <[email-masked]>
Date: Fri, 10 Aug 2012 14:24:08 -0400

> As a side note: as much as I like complaining about Outlook and its use
of IMAP,
> IMAP IDLE command is actually allowed when no folder is selected. The
behavior
> is undocumented (most servers wouldn't send any updates in such case),
but the
> command is still valid in such state.

We left that open because we had talked about the possibility of
unsolicited LIST responses to indicate new mailboxes, and other things like
that.  In practice today, alerts are the only things likely to be sent in
AUTHENTICATED state.

Barry