Subject: Re: Discontinuing the mhonarc archives January 14

From: Nick Hilliard <[email-masked]>
Date: Fri, 8 Mar 2019 14:00:45 +0000

Robert,

a search for "site:mailarchive.ietf.org" gives 5 results on both Google 
search, Bing and duckduckgo.  I.e. the entire IETF mail archive is 
unsearchable via the most commonly used public search engines.  Is there 
any way of resolving this?

Nick
> Robert Sparks <mailto:[email-masked]>
> 10 January 2019 at 15:29
> Thanks for the nudge, Nick. Some conversations started before the end 
> of the year had not run to conclusion.
>
> The robots.txt file has been changed, and some things put in place to 
> help the spiders find all the messages.
>
> RjS
>
>
> Nick Hilliard <mailto:[email-masked]>
> 8 January 2019 at 19:59
>
>
> Robert,
>
> https://mailarchive.ietf.org/robots.txt includes "Disallow: /", which 
> means that the entire mailarch repository is unavailable on regular 
> search engines.
>
> This was reported in ietfdb ticket 2658.
>
> Are there plans to fix this before mhonarc is discontinued?  It would 
> be hugely regressive for ietf mailing lists to be cut off from regular 
> internet searches.
>
> Nick
>
> Robert Sparks <mailto:[email-masked]>
> 8 January 2019 at 19:49
> We will be discontinuing Mhonarc during the day on Monday Jan 14.
>
> As noted below, all existing Mhonarc URLs will redirect to the same 
> message in mailarch.
>
> The mbox files available through rsync will continue to be available.
>
>
>
> ------------------------------------------------------------------------

---

From: S Moonesamy <[email-masked]>
Date: Sat, 09 Mar 2019 09:46:13 -0800

Hi Robert,
At 06:00 AM 3/8/2019, Nick Hilliard wrote:
>a search for "site:mailarchive.ietf.org" gives 5 results on both 
>Google search, Bing and duckduckgo.  I.e. the entire IETF mail 
>archive is unsearchable via the most commonly used public search 
>engines.  Is there any way of resolving this?

A side-effect of discontinuing the mhonarc archives is that the past 
discussions got "lost" as it is not possible to find them through a 
search engine.  Could that be fixed?

Regards,
S. Moonesamy

---

From: Robert Sparks <[email-masked]>
Date: Mon, 11 Mar 2019 08:57:05 -0500

It is the intent for the mailarchive archives to be searchable by these 
public search engines. The settings are being adjusted, and we're 
starting to see some indexing.

While those indexes populate, you can always search using the 
mailarchive's built-in search capability (I disagree with the 
characterization that past discussions got "lost").

RjS

On 3/9/19 11:46 AM, S Moonesamy wrote:
>> Google search, Bing and duckduckgo. I.e. the entire IETF mail archive 
>> is unsearchable via the most commonly used public search engines.  Is 
>> there any way of resolving this?
>

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 11 Mar 2019 15:14:37 +0100

On Mar 11, 2019, at 14:57, Robert Sparks <[email-masked]> wrote:
> 
> I disagree with the characterization that past discussions got "lost"

One of the meanings of “lost” is “too hard to find”.

“Not turning up immediately in a search engine search” is a synonym for “too hard to find”, no?  (People looking for the answer may not know that IETF has it.)

Good to know that the situation is clearing.

Grüße, Carsten

---

From: S Moonesamy <[email-masked]>
Date: Tue, 12 Mar 2019 13:44:00 -0700

Hi Robert,
At 06:57 AM 3/11/2019, Robert Sparks wrote:
>It is the intent for the mailarchive archives to be searchable by 
>these public search engines. The settings are being adjusted, and 
>we're starting to see some indexing.

Thank you for the update.

Regards,
S. Moonesamy