Subject: Re: Response to the Appeal by JFC Morfin dated 2006-02-17 - 2006-05-17.

From: Jefsey Morfin <[email-masked]>
Date: Tue, 11 Jul 2006 00:12:20 +0200

Two appeals have been responded today. A first reading shows an 
impressive distance between the response and the matter of the 
appeal. By respect for the IETF and duty to the IETF users I 
represent, I will however appeal to the IAB.

The real issue is the interoperability between the Multilingual 
Internet as the world wants it, and the Internationalized Internet as 
a small number of locally dominant stakeholders wants to impose it. 
This is why it is important to obtain clear texts permitting clear 
interconnects, or clear circumventions. I painstakingly obtained such 
texts with RFC 3066 Bis together with the serious IESG response to my 
appeal. The second BCP 47 Draft has been approved. As for the first 
part, it will be carefully appealed to obtain a complete review of 
the BCP 47, in the hope the IESG's answer will be of the same quality 
as for the first part. If necessary and appeal to the IAB will be 
carried, in the hope to force the IESG to respect our RFC 3066 Bis consensus.

The interest of the today answers is to confirm that the current 
violation of the RFC 3066 Bis is a deliberate IESG policy, purposely 
reinstalling the confusion RFC 3066 Bis prevents. This is why it is 
important to carry the full appeal procedure, so no one can doubt 
that every clarification effort has been undertaken and everyone can 
benefit from it.

Then, users from all over the world and the international community 
will decide how to consider the influence (cf. RFC 3935) of so few 
using their accidental dominant position to curb the culture, the 
languages, the expectations, the economy of so many, instead of serving them.

jfc

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Dean Anderson <[email-masked]>
Date: Tue, 11 Jul 2006 04:39:52 -0400 (EDT)

On Tue, 11 Jul 2006, Jefsey Morfin wrote:

> impressive distance between the response and the matter of the appeal.

There were only two outstanding appeals, the other one being from me. I
can't see the response to JFC on the IESG site, but JFC has certainly
understated the response I did see. "Impressive distance", indeed.

While some parts of my June Appeal required some knowledge of DNS and
Anycast to understand, the impropriety of the IESG response can be
understood by just about anyone:  The third issue they addressed was the
failure of IESG members accused of wrongdoing to recuse themselves. The
IESG held they do not have to recuse themselves. But it is
well-established notion of professional ethics that one cannot sit in
judgement of one's own actions. That is a blatant conflict of interest.  
Just about anyone can understand that.

But apparently the IESG--as a whole--cannot understand this fundamental
principle of ethics that requires recusal for conflict of interest.  
Even though the IESG as a whole was reminded of this through repeated
requests for the officials (Kessens and Carpenter) to recuse themselves
and halt repeated instances of improper interference. This ethical
dysfunction discredits the IESG and its members as a competent body to
make fair and ethical judgements.  And this dysfunction discredits the
IESG and members as an activity of the ISOC subject to the ISOC bylaws
which require high standards of ethical behavior.  The ethics of recusal
are pretty straightforward in this instance.  The issues are now more
serious: Rather than merely two offending officials, there is now (at
least) a majority of the IESG involved.  This is now a very serious
breach of public integrity.

The other elements of the IESG decision are likewise not credible.  
This decision will also be appealed to the IAB.  Other organizations
will also have to be notified.  I anticipate that honest, ethical
members of the IESG will shortly resign or have already resigned to
protect their reputations. I will give a few days for these notices to
circulate.  The remaining members will be named as offending
participants in further complaints about this impropriety.

Dean Anderson
Av8 Internet, Inc

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: Dean Anderson <[email-masked]>
Date: Tue, 11 Jul 2006 05:32:51 -0400 (EDT)

BTW, the IESG response implied that the allegations of scientific fraud
were (somehow) not substantiated. Following is a summary of the
allegations, and after that is the DNSMON source code for sending
packets. The source code plainly constructs a UDP (and only UDP) packet.

-------

What the full community may not know, [but ISC, RIPE, Joe Abley, David
Kessens, Brian Carpenter, and the IESG do know], is that the report
claiming that stateful anycast was stable was fabricated, and that no
stateful testing was performed by the DNSMON program.  Contrary to
assurances given by Karrenberg, there is no data which supports the
notion that stateful DNS Anycast is safe, nor any data that disputes
data and assertions that show DNS Anycast is unsafe.

The notion of a safe stateful anycast operation as asserted by Daniel
Karrenberg (http://www.nanog.org/mtg-0505/pdf/karrenberg.pdf) has now
been discredited. Karrenberg's document misled people to believe that
stateful anycast was safe, when in fact Karrenberg didn't perform any
stateful testing whatsoever.

The controversy began in 2002 with announcements by RIPE NCC and ISC 
(operators of K-root and F-root, respectively) to sell Anycast instances 
of K and F servers.  

In October, 2004, Mark Kosters reports on data gathered at J root:

http://www.nanog.org/mtg-0410/pdf/kosters.pdf

    + Expected to see a saw tooth distribution .
      instead have a noisy distribution in many cases
    + Does not affect UDP
    + DO NOT RUN Anycast with Stateful Transport

http://www.rssac.org/meetings/04-08/2004WashDC.html
  At the November 7th, 2004 RSSAC meeting, Kosters repeats warning on
stateful DNS Anycast, but is disputed by Karrenberg. In May 2005, after
one day after suppressing discussion on Nanog by Anderson, Karrenberg
unqualified assurance that Anycast is safe, and please "do not to spread
this false rumor" http://www.nanog.org/mtg-0505/pdf/karrenberg.pdf

It is later found (January, 2006)  [reported to IESG March, 2006] that
Karrenberg has done no stateful testing whatsoever, and that Karrenberg
did not reveal that his testing was only for stateless DNS, and
therefore had no relevance to the safety of stateful DNS Anycast.  [To
put this in context of current news, this is the difference between
creating a stem cell line, and creating a new stem cell line.  It was a
scientific fraud to describe the former as the latter. A one-word
difference]

This discovery was only made when Anderson examined the source code to
the DNSMON program written by Karrenberg to conduct this testing.

Now put this in context along with repeated assertions from Joe Abley
and others associated with ISC and RIPE that stateful anycast is safe
and even non-controversial.  More history is found at 
http://www.av8.net/IETF-watch/DNSRootAnycast/History.html

----------  

copyright from main.c:

/* 
 * Copyright (c) 2003, 2004        RIPE Network Coordination Center
 *                                 Mark Santcroos <[email-masked]>
 *
 * All Rights Reserved
 *
 * Permission to use, copy, modify, and distribute this software and its
 * documentation for any purpose and without fee is hereby granted,
 * provided that the above copyright notice appear in all copies and that
 * both that copyright notice and this permission notice appear in
 * supporting documentation, and that the name of the author not be
 * used in advertising or publicity pertaining to distribution of the
 * software without specific, written prior permission.
 *
 * THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
 * ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS; IN NO EVENT SHALL
 * AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY
 * DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
 * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 *
 * $Id: main.c,v 1.12 2005/05/23 12:35:43 kevin Exp $
 *
 */

sendpacket function from sending_process.c:

/*
 *  Sendpacket
 *  takes a buffer and send it out to socket
 */
struct timeval
sendpacket(char *packet, struct server list[], int entry)
{
    int ret;
    int size;
    struct ip *iphdr;
    struct udphdr *udphdr;
    char *sndbuf = (char *)malloc(512);
    int udplen;
    int iplen;
    int datalen;
    struct sockaddr_in saddr;
    struct timeval send_time;

    datalen = list[entry].size;

    iplen = sizeof(struct ip);
    udplen = sizeof(struct udphdr);

    iphdr = (struct ip *) sndbuf;
    udphdr = (struct udphdr *) (sndbuf + sizeof(struct ip));

    /* copy to the actual send buffer */
    memcpy(sndbuf+iplen+udplen, packet, datalen);

    iphdr->ip_v = IPVERSION;
        iphdr->ip_hl=sizeof(struct ip) >> 2;
    iphdr->ip_id = 0;  /* 0 means kernel set appropriate value */
    iphdr->ip_len = sizeof(struct ip) + sizeof(struct udphdr) + datalen;
    iphdr->ip_tos = 0;
        iphdr->ip_off = 0;
        iphdr->ip_p = IPPROTO_UDP;
    iphdr->ip_ttl = 0x40;

    /* set aquired first ip address */
        iphdr->ip_src.s_addr = htonl(src_ip);
        iphdr->ip_dst.s_addr = inet_addr(list[entry].address);

    size = iplen + udplen + datalen;

    udphdr = (struct udphdr *) (sndbuf + sizeof(struct ip));

    udphdr->uh_sport = htons(UDP_SRC_PORT);
    udphdr->uh_dport = htons(53);
    udphdr->uh_ulen = htons(sizeof(struct udphdr) + datalen);
    udphdr->uh_sum = do_checksum(sndbuf, IPPROTO_UDP, 
        sizeof(struct udphdr) + datalen);

    /* this will be the time the measurements get logged */
    gettimeofday(&send_time, NULL);

    ret = sendto(raw_fd, sndbuf, size, 0, 
        (struct sockaddr *)&saddr, sizeof(saddr));
    if (ret < 0) {
        perror("sendto");
        exit(1);
    }
#ifdef SENDING_DEBUG
    printf("%d bytes send.\n", ret);
#endif /* SENDING_DEBUG */

    free(sndbuf);

    return (send_time);
}

---

From: Joe Abley <[email-masked]>
Date: Tue, 11 Jul 2006 14:18:48 -0400

On 11-Jul-2006, at 05:32, Dean Anderson wrote:

> were (somehow) not substantiated.

I haven't seen these specific complaints voiced with this clarity  
before (maybe I overlooked some mail). Perhaps this is a good  
opportunity to dispense some additional perspective.

> [...]

I don't believe the fact that DNSMON sends all its probe queries  
using UDP transport is news to anybody. It's certainly not a secret,  
as you have aptly illustrated by looking at the source code, which is  
freely available.

I was in the meeting in Seattle where Daniel presented his analysis  
of DNSMON and RIS data in an attempt to draw conclusions about the  
stability of various nameservers which had been distributed using  
anycast.

The approach Daniel took (which was analogous to earlier work  
presented by Verisign and also the work done by Peter Booth at the  
University of Oregon) was to look at measurements which had already  
been made by the NCC's DNSMON project, and try to identify whether  
individual DNSMON probes saw oscillations in node selection over time  
and if so, with what frequency.

It is possible to identify oscillations in node selection from  
individual probes without using TCP transport. (In fact, it seems to  
me that it's easier to acquire unambiguous results using UDP  
transport, since if there *are* node oscillations which would damage  
TCP, measurements using TCP would simply indicate failure without  
revealing the nature of the oscillation.)

However, from what I could tell from Daniel's presentation, the fact  
that UDP transport was used by DNSMON was a simple result of the fact  
that UDP measurement data is what was already stored, and hence that  
was the data that was available for analysis.

I can find no example of Daniel (or anybody else) claiming that  
DNSMON in general, or the data which formed the basis of Daniel's  
NANOG presentation in particular, resulted from DNS queries made  
using TCP transport. The only person suggesting otherwise is you.

Surely this whole issue is a red herring.

> http://www.av8.net/IETF-watch/DNSRootAnycast/History.html

I fully support continued measurement of services which have been  
distributed using anycast.

I make no claims that anycast is definitively safe for protocols and  
services which don't involve trivial, stateless transactions. The  
document draft-ietf-grow-anycast-04 goes to great lengths to describe  
considerations in protocol/transaction and network characteristics  
which should be well understood before anycast is chosen as a service  
distribution mechanism.

Kurtis and my slides in the open ops area meeting this afternoon will  
repeat the message that unicast is not a universally applicable  
strategy.

However, I also don't presume to say that (for example) protocols  
based on TCP are always unsafe for deployment using anycast in all  
possible networks. For example, there are people using anycast to  
distribute services using very long-held sessions (e.g. internet  
radio, HTTP) with great success, and to ignore their experience and  
success would be idiotic and arbitrary.

Joe

---

From: Dean Anderson <[email-masked]>
Date: Wed, 12 Jul 2006 15:45:57 -0400 (EDT)

> opportunity to dispense some additional perspective.

Abley's claims are again not true.  Abley's claim of not having seen the
"specific complaints with this clarity", is entirely false.  While the
information I just presented may be somewhat new to the main IETF list,
there was nothing new that hasn't been discussed on DNSOP and elsewhere,
with Mr. Abley, and the IESG. Abley and the IESG are well aware of the
precise details, in greater detail and clarity than the summary I just
presented. Abley is very well aware of these issues, in even greater
detail. 

Indeed, most of my recent message was copied from a message I sent to
DNSOP, which Abley certainly saw.

Abley has almost certainly also seen the DNSMON source code. But few
others have seen the DNSMON source code. (more below)

> as you have aptly illustrated by looking at the source code, which is  

The code is copyrighted free, but is not freely available. RIPE does not
distribute the DNSMON code to the general public. The source code I
posted is not accessible by the general public.  When RIPE gave it to
me, they asked me not to redistribute it---but they indicated an
intention to give away the code, and my limited quote is fair use.

> individual probes without using TCP transport.

The oscillation issue is orthogonal and unrelated to the anycast issue.  
While both Verisign may have been originally looking to find oscillation
issues when the data was collected, the implications of the findings to
stateful anycast was an unexpected discovery. The implications of this
discovery were made clear by Verisign.

> was the data that was available for analysis.

This is false or complete nonsense. There is no prior data "already
stored".  The data was collected by DNSMON.  A tool like DNSMON could
collect stateful data. But DNSMON didn't do that.

Karrenberg didn't report that he only measured stateless UDP data.  
Karrenberg presented his results to refute specific claims of stateful
(TCP) problems raised by Anderson and Verisign.  If Karrenberg had
reported the truth at the time, no one would have treated his data as
having any relevance to stateful anycast.  

An honest person in Karrenberg's position would have reported that his 
data had no relevance to the TCP question. But Karrenberg didn't do what 
an honest researcher would have done. 

> using TCP transport. The only person suggesting otherwise is you.

This is also not true. While Karrenberg didn't explicitly say that he
made TCP queries (indeed Karrenberg didn't say anything at all about
this very important detail--this is a dishonest omission), Karrenberg
__did__ say was that his results refuted the "false rumor" that anycast
was somehow unsafe--this is a dishonest statement.  An honest statement
would have said that his data had no bearing on the question of whether
stateful anycast was safe.  

> Surely this whole issue is a red herring.

We have heard nothing but false assurances for several years.  "Surely"  
is definitely not "certainly".

> distributed using anycast.

With spoofed NSID (draft now before the IESG in Last Call), no such
measurement is possible.  Or is at least made very, very difficult.  It
is plain that you do not actually support this, anymore than Clinton
supported the investigation into the Lewinsky affair.

> services which don't involve trivial, stateless transactions. 

Really?  That's a big change.

> possible networks. 

This is a very fine point. People following this discussion should 
really pay attention here.  Read carefully what Abley has said.

I do make the claim that stateful Anycast is unsafe in networks which
are allowed under RFC1812. Specifically, networks which implement fast
load switching--Such networks currently exist in practice, and their
existence is confirmed by Verisign's data.  Anycast is absolutely unsafe
for such networks.  [Nanog participants originally claimed that no
existing hardware implemented such load switching, and that widely used
Cisco GSR routers were architectually incapable of implementing fast
load switching. This claim also turned out to be entirely false.]

I do make the claim that Root DNS nameservers have to (are obligated to)
provide service to these networks.  Therefore, Root DNS operators cannot
use Anycast.  The use of stateful anycast by root DNS operators presents
a serious stability issue to the global network. This is made plain by
Verisign's data,which showed that nearly 4 percent of the IP addresses
show up at more than 2 anycast instances in less than 2 minutes.

This is important because that implies that approximately 4 percent of
the root DNS queries would fail if they were stateful.  Such a failure
rate is abysmally high: Internet connections that drop 3 percent of
their packets are generally unusable.  If 3 or 4 percent of Root DNS
queries failed, we would see an significant interruption in global
service.  Root DNS is a critical service: When root DNS fails, nothing
works.

> For example, there are people using anycast to distribute services
> using very long-held sessions (e.g. internet radio, HTTP) with great
> success, and to ignore their experience and success would be idiotic
> and arbitrary.

There are no such http services using stateful anycast (though this is
the first I've heard of internet radio being anycast---I'd have to
question what protocol he means. I think RTP streams may be stateless,
but I'm not certain).  The http protocol is definitely stateful,
however. 

The http claim has been made and debunked previously.  The http claim
turned out to be another myth originated by Randy Bush and Paul Vixie as
far as I can tell.  Vixie "warned" Akamai not to do this--Patrick
Gilmore of Akamai eventually responded that Akamai never did this. Bush
asserted vaguely that someone had did that, once, somewhere, sometime,
but without any specifics of who, where, or when.  No specifics were
found other than a vague reference to a one time "speed test" conducted
in the early 1990's that turned out to be rigged with a scheme similar
to anycast.

Dean Anderson 
Av8 Internet, Inc

---

From: Joe Abley <[email-masked]>
Date: Wed, 12 Jul 2006 16:40:20 -0400

> Really?  That's a big change.

Really? There has been text to that effect in the draft since -00.

> but I'm not certain).  The http protocol is definitely stateful,

http://www.computerworld.co.nz/news.nsf/NL/ 
54FCC3B9D0B8BB96CC25702F00740446
http://www.nanog.org/mtg-0606/levine.html

etc, etc. I think I've said enough in this thread, now; if there are  
innocent bystanders who would like more clarification of these issues  
they are more than welcome to drop me a note directly.

Joe

---

From: Dean Anderson <[email-masked]>
Date: Wed, 12 Jul 2006 20:14:32 -0400 (EDT)

> Really? There has been text to that effect in the draft since -00.

Searching draft 4 just now, I can find no instances of the words "safe",
"trivial", or "stateless" in the draft.  So, I suppose, quite literally,
it is true that you didn't make such a claim in the __draft__.  But you
and your group did make these assurances elsewhere.  This is sort of
legalistically dishonest: Rather like an unscrupulous used car dealer
claiming he never promised the car would start in the sales contract,
even though he did say so on the lot.

I did notice a number of "bait and switch" between draft 02 and 03
(which I noted to you and the IESG).  In those cases, you quietly
weakened the statements made.  For example:

===============================================================
Date: Thu, 29 Jun 2006 13:50:42 -0400 (EDT)
From: Dean Anderson <[email-masked]>
To: Joe Abley <[email-masked]>
Cc: Sam Hartman <[email-masked]>, [email-masked]
Subject: Re: grow: draft-ietf-grow-anycast-pre04

[...]

On the issue of "minor changes":

This is a major change:

-   discrete locations.  The service provided by each node is consistent
-   regardless of the particular node chosen by the routing system to
-   handle a particular request.
+   discrete locations.  The service provided by each node is generally
+   consistent regardless of the particular node chosen by the routing
+   system to handle a particular request (although some services may
+   benefit from deliberate differences in the behaviours of individual
+   nodes, in order to facilitate locality-specific behaviour; see
+   Section 4.6).

The change from "is consistent" to "is generally consistent" is a
significant difference. To be a safe stateful operation, "is consistent"
is required behavior.  You assured people that it is safe, and then you
"bait and switched" the text to weaken the official statements in spite
of your stronger assurances. This is a nefarious change.

[...]
===============================================================

> >> using very long-held sessions (e.g. internet radio, HTTP) with
> >> great success, and to ignore their experience and success would be
> >> idiotic and arbitrary.
> >
> > There are no such http services using stateful anycast (though this
> > is the first I've heard of internet radio being anycast---I'd have
> > to question what protocol he means. I think RTP streams may be
> > stateless, but I'm not certain).  The http protocol is definitely
> > stateful, however.
> 
> http://www.computerworld.co.nz/news.nsf/NL/54FCC3B9D0B8BB96CC25702F00740446

This link is a June 30, 2005 announcement from Radio New Zealand.  It
makes no mention of the protocol they plan to use. It does make mention
of anycast root DNS servers, as though that adds credibility to their
scheme. There is no data that this scheme is stateful nor, if it is
actually stateful, that it is safe.  Nor this announcement mean they are
having any success with this scheme over a period of time.  I don't know
if they are still using this scheme, nor if they even implemented the
intentions announced June 30, 2005.  This is a rather weak reference.

> http://www.nanog.org/mtg-0606/levine.html

First, check the date: June, 2006. Not very long time to check their
data, and __after__ you efforts began.  You've been promoting this
scheme since 2002, saying it was stable then, with years of commercial
use. 

Second, you seem to miss a lot: (page 5)
  "State is accomplished with custom hardware."

As much as they bluster about doing "TCP anycast", it is not actually
TCP Anycast.  They have some special hardware doing state behind the
scenes.  They don't give many details, though.  But that is different
from what we are discussing. You've proposed no custom hardware.  You
and they are essentially doing something that akin to confusing the
Mount Everest ride at Disneyland to climbing Mount Everest, because they
gave you a "I conquered Mount Everest" sticker.

Third, they are doing Porn video streaming, so customers probably don't
complain too much about frequent failures.

Fourth, it is my understanding that Real Media and MS Media player
streams are RTP, not TCP.  They are not real clear on those points. 

Fifth, their performance data indicated that over a long period of time,
they never had 100% availability. This is from only 31 locations.  
Their data shows that some connections always failed. They never reached
100% availability. That would be "unsafe".

Sixth, Cache networks was established in 2002:

"CacheNetworks, established in Berkeley, CA in 2002, provides fast,
affordable content delivery through a superior distribution network with
points of presence located throughout the United States. When a visitor
attempts to access content, they are sent to the location that will
yield the quickest access times. CacheNetworks allows companies to
rapidly expand and adapt their websites, seamlessly and without
additional infrastructure, all backed up by 100% availability guaranteed
SLA (Service Level Agreement)."

Established in 2002:  No very long term use of stateful anycast there.  
Certainly not the 9 or 10 years asserted by Vixie and Bush.  I also
notice that the data they posted to Nanog indicates that they never,
ever, achieve 100% availability, in spite of their SLA guarantee 
mentioned in the press release. 

> they are more than welcome to drop me a note directly.

You haven't given __any__ examples of http use, or anything definitely
stateful, even.  Or anything that even predates your beginning this
scheme in 2002.

The examples you did give didn't really support your point. They appear
to be people who have "drunk your koolaid";  they are taking direction
from your group, and hoping it will turn out OK. But of course, they
aren't looking for particularly good results. And where they published
numbers, they don't seem to be getting good results.

		--Dean