Subject: Re: [OSPF] Re: [AOSPF] Anycast Extension to OSPFv3 (draft-wang-aospf-00) - Authors' Technical Considerations

From: Acee Lindem <[email-masked]>
Date: Wed, 3 Jan 2007 11:08:55 -0500

Hi Naiming, et al,

On Dec 29, 2006, at 12:32 AM, Naiming Shen wrote:

> Curtis,
>
> Curtis Villamizar said the following on 12/28/2006 05:58 PM:
>> In message <[email-masked]>
>> Naiming Shen writes:
>>> Curtis Villamizar said the following on 12/28/2006 03:03 PM:
>>>
>>>> In message <[email-masked]>
>>>> Acee Lindem writes:
>>>>
>>>>
>>>>> I don't think the problems above warrant allocating an OSPFv3  
>>>>> prefix
>>>>> option bit since most of them should be solved outside of  
>>>>> OSPFv3.  For
>>>>> example, I'd hope modern routers wouldn't default to per-packet  
>>>>> ECMP
>>>>> forwarding.
>>>>
>>>>
>>>>
>>>> I don't think any router in use in a provider has done per- 
>>>> packet ECMP
>>>> in a long time but there is no telling what an off brand enterprise
>>>> sized router will do.  I recently encountered a DSL router that  
>>>> had to
>>>
>>> But this is not a problem with OSPF specifically. For dynamic  
>>> routing
>>> information on prefixes(including anycast route), one can otionally
>>> add a sub-tlv/attribute/option-bit/opaque-option to signal a
>>> "Do-Not-Per-Packet-ECMP" status of the route for BGP/ISIS/OSPF/etc,
>>> and it's up to the implememtation to automatically disable the
>>> per-packet loadsharing even it's explicitly configured on outbound
>>> interfaces. Anycast route maybe only one of the applications
>>> which can benefit with this trick.
>>> - Naiming
>> Naiming,
>> Just about everything benefits from this trick.  If you do per packet
>> ECMP and the two legs have unequal delay you get out of order
>> delivery.  Real time applications are not at all fond of getting out
>> of order delivery.  TCP can have false triggers of the "fast restart"
>> mechanism which dramatically slow down TCP transfers.  Per-packet  
>> ECMP
>> has been proven in the field to be a very bad idea and the reasons it
>> causes problems are very well understood.
>> Per-packet ECMP should not be done under any circumstances.  Hash
>> based load split is circa 1986/1987 technology and works.  Use it.
>> See RFC2991 and RFC2992 for more detailed explanation.
>> It is unfortunate that routers still allow per-packet ECMP to be
>> configured.  At least providers know not to use this feature.  But we
>> can all be sure that somewhere out there someone ignorant of the
>> problems has enabled it.
>
> I completely agree. Per-packet ECMP should not be used normally,
> especially in provider's networks. But I did experience some cases
> ( a while ago ) on international links, there were some dominant
> prefixes which led to very uneven link bandwidth usages due to flow
> based algorithm. I think if folks in the routing area still think
> things like anycast having this issue, then to signal this  
> attribute is
> useful and can be easily done; otherwise, there is absolutely no issue
> in terms of anycast prefiexes(ipv4 or ipv6).
>
>> Curtis
>> ps - Maybe the attribute semantics should be an "OK to per packet  
>> load
>> split traffic to this route" and all we have to do is never use  
>> it.  :-)
>
> Well, I don't think any one wants to set this flag in the first  
> place;-)

So, with the limited OSPFv3 prefix bits I don't see enough motivation to
allocate one for the purpose of identifying anycast addresses. What  
we've
discussed is a hypothetical problem with anycast and ECMP and not one
anyone is complaining about.

Thanks,
Acee

>
> - Naiming
>
>>>> be reboot to make any configuration change effective so bad design
>>>> even today is clearly possible.
>>>>
>>>> Curtis

_______________________________________________
OSPF mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ospf

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 03 Jan 2007 12:28:14 -0500

> anyone is complaining about.

Naiming and I thought we were far enough off topic that we went off
list and sent a few more messages without copying the WG.

IMO -- There doesn't seem to be any legitimate use for per-packet load
split in a provider network.  There may be some benefit to per-packet
load split in an enterprise network where a very small number (perhaps
one) of the src/dst flows dominates traffic, then per-packet might be
useful if it were not for better ways of bonding two or more parallel
links (from PPP-multilink to proprietary stuff bonding multiple 10G
links or larger).  Using OSPF ECMP with a per-packet split on a pair
of paths with unequal delay is harmful to very many applications and
should never be done, in a provider or in an enterprise.  All of the
better techniques preserve packet ordering within a microflow with
some techniques striping or resequencing if necessary.

Early Cisco implementations (early 1990s) only did per-packet or
per-prefix split because src/dst hash was hard to do in software on
the early (very underpowered M680x0 and software forwarding) Cisco
routers.  Since mid-1990s this has not been an issue.  Although the
option to do per-packet split remains in Cisco routers hopefully no
one is uninformed enough to enable it.  RFC2991 and RFC2992 can help
make them better informed but the problems with reordering were only
mentioned (incompletely, for example the repeated reduction by half in
the TCP window can be a bigger problem than the wasted retransmit
bandwidth on fast-retransmit) with the focus in those RFCs being on
better ways to load split.

I don't know if Naiming reached the same conclusion.

I think we would both agree that there is no need to change OSPF for
this (and Naiming can let us know if he doesn't agree on that point).

Curtis

---

From: Naiming Shen <[email-masked]>
Date: Wed, 03 Jan 2007 11:00:06 -0800

Acee, Curtis,

I agree there is no need for OSPF extension for anycast.

thanks.
- Naiming