Subject: Proposed new 5.4.3 for arch draft

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2004 00:46:42 +0000

Here is the proposed change to 5.4.3 that was discussed
at the WG meeting. Also here is the corresponding change
to section 9.

The arch draft will become informational, and
I will post a pre-draft of the full document with all
the 07 changes incorporated some time next week.

Please consider this the start of a two week last
call on this change.

I will put together a draft for the CW text in the
next few days.

- Stewart
---------------------------------------------------

5.4.3. PW Packets Aliasing IP Packets

To allow accurate packet inspection in an MPLS PSN, and/or to operate
correctly over MPLS PSNs that have deployed equal-cost multiple-path
load-balancing (ECMP), a PW packet must not alias an IP packet.
IP packets are carried in MPLS label stacks without any protocol
identifier. The method of packet discrimination must therefore
be based on some unique characteristic of the IP packet header,
which PWE3 must not mimic.

This is the subject of work in progress.

-----------------------------------------------------

9.  IANA considerations

    IANA considerations will be identified in the PWE3 documents that
    define the PWE3 encapsulation, control and management protocols.

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 3 Mar 2004 03:52:24 +0200

Stewart and all,
Basically fine, please see a couple of minor comments inline (marked
[Sasha]).

Regards,
                 Sasha. 
... snipped ...

To allow accurate packet inspection in an MPLS PSN, and/or to operate
correctly over MPLS PSNs that have deployed equal-cost multiple-path
load-balancing (ECMP), a PW packet must not alias an IP packet.
[Sasha] IMO it should be MUST NOT or, probably, SHOULD NOT.
IP packets are carried in MPLS label stacks without any protocol
identifier. The method of packet discrimination must therefore
be based on some unique characteristic of the IP packet header,
which PWE3 must not mimic.
[Sasha] I honestly believe no one will ever propose verifying the cheksum
[Sasha] of the first 20 bytes after the bottom label as "the unique
[Sasha] characteristic of the IP packet header". But worse things have
[Sasha] happened, so maybe we should explicitly block this.
This is the subject of work in progress.

9.  IANA considerations

_______________________________________________
pwe3 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pwe3

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 3 Mar 2004 04:27:49 +0200

Alia and all,
I agree that checksum is a valid method of verification (even if
occasionally it could fail). My understanding has been, however,
that we do not want to use it, and tried to block a possible
loophole.
Regards,
                      Sasha

-----Original Message-----
From: Alia Atlas
To: Sasha Vainshtein
Cc: 'Stewart Bryant '; [email-masked] '
Sent: 3/3/2004 4:09 AM
Subject: RE: [PWE3] Proposed new 5.4.3 for arch draft

>[Sasha] happened, so maybe we should explicitly block this.

The checksum is legitimate to consider as a verification, if one doesn't

want to depend on just the first nibble.

Alia

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2004 04:33:59 +0000

Thomas D. Nadeau wrote:

> 	Stewart,  you are aware that IP
> employs ECMP as well, yes?  

Of course.

> Seems more
> generic to just mention ECMP instead of
> MPLS, or mention MPLS as an example.

But this is an MPLS specific problem we are addressing.

ECMP in an IP network works just fine (provided you
ECMP per flow and not per packet).

BTW ECMP is just one example case of when you need to
discriminate between an IP packet and a non-IP packet
MPLS payload. IPFIX is another case.

- Stewart

> 	--Tom
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Stewart Bryant
> Sent: Tuesday, March 02, 2004 7:47 PM
> To: [email-masked]
> Subject: [PWE3] Proposed new 5.4.3 for arch draft
> 
> 
> load-balancing (ECMP), a PW packet must not alias an IP packet. IP
> packets are carried in MPLS label stacks without any protocol
> identifier. he method of packet discrimination must therefore be based
> on some unique characteristic of the IP packet header, which PWE3 must
> not mimic.T
>

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2004 05:21:01 +0000

Ron Cohen wrote:

> Stewart,
> 
> Do you want to change to the following section 5.4.4, or intend to leave
> it as is?

5.4.4 should be deleted. 5.4.4 does not make any sense if 5.4.3 is
modified as suggested.

The text that was formally Arch 5.3.4 and 5.4.4  would then be
the starting point for the new draft.

- Stewart

> Thanks
> Ron
>>From: [email-masked] [mailto:[email-masked]] On 
>>Behalf Of Stewart Bryant
>>Sent: Tuesday, March 02, 2004 4:47 PM
>>To: [email-masked]

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 3 Mar 2004 07:23:17 +0200

Stewart and all,
Please see inline.
Regards,
                Sasha 

-----Original Message-----
From: Stewart Bryant
To: Sasha Vainshtein
Cc: [email-masked] '
Sent: 3/3/2004 6:43 AM
Subject: Re: [PWE3] Proposed new 5.4.3 for arch draft

> [Sasha] IMO it should be MUST NOT or, probably, SHOULD NOT.

The draft is going to be informational, so no capitals. This needs
to get fixed in the rest of the document.
[Sasha] Is the requirements draftthe Standards Track? 
[Sasha] (There is lots of capitalized MUSTs there:-).
> [Sasha] happened, so maybe we should explicitly block this.

If we start to define the solution here, we risk delaying the
draft. My thought was to use the most open definition possible and
thereby minimise the debate, both here and at IESG review.

The checksum solution does not work - there is still the 1/64K chance
of a clash.

- Stewart

---

From: "Ron Cohen" <[email-masked]>
Date: Tue, 2 Mar 2004 21:55:52 -0800

Stewart,

There are many follow-up question related to other parts of the draft
(e.g. fig 11 changes). It might be better to wait for you to send the
modified draft to the list and not comment only on section 5.4.3 text. 

Thanks
Ron

---

From: "Yaakov Stein" <[email-masked]>
Date: Wed, 3 Mar 2004 08:42:44 +0200

Stewart

Why don't you try something easier?

For MPLS PSNs there is an additional constraint on the PW packet format.
In order to facilitate proper functioning of LSRs that detect IP packets
based 
on the initial four bits of the packet content, 
these bits in PW packets must not alias IP version numbers.

The use of this field for PWs is work in progress.

Y(J)S

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2004 06:50:48 +0000

> The use of this field for PWs is work in progress.

I would be happy with that text.

- Stewart

---

From: [email-masked]
Date: Wed, 3 Mar 2004 10:37:32 -0000

Stewart Bryant wrote 03 March 2004 04:44
<snip>

> The checksum solution does not work - there is still the 1/64K chance

NH=>Is there not always a possibility of a mistake here if undetected errors affect the 1st nibble solution?  Though I guess if this was *always* followed up by a checksum test (for *both* assumed IP client and assumed X (PW) client cases) this would at least verify/refute the case (subject to a further 1/64k prob of error).  If 1st nibble neither case then bin?

regards, Neil

---

From: [email-masked]
Date: Wed, 3 Mar 2004 10:37:30 -0000

Stewart Bryant wrote 03 March 2004 04:34
> But this is an MPLS specific problem we are addressing.
NH=> Don't want to labour this one but.....if you think about what you are saying here Stewart, this is yet another example of why MPLS is not the same as IP.
> ECMP per flow and not per packet).
NH=> I agree with Stewart here.  Load balancing is fine within a single layer network (any)....I believe its called TE ;-).  The problems occur when folks try and load balance over nested layered networks.  So if you start peeking inside the payload area of *any* layer network (I could apply this to SDH or OTNs in principle) to see what the clients are then you can expect problems.  This another consequence of mp2p LDP due to this just doing SPF routing.  Even our IP design teams wish ECMP did not exist in MPLS (esp when one considers all the fault-management problems it creates too).

As an aside, I think we ought to be looking for solutions that don't promulgate past errors of judgement....if we can find any of course.

regards, Neil
<snipped>

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2004 11:24:46 +0000

[email-masked] wrote:

> NH=>Is there not always a possibility of a mistake here if 
   undetected errors affect the 1st nibble solution?

Yes

   Though
   I guess if this was *always* followed up by a checksum test
   (for *both* assumed IP client and assumed X (PW) client cases)
   this would at least verify/refute the case (subject to a
   further 1/64k prob of error).  If 1st nibble neither case
   then bin?

I think that it would be better to leave it to the application
to decide how important it is to trap undetected datalink and
memory error. If it is important, then check the checksum (not
that the checksum is a very strong check) otherwise don't.
The point about the nibble vs the checksum is that if we control
the nibble, the application gets to choose the certainty of
identification.

---

From: "Yaakov Stein" <[email-masked]>
Date: Wed, 3 Mar 2004 13:36:29 +0200

> > > Seems more generic to just mention ECMP instead of MPLS, or
mention MPLS as an 
> > > example.

> NH=> Don't want to labour this one but.....if you think about 
> what you are saying here Stewart, this is yet another example 
> of why MPLS is not the same as IP.
> > 
> > ECMP in an IP network works just fine (provided you ECMP per flow
and 
> > not per packet).

> NH=> I agree with Stewart here.  Load balancing is fine 
> within a single layer network (any)....I believe its called 
> TE ;-).  The problems occur when folks try and load balance 
> over nested layered networks.  So if you start peeking inside 
> the payload area of *any* layer network (I could apply this 
> to SDH or OTNs in principle) to see what the clients are then 
> you can expect problems.  This another consequence of mp2p 
> LDP due to this just doing SPF routing.  Even our IP design 
> teams wish ECMP did not exist in MPLS (esp when one considers 
> all the fault-management problems it creates too).

Isn't it refreshing to see Neil agree with Stewart ?

Stewart - It isn't completely clear to me why and how an IP network
performs load balancing on a per-flow rather than per-packet basis.
Is this done by IP source and destination address?
What if we have several PWs between these two IP addresses?

Neil - One of the initial justifications of MPLS was to solve
the IP equal-cost problem. Of course this could be done
by only allowing MPLS-TE solutions, and that may be what operators
like yourself need. 

But there are a lot of best-effort uses for MPLS as well, 
and these shouldn't have to suffer from overloaded links
when there are underloaded ones available. 
Admittedly peaking into client layer headers is a hack,
but it is such a useful one that it may qualify as one of those
rare exceptions where the rules can be broken
(or better, the MPLS rules should be modified by adding a type
identifier).

Y(J)S

---

From: [email-masked]
Date: Wed, 3 Mar 2004 15:45:52 -0000

> when there are underloaded ones available. 

I maybe missing something but I don't see how using a best-effort flavour of MPLS (i.e. any non-P2P non-co-ps mode of MPLS) can really solve the IP equal-cost problem when it introduces it's own equal-cost problem (because it tracks the IP IGP).  It's just shifting the problem to somewhere else in the network stack.

Under what circumstances does a cl-ps best-effort flavour of MPLS add value above and beyond what could be achieved using cl-ps best-effort IP (I can't think of any)?

> rare exceptions where the rules can be broken

Useful to whom?  I would argue that if you consider its full implications (including OAM, fault finding/diagnosis, SLA tracking and NMS/OSS implications) then it isn't that useful to service providers, it's just a complicated pain in the backside.

IMO it's a case of treating the symptoms and not the cause, this is further compounded by the additional OAM etc. hacks required to fault manage the original ECMP hack.  IMO time would be better spent deprecating the hack and concentrating on how to migrate away from it than producing new hacks that try and fix the old hacks.

Ben

---

From: "Yaakov Stein" <[email-masked]>
Date: Thu, 4 Mar 2004 15:00:40 +0200

Ben, 
 
> I maybe missing something but I don't see how using a 
> best-effort flavour of MPLS (i.e. any non-P2P non-co-ps mode 
> of MPLS) can really solve the IP equal-cost problem when it 
> introduces it's own equal-cost problem (because it tracks the 
> IP IGP).  It's just shifting the problem to somewhere else in 
> the network stack.

What I meant was that one could define the FEC based on more
than just the destination IP address, e.g. using the source IP address
as well. Then the LER could assign a label and solve "fish diagram"
overload problems for best-effort IP.

> Under what circumstances does a cl-ps best-effort flavour of 
> MPLS add value above and beyond what could be achieved using 
> cl-ps best-effort IP (I can't think of any)?

Continuing my previous remark, the performance of such a network
is better than an IP ECMP scheme that sends packets along different
routes on a packet-by-packet basis, based only on IP destination
address.

Of course the MPLS network is being used in "CO mode",
even though there is no TE. 
I really don't know what "CL MPLS" is,
unless you mean that MPLS switches can also forward unlabeled IP
packets, 
but then they are functioning as an IP routers.

> > Admittedly peaking into client layer headers is a hack, but  it is
such 
> > a useful one that it may qualify as one of those rare exceptions
where 
> > the rules can be broken
 
> Useful to whom?  I would argue that if you consider its full 
> implications (including OAM, fault finding/diagnosis, SLA 
> tracking and NMS/OSS implications) then it isn't that useful 
> to service providers, it's just a complicated pain in the backside.

Once again, I am not forcing you to use ECMP
for well-engineered operator networks with TE. 
But this mechanism IS useful for best-effort traffic,
and there is an awful lot of best-effort traffic out there.
 
> IMO it's a case of treating the symptoms and not the cause, 
> this is further compounded by the additional OAM etc. hacks 
> required to fault manage the original ECMP hack.  IMO time 
> would be better spent deprecating the hack and concentrating 
> on how to migrate away from it than producing new hacks that 
> try and fix the old hacks.

The question is whether MPLS can still be changed,
e.g. whether it is still possible to add type ECN fields.
However, the PWE list is not the best place to ask that question.

Y(J)S