Subject: [rbridge] TRILL over IP

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Wed, 16 Nov 2011 18:45:33 -0800

Hi Margret

Per comments I made during the TRILL WG session, below are links to two
related drafts. First draft cover OTV itself. Second draft utilize the
same frame format. 

http://tools.ietf.org/html/draft-hasmit-otv-03

http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-00

Thanks

Tissa

_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Joe Touch <[email-masked]>
Date: Wed, 16 Nov 2011 19:20:33 -0800

FWIW, Trill does effectively create ingress-to-egress tunnels, since all 
packets are encapsulated with the source of their ingress and 
destination of their egress.

A broader concern is doing L2 over L3, and potential concerns with 
loops, IMO. I.e., how do you ensure that a packet doesn't go down from 
the app, into L3, into L2, encapsulate in L3, then end up back in the 
same L2 somehow and back where it started?

Joe

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 17 Nov 2011 03:50:59 +0000

Joe, 

> -----Original Message-----

[Linda] That is a very valid point. This point needs to be brought to the NV03 work (i.e. Ethernet over IP) as well.  
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Margaret Wasserman <[email-masked]>
Date: Wed, 16 Nov 2011 23:27:42 -0500

Hi Joe,

> same L2 somehow and back where it started?

While I agree that this is a very valid issue for L2 over L3 tunnels, in general, I am not sure that this is a concern for TRILL over IP...

In TRILL over IP, the tunneled link is seen as a  link within the TRILL campus.  As such, I believe that TRILL itself will avoid loops containing the tunnel link.  

Am I missing something or misunderstanding your concern?

Margaret

---

From: Anoop Ghanwani <[email-masked]>
Date: Wed, 16 Nov 2011 20:20:27 -0800

> same L2 somehow and back where it started?

This is an interesting thought.  I guess they would need something
similar to the AF election stuff that we have in TRILL, to make sure
there is only one device on a LAN encapsulating/decapsulating traffic
to/from the overlay.
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Joe Touch <[email-masked]>
Date: Wed, 16 Nov 2011 20:38:19 -0800

> Am I missing something or misunderstanding your concern?

A packet comes into TRILL campus A. In the process f transiting that 
campus, it goes over an L2 link that ends up being encapsulated over L3, 
which then gets encapsulated over L2, and ends up arriving at that 
campus again.

Joe

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 17 Nov 2011 00:13:39 -0500

On Nov 16, 2011, at 11:38 PM, Joe Touch wrote:
> 
> A packet comes into TRILL campus A. In the process f transiting that campus, it goes over an L2 link that ends up being encapsulated over L3, which then gets encapsulated over L2, and ends up arriving at that campus again.

As TRILL over IP is currently defined, it is used as a link between two RBridges within a single TRILL campus, not as a way to connect separate TRILL campuses...  

Does that make a difference here?

Margaret

---

From: Joe Touch <[email-masked]>
Date: Wed, 16 Nov 2011 21:39:06 -0800

On 11/16/2011 9:13 PM, Margaret Wasserman wrote:
>> campus, it goes over an L2 link that ends up being encapsulated over
>> L3, which then gets encapsulated over L2, and ends up arriving at that
>> campus again.
>
> As TRILL over IP is currently defined, it is used as a link between two
> RBridges within a single TRILL campus, not as a way to connect separate
> TRILL campuses...
>

I don't think so; a campus includes the ingress/egress nodes.

Joe

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 17 Nov 2011 01:00:06 -0500

Hi Anoop,

> there is only one device on a LAN encapsulating/decapsulating traffic

The AF mechanism has to do with protection from loops involving native
frames. The assumption here is that the IP link is only between
RBridge ports so you only have TRILL encapsulated data over IP on it.
You do not have native frames over IP. If someone attempted to inject
a native frame by IP encapsulating it to the link, I would think it
would be discarded.

That said, unless the draft ends up saying something to the contrary,
the AF mechanism would be operating on this IP link the same as it
operates on any other potentially multi-access TRILL link. So only one
RBridge connected to the IP link would ingress it, for example.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 17 Nov 2011 01:16:35 -0500

Hi Tissa,

Thanks for these pointers. However, I believe these drafts describe
significantly more complex mechanisms. The TRILL over IP draft is just
aimed at providing an IP technology link between two or more RBridge
ports to carry TRILL Data and TRILL IS-IS frames within a unified
TRILL campus, the same as any other inter-RBridge link based on some
other technology.

Thanks,
Donald

On Wed, Nov 16, 2011 at 9:45 PM, Tissa Senevirathne (tsenevir)
<[email-masked]> wrote:
> related drafts. First draft cover OTV itself. Second draft utilize the same
> frame format.
>
>

---

From: Victor Moreno <[email-masked]>
Date: Wed, 16 Nov 2011 22:43:20 -0800

> there is only one device on a LAN encapsulating/decapsulating traffic

That is correct, that is what OTV does in these scenarios: we elect an Authoritative Edge-device (AE) per VLAN. This also handles the de-duplication of multicast traffic.

-v

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 17 Nov 2011 03:18:50 -0500

Hi Victor,

On Thu, Nov 17, 2011 at 1:43 AM, Victor Moreno <[email-masked]> wrote:
>> ...
> That is correct, that is what OTV does in these scenarios: we elect
> an Authoritative Edge-device (AE) per VLAN. This also handles the
> de-duplication of multicast traffic.

Maybe I'm confused and people will straighten me out, but I do not
think it is correct, at least as I understand it. Consider the
following

|
|    |
|  +-+-+
+--+RB1+===++    |
|  +---+   ||    |
|          ||  +-+-+
|          ++==+RB3+----
|          ||  +-+-+
|  +---+   ||    #
+--+RB2+===++    #
|  +-+-+       +-+-+
|    |         |RB4+----------
|    |         +-+-+
|                |

Anoop: The single lines are Ethernet and the double line is IP (and
the line composed of hash marks (#) is a PPP link that I threw in just
for fun). There is a multi-access Ethernet TRILL link on the left,
probably a bridged LAN, and there is a multi-access IP TRILL link
connecting RB1, RB2, and RB3. There are two devices on the LAN on the
left, RB1 and RB2, both of which are sending/receiving TRILL frames
from the IP link but I don't see that this causes any problem. In
fact, I don't quite understand why you say
"encapsulating/decapsulating traffic". As far as TRILL is concerned,
there is no extra encapsulation of traffic on the IP link. It is just
that the outer envelope (the link header/trailer added at the RBridge
output port) is IP instead of Ethernet. Perhaps the word
"encapsulating" in the abstract should be replaced with "transmitting"
or something...

One of RB1, RB2, and RB3 will be elected DRB and may create an IS-IS
psuedonode for the IP link and this DRB could appoint forwarders if it
felt link it but, since there are not going to be any native frames on
the TRILL IP link, the AFs wouldn't have anything to do. Maybe the
draft should require that the RBridge ports on an IP link be
configured as trunk ports...

Victor: To the extent that I understand what you mean by
"de-duplication of multicast traffic" it is done, to the extent
necessary, by the TRILL Reverse Path Forwarding Check.

Thanks,
Donald

---

From: James Carlson <[email-masked]>
Date: Thu, 17 Nov 2011 07:37:58 -0500

> +--+RB1+===++    |

Now consider a plain IP router (not TRILL-anything) within that
double-line area.  There may be one or more routers in there, as those
are IP connections.

If that router has a connection to a link that is attached to any edge
of the campus, then bad things can happen.  A packet from the campus
could get sent over the IP network (say from RB1 to RB3), hit that IP
router, which then decides (perhaps erroneously) to forward through the
campus (which it sees as just a single network hop; it's all bridged,
after all).  The destination Ethernet address (RB3) causes a campus
TRILL ingress to pick up the IP packet, wrap it in a TRILL header, and
forward along via TRILL headers.  It can then get sent back over the IP
link ...

Lather, rinse, repeat.  Mix in packet amplification via the distribution
mechanisms for unknown destinations and multicast, and you've got a
storm of RFC 1326 proportions.

I think the assumption necessary for those IP links is that they are all
on a "private" IP network of some kind, or that there's never any
connection from any IP router to any TRILL campus edge.

Those assumptions seem a bit harsh (and unverifiable), so I suspect
you'll need some way to prohibit the loop from forming.  One way to do
that would be to force TRILL ingresses to behave as a filter: if it's an
IP packet, and the protocol/transport/port indicates that it's a TRILL
packet, then drop.

-- 
James Carlson         42.703N 71.076W         <[email-masked]>
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Donald Eastlake <[email-masked]>
Date: Sun, 20 Nov 2011 21:50:47 -0500

Hi Jim,

> connection from any IP router to any TRILL campus edge.

I think you need to express the condition more carefully. If you have
a data center using TRILL, it would commonly have RBridges connected
to IP routers leading to the global Internet or at least an enterprise
Internet. It seems to me that the actual condition is that if there is
a path between two RBridge ports, the encapsulations needs to be
balanced between them (sort of like balancing parenthesis). For
example, if for some reason you had a port that was TRILL over PPP
over Ethernet and there was path from it to an Ethernet ingress
RBridge port, you could have a somewhat similar problem, although this
is much less likely than it would be with TRILL over IP.

> IP packet, and the protocol/transport/port indicates that it's a TRILL

I don't think it is reasonable for the TRILL over IP draft to impose a
requirement on all RBridge ingress ports. However, it is reasonable
for it to impose a requirement on all RBridge IP ports. So I would
suggest a check before IP encapsulation to discard the frame if it is
a TRILL over Ethernet frame.

Thanks,
Donald

---

From: Radia Perlman <[email-masked]>
Date: Sun, 20 Nov 2011 23:00:01 -0800

I believe that Margaret's draft is a way of sending
already-TRILL-encapsulated traffic over an RB-to-RB link that just
happens to be IP rather than Ethernet.  So there's no issue about
electing AFs.

Margaret's draft can be used to interconnect two distant data centers
by glueing them together by having at least one RBridge, R1, in one
center, creating a TRILL link to another RBridge RB2, in the other
center.  Once this is done, this becomes a single TRILL fabric.  Other
than RB1 and RB2, the other TRILL RBs don't care that the link between
R1 and R2 is IP or Ethernet.  It's just a link reported in the TRILL
topology.

I suppose if there were a lot of RBridges interconnected with a
backbone IP network, then instead of modeling the links across the
backbone as a bunch of point-to-point TRILL links, one might want to
consider the IP backbone as a single shared link, elect a DRB, create
a pseudonode, etc.  This adds complications like wanting to multicast
Hellos, LSPs, CSNPs, etc. But in most cases, just reporting the
pt-to-pt links between pairs of RBs that are configured to create IP
tunnels to each other, and unicasting all the control traffic between
the pairs of RBs that want to mate across the IP backbone seems simple
and useful and sufficient.

Unless I'm confused about the problem being solved, or something, of course.

Radia

On Wed, Nov 16, 2011 at 9:13 PM, Margaret Wasserman <[email-masked]> wrote:
> A packet comes into TRILL campus A. In the process f transiting that campus,
> it goes over an L2 link that ends up being encapsulated over L3, which then
> gets encapsulated over L2, and ends up arriving at that campus again.
>

---

From: James Carlson <[email-masked]>
Date: Mon, 21 Nov 2011 17:20:51 -0500

Donald Eastlake wrote:
> balanced between them (sort of like balancing parenthesis).

I don't think that's good enough.  Any place where this IP-encapsulated
packet can enter the campus TRILL area to be forwarded by TRILL itself
is a risk.  That entry could be by accident, such as a long-lived
forwarding loop.

IP has some protection against events like that, as does TRILL itself,
but mutually-encapsulating protocols (somewhat by definition) do not.

> For

True, that's possible.  It would probably be wise to have a usage note
in the TRILL over PPP text to say that using it with PPPoE or with L2TP
running over a network that may include Ethernet is discouraged, due to
the risk of mutual encapsulation, and lack of any coherent reason to do
that.

> a TRILL over Ethernet frame.

The frame you're about to encapsulate badly won't look like TRILL over
Ethernet.  Instead, it will look like TRILL over IP on Ethernet.  That's
the failure case: where you get your TRILL-over-IP-on-Ethernet traffic
entering the campus to be forwarded along just like all of the other
blah-on-Ethernet traffic.

In a belt-and-suspenders way, I think it's wise for anyone dealing with
this stuff to drop bits that can form an encapsulation-nesting loop like
that.  I don't think (in a global sense) that it matters who is being
"imposed" upon, because the RBridges shouldn't have feelings.

But, yes, as long as the loop is cut in at least one place, the risk
should go away.  At least until someone does something you didn't expect
-- like Ethernet on L2TP on UDP on IP on Ethernet.  Then you're hosed.  :-/

---

From: Donald Eastlake <[email-masked]>
Date: Tue, 22 Nov 2011 00:38:23 -0500

Hi Jim,

> is a risk.  That entry could be by accident, such as a long-lived

But the parenthesis are not balanced in that case. If something goes
out a TRILL port that is TRILL over IP over Etherenet the port is sort
of (IP(Ethernet. If that is accepted by a port on the TRILL campus
that doesn't first strip Ethernet and then strip IP, that is to say is
not Ethernet)IP), you have a problem.

> blah-on-Ethernet traffic.

Right. My statement above is incorrect and should have said "TRILL
over IP over Ethernet" at the end.

> "imposed" upon, because the RBridges shouldn't have feelings.

I wasn't talking about feelings. Since TRILL assumes all end station
frames are Ethernet, it just seems reasonable that there will
typically be more Ethernet ports than IP ports so, if additional
complexity is needed, it should be avoided at Ethernet ports and put
at other ports.

> -- like Ethernet on L2TP on UDP on IP on Ethernet.  Then you're hosed.  :-/

Sounds to me like someone doing that already has a potential looping
problem even in the absence of TRILL.

Donald

---

From: James Carlson <[email-masked]>
Date: Tue, 22 Nov 2011 10:28:01 -0500

On 11/22/11 00:38, Donald Eastlake wrote:
> But the parenthesis are not balanced in that case.

The "parenthesis" you're referring to are exactly equivalent to my
assertion that the IP network must be 'private' relative to the campus.

In other words, as a datagram-routed protocol, IP datagrams can go
"anywhere" on their journey towards the L3 destination, where the
TRILL-IP decapsulation and TRILL-based forwarding should occur.  That
"anywhere" is subject to the whims and foibles of IP forwarding.  There
may be loops.  There may be transient errors.  And the fact that the
"TRILL campus" is complicated internally is of no consequence to IP; it
sees this as just a link.

IP node to IP node communication is not like a fixed wire, which I think
is implied by the "parenthesis" you've invoked.

If the IP network on which these TRILL-IP packets are flowing happens to
include a router that connects to an edge of the TRILL campus itself,
then mutual encapsulation -- no matter how unlikely -- is _possible_.
All that it takes is a misdirected TRILL-IP packet routed to some
destination within the campus.

I think that could easily be the result of misconfiguration.  There's no
automated TRILL-IP-tunnel establishment protocol (as there is with L2TP,
for example), so someone has to type in addresses for the endpoints.  If
the poor admin enters the inside ("campus") IP address for the
destination rather than an external link IP address, IP forwarding could
easily be by way of the campus itself.

> If something goes

The problem I think may be missing here is that ports on the TRILL
campus simply accept Ethernet.  They don't really care about stuff
inside, because they're forwarding based on the L2 addresses.  The fact
that IP is inside there doesn't matter.  The fact that TRILL itself may
be running on top of IP inside the packet also doesn't matter.

If some external IP router presents a campus edge with one of these
TRILL-IP packets, then nothing is going to know that any stripping of
anything needs to be done.  Only the IP destination knows it needs to
strip, and we haven't necessarily reached there yet.

> complexity is needed, it should be avoided at Ethernet ports and put

OK.  But everything is fine until some Ethernet-only port inadvertently
accepts one of these TRILL-IP-Ethernet packets from an ordinary IP
router.  That's when things really go south.

> problem even in the absence of TRILL.

Yes.  I was just pointing out the lack of any good way to avoid the
original problem completely in the face of malicious stupidity.

But I do think it's important to avoid creating a new and potentially
easy to encounter problem caused by this new form of mutual encapsulation.