Subject: Re: [rbridge] Hop Count processing

From: James Carlson <[email-masked]>
Date: Mon, 1 Jun 2009 07:49:47 -0400

Joe Touch writes:
> > Much better.  The "forwarding" part is the qualifier most likely to
> > confuse, and it doesn't actually change the meaning.
> 
> First, anyone who doesn't know what forwarding means is going to have a
> lot of problems implementing an rbridge.

There are at least two possible senses of "forwarding" here.  One
would be examining the TRILL header's destination nickname, finding
that it's not equal to the local system's nickname, and sending the
message on its merry way.  Another would be decapsulating the TRILL
header and then _forwarding_ the packet inside to the bridge links.

That's why I said it's likely to confuse.

And it doesn't change the meaning with respect to what Dinesh, Radia,
and I (among others) have been suggesting for TRILL.  That would be:

	- Check the Hop Count value on input, and drop the packet if
          0, *regardless* of the destination nickname.

	- When you find that the destination nickname is not your own,
          and that (due to IS-IS) you have a next hop and link for the
          destination nickname, decrement the Hop Count and forward
          the packet along without checking the new Hop Count value.

> Third, it not only changes the meaning, it changes the behavior. See the
> walkthrough in my previous mail.

The change is intentional.  The existing Hop Count behavior (as
pointed out by Dinesh) would not work if we wanted to trace the path
of packets through the network.  Altering it as suggested does support
that goal.

I agree that it's not at all like IPv4.  "Normal" hop count / TTL
processing involves ignoring the value completely at input time, and
checking it only when attempting to forward -- after determining that
the packet isn't intended for the local system.

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 07:03:45 -0700

> header and then _forwarding_ the packet inside to the bridge links.

That's solved easily by referring to the former as forwarding the TRILL
packet.

Further, bridges don't forward segments; they switch them (i.e.,
forwarding is usually a L3 term), so the final step would be a
"decapsulate and switch" operation, but as I noted, that is solved above.

>           the packet along without checking the new Hop Count value.

By removing 'forwarding' from the language I proposed, Radia removed
behavior that happens only at TRILL forwarding steps - which is what
you've just described.

> The change is intentional.

Well, you said (and I disagreed) that:

" The "forwarding" part is the qualifier most likely to
confuse, and it doesn't actually change the meaning."

My point is that it DOES change the meaning. It is clearly important to
include.

> The existing Hop Count behavior (as
> of packets through the network.

Please explain that, and use an example that shows hopcounts for
ingress-egress paths that include 0, 1, and 2 rbridges (as I have

> the packet isn't intended for the local system.

That second step is required anyway. Why is it important to consider
hopcount for packets that have already been received? That seems to say
"OK, we limit the hops in a network, but drop you if your hopcount is
wrong even if you've made it to the destination".

Why is that useful for traceroute?

(I'm guessing that one of you thinks that we can get away without a
"ping" message, i.e., that we can trace regular packets. That won't
work, however, if the egress rbridge has more than one egress address,
as I discussed.)

Joe

---

From: Dinesh G Dutt <[email-masked]>
Date: Mon, 01 Jun 2009 08:02:36 -0700

Using regular frames was my intention. Since end stations are not 
invlved, I can test actual customer frame and thereby know the exact 
path for a frame.

> work, however, if the egress rbridge has more than one egress address,
Do you mean for multi-dst addresses ?

Dinesh

-- 
We make our world significant by the courage of our questions and by 
the depth of our answers.                               - Carl Sagan

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 08:30:01 -0700

> invlved, I can test actual customer frame and thereby know the exact

That won't work as per below.

> Do you mean for multi-dst addresses ?

I believe I can have an rbridge that sits on two different ethernet
segments, i.e., that has two different egress addresses.

When I get something with hopcount=0, I respond back with an "error
hopcount exceeded". I need to pick one of the egress tags to use as the
message source address. I'm presuming I pick the "canonical address" of
that rbridge, for which there would be only one.

Consider a frame sent to the non-canonical egress. You would receive
back a sequence of error messages:

	rbridge1
	rbridge2
	rbridge3
	...
	rbridge_last

Now one of two things has happened. Either the last rbridge was the
correct one that decapsulated the packet, or it wasn't. The received
source address of the error message will not match in both cases.

So how do you know you reached the last hop?

Why is it even important to be able to do a traceroute with a regular
packet? Why not just require something like PING, i.e., a TRILL message
that, once received at the destination address in the TRILL header,
responds with a "success" response?

Joe

---

From: James Carlson <[email-masked]>
Date: Mon, 1 Jun 2009 11:50:40 -0400

> that rbridge, for which there would be only one.

You'd pick the MAC address of the interface over which you received
the frame.

> So how do you know you reached the last hop?

Because frames with higher Hop Count never return.

> responds with a "success" response?

That's good question.  I have exactly the same concern -- basically,
that we're designing the 'traceroute' infrastructure before we know
anything about how the functionality will be implemented -- but Dinesh
convinced me that it doesn't matter.

If we do this as Dinesh has argued, then we'll always have the drop
information from that last hop.  If we don't, then we won't.  That
means that the method proposed provides additional information without
significant expense.

I agree that a decent 'traceroute' feature should use a special packet
that allows us to send back "and here's what I'd do with that once
I've decapsulated" information from the last (destination) TRILL node.

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 09:00:29 -0700

> You'd pick the MAC address of the interface over which you received

What if there are multiple such addresses (i.e., MAC overloading)?

> Because frames with higher Hop Count never return.

That could also mean:

	- the return error message was lost

	- routing had an incomplete path (never reaches the dest)

> information from that last hop. 

But you don't know it's the last hop, as per above.

> If we don't, then we won't.  That

As in my other message, this doesn't correlate to user packet behavior
anyway. We need to keep sending the same packet repeatedly with
increasing hopcount, which is a separate function that needs to be
implemented. That function has no benefit from using a user packet as
payload, since the TRILL header hides that content anyway.

> I've decapsulated" information from the last (destination) TRILL node.

That'd be "PING" ('echo request') (i.e., that's how traceroute works in
IPv4/IPv6).

Joe

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 08:55:22 -0700

Dinesh G Dutt wrote:
> When I send a unicast traceroute, I put in an address of the egress
> Rbridge I expect is the final destination (known by looking at my local
> MAC table, for example). Then, why do I have a problem ? I guess I don't
> understand what you're saying below,

I said it below fairly directly:

1) let there be an rbridge, Y, with two egress addresses C and D
let there be an rbridge, X, with ingress address A

Note: C and D are different so that Y can be attached to two different
segments; packets decapsulated with C go onto one segment, packets
decapsulated with D go to the other.

2) send a packet through an rbridge encapsulated with the TRILL header
"A->C" (A is ingress; C is egress)

Send the packet through with a hopcount such that it reaches Y with
hopcount=0. At that point, Y will send a "hopcount exceeded" packet back
to X. What is the source address of that packet?

- ----

Note: I'm very confused at the purpose of wanting see the path of a
"user packet". TRILL doesn't examine anything but the TRILL headers
anyway. Users can't control the TRILL hopcount by the segments they
send. Traceroute works only if the segment is resent with increasing
hopcounts anyway - which users can't do.

Joe

---

From: James Carlson <[email-masked]>
Date: Mon, 1 Jun 2009 11:56:19 -0400

> "decapsulate and switch" operation, but as I noted, that is solved above.

Not per the standards.  802.1D calls this "forwarding" and "switching"
sounds like marketing to me ... but I guess I don't care to argue the
point.

> behavior that happens only at TRILL forwarding steps - which is what

Yes; just explaining the idea.

> My point is that it DOES change the meaning. It is clearly important to

If you include it, and if it does mean something, then it breaks the
proposal that Radia, Dinesh, and others are putting forward.

> ingress-egress paths that include 0, 1, and 2 rbridges (as I have

For ingress sending directly to egress, you'd send with a Hop Count of
0, and the target would detect this as a drop condition on input and
reply.  Higher hop counts would not return.

For ingress sending through one TRILL hop to the egress, you'd send
with Hop Count of 0, get a response from the hop node, then send with
1, and get a response from the destination (final) node as above.

For higher numbers, prove by induction.

> wrong even if you've made it to the destination".

Yes, that's exactly what Dinesh is proposing.

> Why is that useful for traceroute?

It allows us to capture a drop at the last hop.

(Dinesh insists that it's also how routers have "always" worked, but I
have my doubts about that.)

> "ping" message, i.e., that we can trace regular packets.

Exactly.

(Of course, without an actual traceroute-for-TRILL proposal in front
of us, it's hard for me to argue that this is in fact how it should
work.)

> That won't

I don't follow.  A TRILL "traceroute" can't see anything past the
TRILL destination node, so who cares how L2 is forwarded^Wswitched
past there?

---

From: Dinesh G Dutt <[email-masked]>
Date: Mon, 01 Jun 2009 08:46:41 -0700

Dinesh
Joe Touch wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Dinesh G Dutt wrote:
>   
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkoj8/kACgkQE5f5cImnZrtJxACfWRpCTqUO4X6z8o9AW/i07NoC
> UK4An1CMJT3BYM9F2/cFrjF7bhZDFbuH
> =efIl
> -----END PGP SIGNATURE-----
>
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 09:18:09 -0700

James Carlson wrote:
...
> proposal that Radia, Dinesh, and others are putting forward.

There are different things here:

1) what hopcount does a packet have that successfully reaches its
destination?

	we can make that 0 (as with IPv4 and IPv6)

	we can make that 1

Making it 1 has the effect of making a traceroute send a "hopcount
exceeded" from the rbridge at the destination, but unfortunately there
are ways in which that message might not be interpreted as "it got
there" (i.e., it could be "this is the last rbridge I saw, but I have no
idea where it got).

2) whether we need to refer to rbridge forwarding in the description of
hopcount processing

	we absolutely do.

I've seen three agreements that claim we don't, but I've demonstrated
this is not possible. We need to differentiate how we handle packets
that have arrived at their destination (which are decapsulated and their
payload is sent out an interface) from packets that are forwarded (which
are sent out an interface themselves). I.e., we need to distinguish
between receiving and forwarding *TRILL* frames. When we receive a TRILL
frame, we forward (if you like that term) its payload, but not the TRILL
frame.

> 1, and get a response from the destination (final) node as above.

Agreed, though as I've shown:

a) we still need active participation by the source rbridge (this cannot
be initiated by a user's packet)
	- to set the hopcount
	- to repeatedly send the same packet

b) we still need to differentiate when a packet has reached its
destination in how hopcounts are processed. you can't just say "remove
the word forwarding" from hopcount processing; it needs to be there

> have my doubts about that.)

That may be how some routers incorrectly implement the requirements for
IPv4 or IPv6, but it fails to explain the success of sending TTL=0
between hosts on the same LAN.

> of us, it's hard for me to argue that this is in fact how it should

That'd be very useful.

> TRILL destination node, so who cares how L2 is forwarded^Wswitched

I'm talking about an rbridge that has more than one egress address,
e.g., via overloading on a single interface or with multiple interfaces.
The problem is that your "hopcount exceeded" message won't know what
address to use, and if it picks one that isn't the packet's destination
then you will not know the difference between a packet that arrives and
one that hits a black hole.

Joe

---

From: James Carlson <[email-masked]>
Date: Mon, 1 Jun 2009 13:11:09 -0400

> What if there are multiple such addresses (i.e., MAC overloading)?

Who cares?

How is that any different from having potentially multiple addresses
on a link at _any_ point in the middle of the path with multiple TRILL
hops?

I would argue that -- if this is actually an issue for any real
implementation (I don't know that it is) -- then it's a generic
issue.  The TRILL implementation in that system would need to know how
to deal with multiple MAC addresses on each port, and the current
document does not deal with that.

It's not merely an issue with traceroute-like functionality, nor is a
special issue with the last hop.  Thus I don't understand why you're
bringing it up here.

> 	- routing had an incomplete path (never reaches the dest)

Sure.  The other way you know that your path is complete is that the
system that sends back the last message happens to have the ID of the
original TRILL destination.

> But you don't know it's the last hop, as per above.

You can.  See above.  :-/

> payload, since the TRILL header hides that content anyway.

Correct; this would need to be initiated from a TRILL node.

> That'd be "PING" ('echo request') (i.e., that's how traceroute works in

It could be *anything*.  We're designing a new protocol.  We're not
constrained to the hackery used in ping.  (E.g., having an undefined
payload format that is used by each node in proprietary ways.)

Note that _real_ traceroute doesn't use ping, but instead uses UDP
datagrams sent to an arbitrary (unused) port.  ICMP Echo for
traceroute is a Windowism.  ;-}

> there" (i.e., it could be "this is the last rbridge I saw, but I have no

So far, that seems to be "good enough."  Plus, since we have a
database of IDs for the RBridges (the nickname database), it's easy to
figure out what you're looking at.

It's worth noting that IPv4 traceroute has the same ambiguity.  Not
all nodes bother responding to arbitrary packets, nor do they all
allow ICMP errors to get through.  The result is that some traceroute
attempts just trail off in a bunch of stars.  (Try tracerouting to
www.sun.com ...)

> this is not possible.

I don't believe you have.

> We need to differentiate how we handle packets

Actually, I don't believe that's necessary.

I agree that it'd be nice to have a distinguished message for the
"last" hop, and to have important information sent back from all
hops.  Without a definition of how traceroute is supposed to function
in TRILL, we'll need to defer that.

> be initiated by a user's packet)

Yep; agreed.  I don't think anyone was suggesting otherwise.

> the word forwarding" from hopcount processing; it needs to be there

Actually, it works without it.

> between hosts on the same LAN.

At least for the proposal under discussion, that particular router bug
is immaterial.

> one that hits a black hole.

It must have had some way of receiving the message in the first place.

If it's unicast, then the original destination on the 'trace' message
should be the source for the reply.  If it's multicast (a frightening
concept), then the source should be whatever source MAC address is
usually used for TRILL messages from that node when sending over the
given link on which the original message was received.

Note, of course, that we're arguing about the number of angels on the
head of a pin at this point, as the traceroute protocol itself *HAS
NOT BEEN PROPOSED*.  How it might choose MAC addresses to use seems
quite well off-topic and doesn't particularly help illuminate the
issue.

---

From: Joe Touch <[email-masked]>
Date: Mon, 01 Jun 2009 11:08:31 -0700

> on a link at _any_ point in the middle of the path with multiple TRILL

Th difference is that you'll get a set of responses back, where you send
a packet from X->Y and you see:
	P
	Q
	R
	S
	T

And that's it. At that point, you have no idea if T is the same node as
Y, or whether T is just a dead end in the routing (a black hole, e.g.).

If your point is to know you reached the end, you basically need:

	failed - hopcount exceeded
	failed - hopcount exceeded
	succeeded

Another "failed - hopcount exceeded" won't cut it.

> document does not deal with that.

You only need to know whether the multiple MAC addresses are the same
rbridge when you are doing this sort of thing; for pings or anything
direct that you get "postitive feedback" on, you have no problem.

> special issue with the last hop.  Thus I don't understand why you're

Because it *is* both an issue with only this sort of traceroute solution
and on the last hop, as noted above.

> original TRILL destination.

See above; you could easily get back an ID that is for the last hop, but
NOT the original destination.

> payload format that is used by each node in proprietary ways.)

Ping is a useful protocol regardless - it tells you the other end is up.
 We probably can use that for diagnostics anyway.

The "hack" here, if anything, is trying to infer a packet has arrived
when you get an error message from the place where the TTL goes to zero,
rather than by getting a "success" message.

> datagrams sent to an arbitrary (unused) port. 

The first version of traceroute used PING. It was changed because PING
isn't required. We don't have ports, but we can require the PING be
implemented.

> ICMP Echo for

See above. Windows uses the original definition. Using ICMP pings
remains an option in most traceroute implementations.

> attempts just trail off in a bunch of stars.  (Try tracerouting to

This matters only for the destination; at the destination, you get a
different message - THAT is what makes it work, and that's not what
we're doing yet.

> I don't believe you have.

The definition you gave had it. I showed how all other definitions
proposed failed to give sensible results. What other sort of proof do
you need?

Or do you have another description of the processing that does not refer
to forwarding that you would like checked?

> in TRILL, we'll need to defer that.

If you aren't going to define traceroute, then how can you argue what we
need in terms of hopcount to support it?

> Actually, it works without it.

Again, show me a description that works.

...
> quite well off-topic and doesn't particularly help illuminate the

Until someone has a description of traceroute, it's premature to use
traceroute as an argument for how to do hopcount processing.

Joe

---

From: James Carlson <[email-masked]>
Date: Mon, 1 Jun 2009 15:26:28 -0400

> traceroute as an argument for how to do hopcount processing.

I don't believe that there is another reasonable argument that
supports it.

(I've heard privately that some implementations may have unspecified
"performance problems" if they have to check after decrementing, but
that's just baffling to me, and certainly not a good justification for
changing the way this feature operates.)