Subject: Re: [RRG] draft-farinacci-lisp-05

From: Iljitsch van Beijnum <[email-masked]>
Date: Sat, 22 Dec 2007 04:04:24 +0100

On 19 dec 2007, at 6:20, Dino Farinacci wrote:

>> There is no version field in the LISP header. There should be. At  
>> least a few "set to zero on send, ignore on receive" bits that can  
>> be used for extensions without bumping the version number is also a  
>> good idea.

> We decided to use control-plane type codes.

Unless I missed something, this means you can never make backwards- 
incompatible changes to the LISP header without running the old and  
new versions on different RLOCs.

>> "In order to eliminate the need for a mapping lookup in the reverse  
>> direction, the ETR gleans RLOC information from the LISP header."

>> I find this undesirable because this way, the behavior of the  
>> system can be different depending on which end initiated the  
>> communication. Trusting information supplied directly by the other  
>> end is also problematic security-wise. I would much prefer it if  
>> both ends did an independent mapping lookup.

> We stated this because there was a requirement from big content  
> providers. I will loosen the  language and say "MAY glean".

This is one of the big problems with GSE: if someone contacts you with  
EID=windowsupdate.com and RLOC=l33th4x0r, and you trust this  
relationship, an attacker gets to redirect traffic for that EID to a  
random place. This is especially bad when the attacker can set up this  
state just as you're about to set up an outgoing connection to that  
EID, because then they get to intercept your outgoing traffic.

In the case of content sites that only receive incoming sessions it is  
probably possible to come up with a set of contraints within which  
there are no problems, but that would still make me EXTREMELY  
uncomfortable as people do stuff that they weren't planning on doing  
when they set up their networks all the time. Also, as someone who has  
spent a fair bit of time debugging network problems, I am very much in  
favor of deterministic behavior. So one set of behavior when the  
connection is set up from A to B and another when it's from B to A is  
not good.

>> "LISP Locator Reach Bits: in the LISP header are set by an ITR to  
>> indicate to an ETR the reachability of the Locators in the source  
>> site."

>> I wonder how useful this is in practice. First of all, having 32  
>> RLOCs is way too many.

> We have received feedback that it may not be enough.  ;-)

As the LISP related documents that I've read are light on failure  
detection and repair, it's hard to say anything definitive, but with  
32 ITRs and 32 ETRs TCP has probably long since given up when you get  
around to learning that it's ITR 31 that can talk to ETR 31 but the  
other 1023 combinations don't work.

I'd be interested in learning the rationale behind that feedback,  
though.

[xTRs in ISP networks or (also) in end-user sites]

> As I said at the IETF, for a CE deployment of xTRs, the most common  
> failure points in the network, which affects connectivity to the  
> site, is the CE router going down, the CE-to-PE link going down, or  
> the PE router going down. Other failures are rerouted in the core  
> based on richness of connectivity or are damped out due to  
> aggregation.

That's what the big ISPs tell us. Myself, I haven't had too much  
trouble with my last kilometers, so my experiences may not be  
representative, but I can tell you that routing failurs and brownouts  
DO happen and anyone who cares enough about their connectivity to be  
multihomed, wants to be protected against that, too. Especially having  
a single ETR go down or function incorrectly (claiming incorrect  
(un)reachability for other ETRs serving the same EID) and then being  
unreachable or having severely degraded reachability would be highly  
unacceptable to any multihomer that I've ever known.

> In these cases, the loc-reach-bits are extremely effective and gets  
> the new status information to the other sites at data-plane rates.

Only if the return traffic uses the ITR for the forward traffic as its  
ETR, which makes sense in your view. However, this is a good example  
of a more general tendency in LISP to commit to a narrow mode of  
operation rather than to make the whole thing more open so it's easy  
to change the protocol later for the IETF and to make different  
deployment tradeoffs for operators.

It's always difficult to come up with the right amount of  
extensiblity: too much, and the protocol becomes unwieldy and  
inefficient, too little and obvious improvements can't be made or  
require cumbersome kluges. I would point to BGP as a mostly successful  
example: we didn't have to come up with BGP-5 for more than a decade  
even though we added tons of new stuff because there was an adequate  
amount of extensibility.

I'm currently not seeing this amount of extensiblity in LISP. The fact  
that all of this is happening in an IRTF wg and that we've had  
numerous previous efforts before that either failed to address the  
issue completely (IPv6) or created something that only solves part of  
the problem (shim6, and some would argue that I'm being generous)  
shows that closing off too many paths at this stage is probably not a  
good idea.

>> There are also potential issues with synchronizing the numbering of  
>> the RLOCs in the mapping system and in this field.

> Right, there is no such thing as a free lunch. As mentioned at the  
> Chicago IETF, inserting new RLOCs don't have this problem. That is  
> appending is easy to do. Removing from the middle of the RLOC list  
> gets tricky but you could use a dummy slot for a certainly period of  
> time.
> We are experimenting with this to see the best way to stale existing  
> cache entries.

Right.

> We did consider sending a "free cache" bit in the data plane so  
> sites that have cached state could time out the state and re-request  
> a new mapping if they needed it, but as you might guess it would  
> cause a request implosion to the site.

This is the same argument that you used against my idea of having a  
code point in the LISP header to signal back reachability and other  
information on request.

> So if you have any ideas on how to solve this, we would like to hear  
> with it and prototype the idea.

Well, mostly just the same idea as I had before, but let's flesh it  
out a bit:

The point is to avoid state on the ETRs but still signal information  
that becomes available at the ETR back to the ITR as fast as possible  
without making the ETR keep state or otherwise work too hard.

We do this by having the ITR ask the ETR to send updates of the  
relevant information (not important what exactly that information is  
right now) by setting a code point in the LISP header. To avoid having  
to send back information that's already known, the ETR keeps a "table  
version" like value. I think we only need two or three bits for this.  
When the ITR asks for an update, it supplies the "table version" of  
the latest information that it learned. If the table version at the  
ETR is the same as in the ITR's request, it doesn't do anything. If  
the table version is different, the ETR sends back an update. The ITR  
keeps an RTT estimate and makes sure there is only one request in  
flight per RTT so the ETR won't be sending unnecessary copies of the  
information packets.

(We'll have to figure out if we need to do this per-ETR or per-EID, or  
possibly both.)

> But, umm, isn't determinism better?

Yes, that was my point. (?)

>> " Priority: each RLOC is assigned a priority. Lower values are more  
>> preferable. When multiple RLOCs have the same priority, they are  
>> used in a load-split fashion. A value of 255 means the RLOC MUST  
>> NOT be used."

>> Isn't it more natural to use 0 as the special case value?

> We choose this way to be consistent with other priority values.  
> Namely administrative distance for routing protocols.

Hm, if you make it more like a HSRP priority or local preference then  
0 would be the right  choice.  :-)

Isn't higher == better also in DNS SRV records? I think that would be  
the right thing to copy here.

>> "Note that the destination RLOC address MAY be an anycast address  
>> if the tunnel egress point may be via more than one physical device."

>> This makes me rather uncomfortable, as anycasting could get in the  
>> way of reachability testing. We have room for a number of RLOCs,  
>> why not put in RLOCs for all physical ETRs rather than anycast them?

> Well it provides another of indirection at really no cost. The xTRs  
> at a site now if they are part of an anycast group and should know  
> not to clear a loc-reach-bit for another xTR that has gone down in  
> it's anycast group.

I guess... Wouldn't it be better to leave this open and come up with a  
separate document that details how anycast would work here?

>> Is the "Loc-AFI" field the AFI for an RLOC? Why is this field only  
>> 8 bits while other AFI fields are 16 bits?

> You found a bug, I will fix it. Thanks. Since I don't have an extra  
> bit for the R-bit, how about if we do this:

>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
> +-+
>      /|    Priority   |    Weight     |R|         Loc- 
> AFI             |
>   Loc +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
> +-+
>      \|                              
> Locator                           |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
> +-+

> But since we won't use all the AFIs that are encoded, a byte should  
> be sufficient. Please comment if you think we should stick with 16- 
> bits or compress all occurences of AFI to 8 bits.

Aren't multiprotocol BGP AFIs 16 bits? In that case, it saves IANA  
from creating another registry for something that they already  
register so if 16 bits is no imposition then that would be the natural  
choice. But if you need the bits for something else, then let IANA  
work a bit harder for their money... Obviously, they should all be the  
same size at least in the context of LISP.

> We enumerated the methods, we didn't say you should use all of them.  
> We recommend using only the loc-reach-bits.

That would be a good example of something I'd like to see moved to  
another document.

> Well if an ITR didn't have any data to know if the destination site  
> was LISP-capable and encapsulated the packet by copying the inner DA  
> to the outer DA, and the site was using PA addressing, the packet  
> would enter the destination site and travel to the host. The host  
> would not recognize a destination port of 4341, so it would respond  
> with a port unreachable.

> But we would not do this anymore with the advent of the lisp- 
> interworking draft.

That's good, because it doesn't make much sense to me to tunnel  
packets to destinations for which you don't know if they support the  
tunneling, even if we ignore for a moment how you would discover the  
non-existant RLOC in that case.

>> So basically the use of mapping requests/replies is the only  
>> reliable (implicit) reachability detection mechanism. However, it  
>> is largely unspecified how ITRs should use this mechanism to  
>> determine reachability.

> That and receiving data packets from the site.

Sometimes traffic only flows in one direction. Although this is rare  
for long periods unless you count asymmetric traffic flow, it's much  
more common for short times when a session goes from active to idle.

>> "perform a route-returnability check"

>> Return routability check?

> A simple anti-spoofing technique.

Yes, but I think you used a less common ordering.

[MTU issue]

> I guess we have had enough discussion on this.  ;-)

Well, let me put it this way: I'll gladly forego more discussion in  
lieu of more consensus. Unfortunately, most people haven't spoken out  
in favor of an approach towards the MTU thing.

>> In my opinion, it would be beneficial to remove pretty much all of  
>> the text that doesn't pertain to actual LISP operation from this  
>> draft, and move that which is still useful (some of it is a bit  
>> stale after five iterations) to a new "LISP architecture" document.

> Can you list what is in this document doesn't pertain to LISP  
> operation?

I have a long list of other documents that I should review at some  
point, so I don't want to go over the LISP draft again at this time.  
If you agree with my suggestion, just keep your eye open for text that  
qualifies on the next iteration of the document. If you don't, don't.

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg