Subject: RE: Few questions on L2TP..

From: Keyur Parikh <[email-masked]>
Date: Fri, 16 Mar 2001 15:09:35 -0500

I think there is a general consensus that the RFC should be extended for
this support. I am ok with a explicit signaling message, but I don't agree
with #1 and #2 solutions proposed by Mark.  Here are couple of hybrid
proposal.

1. (if protocol version number can be incremented)
-The common header parameter for this message such as: tunnel id, Ns, Nr
should be set 0.
-make a new message type = sync request
-create new AVP -  sync info (this may be hidden for authentication)
- the new avp should contain following:
	- indication to reset all tunnels attached to this endpoint OR
	- a specific list of tunnels to reset to attached to this endpoint.
 the receiving end should reset the sequence numbers for the control channel
and the all the associated data sessions (if sequencing enabled) of each
specified tunnel.  The receiving end should format a sync response message
with sync info. The sync info should contain the list of tunnels which were
reset or an indication that all tunnels between the peers are reset.

2. if protocol version needs to stay the same.

- the requester sends SCCRQ with the AVP - sync info.  The M bit in the AVP
must be 1.
- The responder, if upgraded with the change should send StopCCN as a
response to the SCCRQ and include the AVP-sync info containing a list of
tunnel which were reset or an indication that all the tunnels between the
peers were reset.
- if the responder is an older version, then the responder will terminate
the tunnel without the new AVP. In this case, the requester falls back to
the original implementation (which may be re-establishing the tunnel)

In both cases, the requester will discard any packets from peer until a
response is received. The discarded requests will get re-transmitted, once
the peers enter normal operation.

regards,
keyur

-----Original Message-----
From: Andy Koscinski [mailto:[email-masked]]
Sent: Friday, March 16, 2001 2:03 AM
To: Vipin Jain; 'W. Mark Townsley'
Cc: 'Ignacio Goyret'; 'Keyur Parikh'; Andy Koscinski; [email-masked]
Subject: RE: Few questions on L2TP..

Gentelmen:

my comments inline with "andyk"

> -----Original Message-----
> From: Vipin Jain [mailto:[email-masked]]
> Sent: Wednesday, March 14, 2001 10:07 PM
> To: 'W. Mark Townsley'; Vipin Jain
> Cc: 'Ignacio Goyret'; 'Keyur Parikh'; Andy Koscinski; [email-masked]
> Subject: RE: Few questions on L2TP..
> 
> 
> Mark,
> 
> comments inline.. with <<vipin>>
> 
> -----Original Message-----
> From: W. Mark Townsley [mailto:[email-masked]]
> Sent: Wednesday, March 14, 2001 8:38 PM
> To: Vipin Jain
> Cc: 'Ignacio Goyret'; 'Keyur Parikh'; Andy Koscinski; [email-masked]
> Subject: Re: Few questions on L2TP..
> 
> 
> 
> Vipin Jain wrote:
> > 
> > Ignacio, Mark:
> > 
> > I have commented on your ideas in following with <<vipin>> tags..
> > 
> > -----Original Message-----
> > From: Ignacio Goyret [mailto:[email-masked]]
> > Sent: Wednesday, March 14, 2001 10:19 AM
> > To: W. Mark Townsley
> > Cc: Vipin Jain; 'Keyur Parikh'; Andy Koscinski; [email-masked]
> > Subject: Re: Few questions on L2TP..
> > 
> > I normally hate jumping into a heated discussion, but here 
> are my $.02:
> > 
> > Abusing Ns=0 or adding a "reset" bit is just a very bad 
> idea. It makes
> > L2TP so much more vulnerable to attacks. It also negates any
> authentication
> > that may have been negotiated between the two endpoints: 
> how can I verify
> > that the request to resync is from a bona fide source?
> > 
> > <<vipin>>
> > Regarding security, as keyur also pointed out, how do you 
> stop a person
> > from sending a StopCCN on a tunnel; In other words, you are 
> advocating
> > something like "password digest" to authenticate every 
> message coming from
> > peer,
> > like how radius does it.
> > <<vipin>>
> 
> A StopCCN attack is a Denial of Service attack. Even so, in a blind
> situation you have to guess your peer's tunnel ID and the 
> proper Ns for
> the tunnel. The "reset bit" proposal negates the second item here,
> reducing the values one has to guess down to just 16 bits.
> 
> <<vipin>>
> agree that guess work is more when somebody wants to attack 
> using StopCCN.
> But snooping just one valid control packet makes it equally easier.
> Nevertheless, here is a better suggestion:
> Why not include challenge response AVP, which was used in 
> SCCRP or SCCN,
> while sending this message with "reset bit"..
> <<vipin>>

andyk> agreed, the security was a problem, but I find this a good idea.

> 
> > 
> > Finally, it is a hack that has been demonstrated to fail 
> (guaranteed!) in
> > some cases.

andyk> I disagree with Ignacio. how did you demonstrate, it fails?

> > 
> > <<vipin>>
> > Not really, if we ensure an implementation following today's rfc2661
> > legitimately drops the packet because of one of the following:
> > - the version number of newer packet is different
> > - the message contains an AVP which it doesn't like
> > - the message has some header field which it thinks is invalid).
 
andyk> I agree with vipin here.

> Changing the version number in the header for one new feature is a bit
> drastic. Header bits are precious.
> <<vipin>>
> agree. using version number would be too drastic..
> I was just validating in my mind if it would be feasible at all :-)
> Do you have any thoughts on how it would make today's rfc2661 
> implementation
> drop the packet but still convey the information. Like set L 
> bit to 'off'
> for a control packet?
> <<vipin>>
> 
> > 
> > Where as, a newer implementation of L2TP could interpret it 
> correctly and
> > get something sensible out of it.
> > <<vipin>>
> > 
> > A *MUCH* better solution is to add new commands to the 
> protocol so that
> > the failed end can obtain all the resync information over a 
> _new_ tunnel,
> > as Mark suggested. 

andyk> I think this is not a better, but worse solution.
I doubt it works for more than 500 tunnels at all.
if every tunnel terminates at a different node, this solution
requires the failed node to establish a new synchronization tunnel 
for every tunnel which existed before the fail over.
This creates a big burst of traffic after the failure and therefore
dramatically reduces the scalability of this solution. 

> > Whether the remote end supports or not these new
> > commands can easily be determined by a judicious use of the M-bit or
> > by using a different number in AVP 2, or even with a new AVP (resync
> > capable)
> > added to the SCCRQ and SCCRP. This is a significantly more 
> robust design
> > since it will allow you to select different strategies 
> depending on the
> > capabilities of the remote endpoint.
> > 
> > <<vipin>>
> > Relying peer to give all information over a new tunnel has following
> > problems:
> > - it invites a significant amount of traffic during resync 
> and probably
> > that's when you would want to avoid all un-necessary traffic flowing
> between
> > you and peer as there is already a dozen of other things 
> going on when a
> > failover occurs.

andyk> I think you've got a point, vipin. It's not only the L2TP who
wants to resync. all modules want to resync as quickly as possible.

> 
> Let's see.
> 
> First, assume that there is only one tunnel between a given 
> LAC and LNS:
> 
> - If you consider a hot standby case, the control connection could
> already be setup between the standby and the peer, waiting for a state
> update in case of a failover. One control connection should 
> not take up
> considerable resource, and you peer has the bonus of knowing that your
> standby is ready and waiting in the wings.
> 
> - If you consider a restoration after reboot case, sending 3 
> packets for
> tunnel setup is likely insignificant to the time it took to reload the
> box.

andyk> please think about thousands of tunnels and how to minimize
the traffic after the fail over. while the failed node is busy establishing
the new sync tunnels, it has to respond (time is limited) to messages 
on existing tunnels. the responses to
to the existing tunnels will compete for resources with the creation
of the new tunnels. Should the responses to messages arriving on existing
tunnels have priority over generating new tunnels? otherwise we
might end up with many new tunnels, but could loose old, good tunnels
we wanted to keep up in the first place.

> <<vipin>>
> I thought Ignacio mentioned downloading whole databases of 
> sessions and
> tunnels over from peer. hence..
> <<vipin>>
> 
> As a more pathological example, suppose I have 2000 tunnels between an
> LAC and an LNS and one side goes down. With the "reset bit" method, I
> have to send 2000 packets of some sort with the reset bit to resync.
> With the #2 proposal I sent earlier, I bring up one control connection
> and can obtain whatever state information I wish to compact within a
> single control message exchange, potentiall one message for all 2000
> tunnels (Ignacio pointed out to me that RFC 2661 removed the 1500 byte
> control message limitation somewhere along the way, so if you 
> don't mind
> your control messages potentially being fragmented by IP, 
> then you could
> packet one message chock full of whatever you wish... I still think it
> is probably prudent to keep it under 1500 in separate 
> messages though).

andyk> IMO 2000 tunnels and more is the scale to think about.
maybe it should work for all 64k tunnels possible? 

> 
> > - more serious problems like peer can propagate incorrect 
> information to
> > you.
> 
> If your peer is going to lie to you then you have much bigger problems
> at hand.
> 
> > 
> > Asking peer to restart sequencing (using "resync bit" was 
> just one way to
> do
> > it) is much more cleaner.
> > <<vipin>>
> > 
> > If you were to implement something like the Ns=0 hack, I 
> think you have a
> > duty to warn your customers that your box is not 
> 2661-compliant and that
> > this non-compliance may lead to unexplained behavior and failures.
> > <<vipin>>
> > As of today, nobody should claim it will be 2661 compliant..

andyk> any proprietary extension of the rfc 2661 with the resync signal
(Ns=0, Nr=0 or any other explicit resync request signalling), 
does not claim to be compliant with the rfc 2661. I think this is obvious.
Such an extension serves the purpose to demonstrate, that
a scalable hot standby concept without keeping full state can be implemented
with
a lesser effort, than a full state hot standby. IMO it is trivial 
to configure the proprietory extension off, and stay rfc 2661 compliant.

> 
> Interesting accusation.
> 
> > IMHO, that's doesn't mean we should stop trying to work on 
> a cleaner way
> of
> > solving problems.
> 
> Using bits in the header (so far you are suggesting at least two, one
> for the "reset" bit and one for the header version increment) for new
> features that can be just as well served with control 
> messages and AVPs
> is not a good use of the few header bits we have. 
> <<vipin>>
> header version increment is not something I recommend.. that 
> was just a
> thought..
> yes "reset" bit is something I would suggest and the only change.
> <<vipin>>
> 
> 
> > <<vipin>>
> > 
> > thanks,
> > vipin
> > 
> > At 11:50 AM 3/14/01 -0500, W. Mark Townsley wrote:
> > >Vipin Jain wrote:
> > >>
> > >> Mark,
> > >>
> > >> Agree with most of your ideas.
> > >> I wanted to add the following:
> > >> Querying or probing peer about your state is different 
> from asking peer
> > to
> > >> reset sequence number and retransmit control packets 
> which has not been
> > >> acknowledged.
> > >
> > >OK, so you are suggesting that we tell a peer to *start 
> over* at a new
> > >sequence number. Make sense, except that (1) you have to 
> ensure that
> > >there are no local messages that you may have acknowledged 
> but did not
> > >transfer state of to your backup node yet (possible, but 
> if we really
> > >want to talk about scaling then this can be limiting), (2) 
> this requires
> > >your peer to poke down into its transport level unsent queue and
> > >resequence messages (nasty layer violation), (3) this is not at all
> > >extensible, (4) most importantly, this is NOT backwards compatible
> > >(illustrated below).
> > >
> > >> Following can be fairly easily implemented and backward
> > >> compatible as well:
> > >>
> > >> * After fail over/shutdown on LNS (or LAC) when 
> redundant one takes
> over,
> > it
> > >> initiates a packet on tunnel with Ns=0, Nr=0 and "reset 
> sequence bit"
> > set.
> > >> * LAC (or LNS) responds with first acknowledged packet 
> with Ns=0 Nr=0
> and
> > >> "reset sequence bit" set
> > >
> > >An implementation that does not support the reset bit 
> could consider
> > >this a valid "future" packet and simply drop it. Again, 
> run your logic
> > >with Ns=100, Nr=100 and tell me if it works in a backwards 
> compatible
> > >manner. There is NOTHING SPECIAL about Ns=0 and Nr=0 to the L2TP
> > >transport unless the TunnelID=0 as well (the zero TunnelID 
> being the
> > >more important of these fields).
> > >
> > >Further, an implementation MAY update its resend queue with the Nr
> > >present in an out of order packet (see the l2tpbis draft). 
> So, your Nr=0
> > >could end up inadverdantly acknowledging a lot of packets. 
> I know of at
> > >*least* one widely deployed implementation that does this, 
> so get ready
> > >for some hard to figure out bug reports if you deploy this 
> non-standard
> > >extension in a multivendor environment!
> > >
> > >> * LNS (or LAC) responds to it with Ns=0 Nr=0 without 
> "reset sequence
> bit"
> > >> set.
> > >
> > >The following steps are not completely clear to me, but it doesn't
> > >matter as the step above breaks backwards compatibility already.
> > >
> > >> * After having sent a packet with Ns=0 and Nr=0 and 
> "reset sequence
> bit"
> > >> set, if LAC or LNS receives a packet with Ns or Nr with 
> NOT EQUAL to
> ZERO
> > it
> > >> considers it out of sequence.
> > >> * They can then start exchanging control packets as they 
> were before.
> > >
> > >> * Data plane can use the same logic if it wishes.
> > >
> > >There is no concept of "replying" on the data plane, so I 
> don't see how
> > >this same logic can be applied. Also, treating Ns=0 as 
> somehow "special"
> > >on the data plane is as incidious as it is on the control 
> plane. Nr on
> > >the data plane is just a reserved field, you cannot use it 
> for anything.
> > >
> > >If you just want to set an "R-bit" ala the old flow control days to
> > >reset your requence numbers, fine, I guess that would work 
> to signal
> > >your peer to accept an out of order packet (assuming it is even
> > >checking). But, this adds a bit to the data plane and 
> makes the data
> > >header even more complex than it already is. I still think 
> for the data
> > >plane you could just start over at 0 and let the peer drop 
> a couple of
> > >packets (again, assuming it is even checking sequence numbers).
> > >
>

---

From: "W. Mark Townsley" <[email-masked]>
Date: Fri, 16 Mar 2001 16:26:42 -0500

A careful analysis will show that your #2 is the same as my #1 from
earlier except you use an AVP within an SCCRQ and StopCCN to identify
what is essentially a new message, and I just use a new message type.
Note that it has the same security problems I identified in my #1
proposal.

- Mark

---

From: "W. Mark Townsley" <[email-masked]>
Date: Fri, 16 Mar 2001 18:34:07 -0500

> Note that it has the same security problems I identified in my #1

I thought you were suggesting that state information be returned in the
StopCCN vs. back to the "resync" again.

I think "resync" alone is not an extensible solution for full-featured
failover.