Subject: [BEHAVE] draft-xu-behave-nat-state-sync-00

From: Brian E Carpenter <[email-masked]>
Date: Fri, 13 Nov 2009 14:55:19 +1300

My question about this draft is whether there is available code
and implementation experience with SCSP, which was defined in 1998.

If there isn't code and experience, since it is a quite complex
design, I would be a bit worried.

On the other hand, I believe that something of the complexity
of SCSP is absolutely required to provide reliable synchronisation.
There is no simple, lightweight way to do this reliably.

    Brian

---

From: xuxiaohu 41208 <[email-masked]>
Date: Fri, 13 Nov 2009 11:16:14 +0900

----- 原邮件 -----
发件人: Brian E Carpenter <[email-masked]>
日期: 星期五, 十一月 13日, 2009 上午10:57
主题: [BEHAVE] draft-xu-behave-nat-state-sync-00
收件人: [email-masked]

> design, I would be a bit worried.

Reasonable. We are now scheduled to implement it. Once the experiment is done, we will give a report to Behave WG.

> There is no simple, lightweight way to do this reliably.

Absolutely agree.

Xiaohu

> _______________________________________________
> Behave mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/behave
>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 13 Nov 2009 03:25:22 +0100

> Reasonable. We are now scheduled to implement it. Once the experiment is done, we will give a report to Behave WG.
I understand what Brian is asking is that since the protocol has been
defined over more than 10 years now, whehter there are implementations
out there. If not, then this may mean something.

Regards, marcelo

---

From: Dean Cheng <[email-masked]>
Date: Thu, 12 Nov 2009 18:25:45 -0800

Brian,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
Of
> Brian E Carpenter
> Sent: Thursday, November 12, 2009 5:55 PM
> To: [email-masked]
> Subject: [BEHAVE] draft-xu-behave-nat-state-sync-00
> 

I don't know personally whether there is any available code, but
RFC2335 and RFC2443 documented two separate applications using SCSP,
although also about 10 years ago.

However, SCSP (as mentioned in my presentation) itself heavily 
"borrowed" link-state based algorithm and mechanisms from OSPF/ISIS
(also defined in 90's or earlier) that have been widely deployed since.

> There is no simple, lightweight way to do this reliably.

Totally agreed.

Dean

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 13 Nov 2009 03:28:40 +0100

> although also about 10 years ago.

Right and the question is: are there any implementations of the SCSP 
protocol?

Regards, marcelo

---

From: Dean Cheng <[email-masked]>
Date: Thu, 12 Nov 2009 18:52:31 -0800

Of
> marcelo bagnulo braun
> Sent: Thursday, November 12, 2009 6:29 PM
> To: Dean Cheng
> Cc: [email-masked]
> Subject: Re: [BEHAVE] draft-xu-behave-nat-state-sync-00
> 
> Dean Cheng escribió:
> > Brian,
> >
> >

I don’t know any at this time.

Dean

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 13 Nov 2009 17:12:45 +1300

It seems there was SCSP code in BSD some years ago, as part of the
ATM support:

http://www.freebsd.org/releases/3.0R/notes.html

Also for Linux, it seems to be tied to ATM:

http://www.digipedia.pl/man/doc/view/scspd.8.html

I also found an MSc thesis discussing SCSP in SunOS 5.6.

Regards
   Brian

---

From: <[email-masked]>
Date: Fri, 13 Nov 2009 07:40:44 +0100

Dear all,

I guess that the question should be asked priori to yours:

Do we let vendors define their proprietary solutions or does the IETF define a solution based on standardised protocols to achieve reliable state synchronisation?

>From a service provider perspective, I'd like to see a solution with IETF stamp so as to be included in our RFPs/analysis. Vendors are then free to propose more reliable solutions, if any, compared to the one standardised by IETF. 

Cheers,
Med

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de Brian E Carpenter
Envoyé : vendredi 13 novembre 2009 02:55
À : [email-masked]
Objet : [BEHAVE] draft-xu-behave-nat-state-sync-00

My question about this draft is whether there is available code and implementation experience with SCSP, which was defined in 1998.

If there isn't code and experience, since it is a quite complex design, I would be a bit worried.

On the other hand, I believe that something of the complexity of SCSP is absolutely required to provide reliable synchronisation.
There is no simple, lightweight way to do this reliably.

    Brian

*********************************
This message and any attachments (the "message") are confidential and intended solely for the addressees. 
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration. 
France Telecom Group shall not be liable for the message if altered, changed or falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender.
********************************

---

From: Jan Melen <[email-masked]>
Date: Sun, 15 Nov 2009 16:37:45 +0200

Hi all,

If we are planning to standardize something should we also take a look  
at existing open source solutions that are not based on SCSP such as  
OpenBSD pfsync http://www.openbsd.org/cgi-bin/man.cgi?query=pfsync&sektion=4 
   which is used pretty widely at least in the BSD community for nat(/ 
fw) state synchronization?

   Regards,
     Jan

On Nov 13, 2009, at 8:40 AM, <[email-masked]> <[email-masked] 
 > wrote:

> Do we let vendors define their proprietary solutions or does the  
> IETF define a solution based on standardised protocols to achieve  
> reliable state synchronisation?
>
> From a service provider perspective, I'd like to see a solution with  
> IETF stamp so as to be included in our RFPs/analysis. Vendors are  
> then free to propose more reliable solutions, if any, compared to  
> the one standardised by IETF.
>
> Cheers,
> Med
>
>
> My question about this draft is whether there is available code and  
> implementation experience with SCSP, which was defined in 1998.
>
> On the other hand, I believe that something of the complexity of  
> SCSP is absolutely required to provide reliable synchronisation.

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 23 Nov 2009 11:51:07 +0800

Hi Jan,

Sorry for late response. Thanks a lot for your following information.

Could you or anybody else provide me some protocol specification for the pfsync?

Xiaohu

> -----邮件原件-----
> 发件人: [email-masked] [mailto:[email-masked]] 代表 Jan
> Melen
> 发送时间: 2009年11月15日 22:38
> 收件人: <[email-masked]>
> 抄送: [email-masked]
> 主题: Re: [BEHAVE] draft-xu-behave-nat-state-sync-00
> 
> Hi all,
> 
> OpenBSD pfsync
> http://www.openbsd.org/cgi-bin/man.cgi?query=pfsync&sektion=4

---

From: Dean Cheng <[email-masked]>
Date: Mon, 23 Nov 2009 11:19:34 -0800

Hi Jan,

Do you know whether there exists a protocol specification
for pfsync somewhere so we can take a look?

Thanks
Dean

Of
> Jan Melen
> Sent: Sunday, November 15, 2009 6:38 AM
> To: <[email-masked]>
> Cc: [email-masked]

---

From: Xu Xiaohu <[email-masked]>
Date: Wed, 25 Nov 2009 18:05:05 +0800

> -----邮件原件-----
> 发件人: [email-masked] [mailto:[email-masked]] 代表
> [email-masked]
> 发送时间: 2009年11月13日 14:41
> 收件人: Brian E Carpenter; [email-masked]
> Do we let vendors define their proprietary solutions or does the IETF define
> a solution based on standardised protocols to achieve reliable state
> synchronisation?

For a small enterprise network, maybe it's acceptable to deploy two or more NAT boxes purchased from the same vendor for redundancy. However, for a large ISP network or large enterprise network, it is not reliable enough. For example, an abnormal packet will cause the router OS to crash, it is not absolutely acceptable. Hence I believe the standardization of NAT redundancy is necessary.

Xiaohu

> From a service provider perspective, I'd like to see a solution with IETF stamp
> so as to be included in our RFPs/analysis. Vendors are then free to propose
> more reliable solutions, if any, compared to the one standardised by IETF.
> 
> Cheers,
> Med
> 
> 
> On the other hand, I believe that something of the complexity of SCSP is
> absolutely required to provide reliable synchronisation.

---

From: Simon Perreault <[email-masked]>
Date: Wed, 25 Nov 2009 08:17:39 -0500

Here's what Ryan McBride had to say about this:

> There is no formal protocol description.  It's been asked for before,
> but I've never been really interested in producing such a document.
> Several people have offered to write it up as a draft, and I'm willing
> to help review and edit a draft that someone else puts together, but
> somehow people never manage to produce anything for me to look at :-)
> 
> This links below are probably the best description of the protocol
> as it currently exists:
> 
> http://undeadly.org/cgi?action=article&sid=20090220014805
> http://undeadly.org/cgi?action=article&sid=20090222155558
> http://undeadly.org/cgi?action=article&sid=20090226064146
> http://undeadly.org/cgi?action=article&sid=20090301211402
> 
> Anyone designing a sync protocol should look at the discussion in these
> pages, because we've done a lot of learning with pfsync over
> the years, and what dlg@ has done recently is fantastic.
> 
> However, while the PFSYNC protocol itself may be a canditate for
> standardization, the message payload layout is most certainly not.
> They're tightly tied to the implementation of PF, so unless you simply
> define the messages to be sent and left the content up to the
> implementation, it's going to be an ill fit for other code. The protocol
> is pretty flexible about message types though, so it would be possible
> to specify a bare-minimum "required" message, and allow space for
> vendor-specific messages.

Simon
-- 
DNS64 open-source   --> http://ecdysis.viagenie.ca
STUN/TURN server    --> http://numb.viagenie.ca
vCard 4.0           --> http://www.vcarddav.org

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 25 Nov 2009 09:50:25 -0800

> For a small enterprise network, maybe it's acceptable to deploy two or more NAT boxes purchased from the same vendor for redundancy. However, for a large ISP network or large enterprise network, it…

In the large scale NAT44 we run today, all vendors have 1+1
proprietary state sync.  They also sync configuration and other OAM
elements over this sync channel.  I do not think it is at all required
for vendors to have a standard state sync.  If I deploy multiple
vendors for NAT, i will keep the 1+1 pairs of the same vendor and use
different vendor 1+1 pairs for higher level network topology driven
redundancy, not local state synchronization.

Cameron Byrne
Principal Engineer
T-Mobile USA

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 25 Nov 2009 12:57:24 -0500

If the boxes are from different vendors or even different platforms from the
same vendor, how to deal with issues such as different memory footprint for
NAT state, different NAT implementations, different processor speeds, etc?

Most NAT redundancy schemes are 1+1 because the implementation specific
issues above preclude a heterogeneous solution.

Thanks,

Reinaldo

On 11/25/09 2:05 AM, "Xu Xiaohu" <[email-masked]> wrote:

> 发
> 送时间: 2009年11月13日 14:41
> 主题:
> Re: [BEHAVE] draft-xu-behave-nat-state-sync-00
> 
> 
> Dear all,
> 
> I guess
> that the question should be asked priori to yours:
> 
> Do we let vendors
> define their proprietary solutions or does the IETF define
> a solution based
> on standardised protocols to achieve reliable state
> synchronisation?

For a
> small enterprise network, maybe it's acceptable to deploy two or more NAT
> boxes purchased from the same vendor for redundancy. However, for a large ISP
> network or large enterprise network, it is not reliable enough. For example,
> an abnormal packet will cause the router OS to crash, it is not absolutely
> acceptable. Hence I believe the standardization of NAT redundancy is
> necessary.

Xiaohu

> De : [email-masked] [mailto:[email-masked]]
> De la part de
> Brian E Carpenter
> If there
> isn't code and experience, since it is a quite complex design, I would
> be a
> bit worried.
> 
> Behave
> mailing list
> This message and any attachments (the
> "message") are confidential and intended
> solely for the addressees.
> Any
> unauthorised use or dissemination is prohibited.
> France Telecom Group shall not be liable for the message if
> altered, changed
> or falsified.
> If you are not the intended addressee of
> this message, please cancel it
> immediately and inform the sender.
>

________________________________
> _______________
Behave mailing
> list
[email-masked]

---

From: Jan Melen <[email-masked]>
Date: Wed, 25 Nov 2009 21:57:18 +0200

Hi,

Well I'm not aware of any complete documentation but I'll try to  
capture the main points here. First of all it is a binary protocol and  
in BSD system you can view the state changes by invoking tcpdump for  
device pfsync.  Each packet retrieved on this interface has a header  
associated with it of length PFSYNC_HDRLEN. The header indicates the  
version of the protocol, address family, action taken on the following  
states, and the number of state table entries attached in this packet.  
This structure is defined in <net/if_pfsync.h> as:
struct pfsync_header {
          u_int8_t version;
          u_int8_t af;
          u_int8_t action;
#define PFSYNC_ACT_CLR          0       /* clear all states */
#define PFSYNC_ACT_INS          1       /* insert state */
#define PFSYNC_ACT_UPD          2       /* update state */
#define PFSYNC_ACT_DEL          3       /* delete state */
#define PFSYNC_ACT_UPD_C        4       /* "compressed" state update */
#define PFSYNC_ACT_DEL_C        5       /* "compressed" state delete */
#define PFSYNC_ACT_INS_F        6       /* insert fragment */
#define PFSYNC_ACT_DEL_F        7       /* delete fragments */
#define PFSYNC_ACT_UREQ         8       /* request "uncompressed"  
state */
#define PFSYNC_ACT_BUS          9       /* Bulk Update Status */
#define PFSYNC_ACT_TDB_UPD      10      /* TDB replay counter update */
#define PFSYNC_ACT_MAX          11
          u_int8_t count;
          u_int8_t pf_chksum[PF_MD5_DIGEST_LENGTH];
};

And the state table entries are:
struct pfsync_tdb {
         u_int32_t       spi;
         union sockaddr_union dst;
         u_int32_t       rpl;
         u_int64_t       cur_bytes;
         u_int8_t        sproto;
         u_int8_t        updates;
         u_int8_t        pad[2];
};

struct pfsync_state_upd {
         u_int32_t               id[2];
         struct pfsync_state_peer        src;
         struct pfsync_state_peer        dst;
         u_int32_t               creatorid;
         u_int32_t               expire;
         u_int8_t                timeout;
         u_int8_t                updates;
         u_int8_t                pad[6];
};

struct pfsync_state_del {
         u_int32_t               id[2];
         u_int32_t               creatorid;
         struct {
                 u_int8_t        state;
         } src;
         struct {
                 u_int8_t        state;
         } dst;
         u_int8_t                pad[2];
};

struct pfsync_state_upd_req {
         u_int32_t               id[2];
         u_int32_t               creatorid;
         u_int32_t               pad;
};

struct pfsync_state_clr {
         char                    ifname[IFNAMSIZ];
         u_int32_t               creatorid;
         u_int32_t               pad;
};

struct pfsync_state_bus {
         u_int32_t               creatorid;
         u_int32_t               endtime;
         u_int8_t                status;
#define PFSYNC_BUS_START        1
#define PFSYNC_BUS_END          2
         u_int8_t                pad[7];
};

     Regards,
        Jan

---

From: Simon Perreault <[email-masked]>
Date: Wed, 25 Nov 2009 18:24:15 -0500

> issues above preclude a heterogeneous solution.

I guess the point is that NAT64 is precise enough that a standard state 
sync protocol is possible.

Compare with NAT44, which was never standardized.

Simon

---

From: Dave Thaler <[email-masked]>
Date: Wed, 25 Nov 2009 23:31:38 +0000

Some other references:
http://genie.iitd.ernet.in/wipo/report/node28.html seems to use
it for mailbox sync in another thesis (with WinCE & Linux) in 2004.

http://www.netlab.tkk.fi/tutkimus/ipana/paperit/spiejose2.ppt
discusses an implementation in 1999.

It also looks like Nortel might have IPR in this area:
http://www.google.com/patents/about?id=YKYIAAAAEBAJ
although I don't see anything in the IETF IPR database.

-Dave

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Brian E Carpenter
> Sent: Thursday, November 12, 2009 8:13 PM
> To: Dean Cheng
> Cc: [email-masked]

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 26 Nov 2009 09:53:02 +0800

> 发送时间: 2009年11月26日 1:50
> 收件人: Xu Xiaohu
> 抄送: [email-masked]; [email-masked]
> 主题: Re: [BEHAVE] proprietary implementation v.s standardised protocols//re:
> draft-xu-behave-nat-state-sync-00
> 
> > For a small enterprise network, maybe it's acceptable to deploy two or more
> NAT boxes purchased from the same vendor for redundancy. However, for a large
> ISP network or large enterprise network, it is not reliable enough. For example,
> acceptable. Hence I believe the standardization of NAT redundancy is necessary.
> >
> 

There is a possibility that some abnormal packets could cause the two NAT boxes (using the same OS) to crash simultaneously due to a bug in that OS. I know this is an infrequent case which is acceptable for small enterprise networks. However, I wonder whether this failure is acceptable for all ISPs who deploy large scale NATs?

Xiaohu

> >> De : [email-masked] [mailto:[email-masked]] De la part
> de
> >> Brian E Carpenter
> >> If there isn't code and experience, since it is a quite complex design, I
> would
> >> be a bit worried.
> >>
> >> This message and any attachments (the "message") are confidential and
> intended

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 25 Nov 2009 17:58:06 -0800

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Dave Thaler
> Sent: Wednesday, November 25, 2009 3:32 PM
> To: Brian E Carpenter; Dean Cheng
> Cc: [email-masked]

Cisco apparently, too.  I just asked the inventors to work with Cisco legal to
get an IPR declaration issued if one is necessary.

-d

> > Behalf

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 25 Nov 2009 17:58:17 -0800

> [mailto:[email-masked]] On Behalf Of Simon Perreault
> Sent: Wednesday, November 25, 2009 3:24 PM
> To: [email-masked]
> Subject: Re: [BEHAVE] proprietary implementation v.s 
> standardised protocols //re: draft-xu-behave-nat-state-sync-00
> 
> > Most NAT redundancy schemes are 1+1 because the 
> > implementation specific issues above preclude a 
> > heterogeneous solution.
> 
> I guess the point is that NAT64 is precise enough that a 
> standard state sync protocol is possible.
>

Section 2 of draft-xu-behave-nat-state-sync-00 describes both
NAT44 and NAT64.  So you're suggesting its scope be reduced,
I believe.

-d

---

From: Simon Perreault <[email-masked]>
Date: Wed, 25 Nov 2009 21:00:09 -0500

> NAT44 and NAT64.  So you're suggesting its scope be reduced,

Oh, I missed that.

Yes, definitely. Doing it for NAT44 would be wishful thinking.

Simon

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 25 Nov 2009 18:11:01 -0800

> If the boxes are from different vendors or even different 
> platforms from the same vendor, how to deal with issues 
> such as different memory footprint for NAT state, different 
> NAT implementations, different processor speeds, etc?

If everyone everyone implements NAT44 or NAT64 the same (which they
won't), there will still be differences in pps vs. bandwidth
vs. new-mappings-per-second.

-d

> implementation specific

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 26 Nov 2009 11:53:01 +0800

> -----邮件原件-----
> 发件人: Dan Wing [mailto:[email-masked]]
> 发送时间: 2009年11月26日 10:11
> 收件人: 'Reinaldo Penno'; 'Xu Xiaohu'; [email-masked];
> 'Brian E Carpenter'; [email-masked]
> 主题: RE: [BEHAVE] proprietary implementation v.s standardised protocols //re:

I think the RFP and the testing before purchase could help the customers choose a good match from various NAT boxes of different vendors.

Xiaohu

---

From: Reinaldo Penno <[email-masked]>
Date: Thu, 26 Nov 2009 00:32:03 -0500

> sync protocol is possible.

The proposed protocol as it stands today covers NAT44 as well since NAT44 is
used in a few transition scenarios (DS-Lite and NAT444). And I think that is
why it has some potential.

Having said that, there will always be variation in implementation and the
protocol needs to cater for that by allowing extensions.

> Compare with NAT44, which was never standardized.

Hummm....I would say the original BEHAVE RFCs made great strides in
standardizing NAT44. At this point I would argue the other way around - that
NAT44 behavior is more mature and well-known than NAT64.

---

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 26 Nov 2009 08:42:18 +0100

Xu Xiaohu escribió:
> There is a possibility that some abnormal packets could cause the two NAT boxes (using the same OS) to crash simultaneously due to a bug in that OS.

the counter argument, is that if you have multiple implementation you 
end up with the union of the sets of bugs of all implementations....

regards, marcelo

>  I know this is an infrequent case which is acceptable for small enterprise networks. However, I wonder whether this failure is acceptable for all ISPs who deploy large scale NATs?
>
> Xiaohu
>
>
>   
>>> Xiaohu
>>>
>>>
>>>       
>>>> From a service provider perspective, I'd like to see a solution with IETF
>>>>         
>> stamp
>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Nov 2009 02:52:54 -0500

I am having trouble figuring out what this discussion is about.
We standardize protocols to enable interoperation.
Unless there is something special that requires that distinct devices 
come from the same vendor, we assume that they may come from different 
vendors, and that is why there is value in our work.

Whether a given operator chooses to use multiple vendors, or chooses to 
use one vendor, is a tradeoff made by that (enterprise, edge, or core) 
operator.  One could have (and folks have) exactly the same debate about 
routing protocols.

Yours,
Joel

marcelo bagnulo braun wrote:
>>>>> IETF define
>> There is a possibility that some abnormal packets could cause the two 
>> NAT boxes (using the same OS) to crash simultaneously due to a bug in 
>> that OS.
> 
>>  I know this is an infrequent case which is acceptable for small 
>> enterprise networks. However, I wonder whether this failure is 
>> acceptable for all ISPs who deploy large scale NATs?
>>
>> Xiaohu
>>
>>
>>  
>>>> Xiaohu
>>>>
>>>>
>>>>      
>>>>> design, I
>>>>> SCSP is
>>>>> If you are not the intended addressee of this message, please 
>>>>> cancel it

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 26 Nov 2009 15:56:25 +0800

> -----邮件原件-----
> 发件人: marcelo bagnulo braun [mailto:[email-masked]]
> 发送时间: 2009年11月26日 15:42
> 收件人: Xu Xiaohu
> 抄送: 'Cameron Byrne'; [email-masked]; [email-masked]
> > There is a possibility that some abnormal packets could cause the two NAT
> boxes (using the same OS) to crash simultaneously due to a bug in that OS.
> 

Yes. However, as long as these bugs of different implementations are not triggered by the same condition, it is safe.

There was a famous words, "Don't put all eggs in one basket."

Xiaohu

> >  I know this is an infrequent case which is acceptable for small enterprise
> networks. However, I wonder whether this failure is acceptable for all ISPs
> who deploy large scale NATs?
> >
> > Xiaohu
> >
> >
> >
> >>> Xiaohu
> >>>
> >>>
> >>>
> >>>> France Telecom Group shall not be liable for the message if altered,
> changed
> >>>> or falsified.

---

From: <[email-masked]>
Date: Thu, 26 Nov 2009 09:04:19 +0100

Dear Cameron,

This may be true for 1+1 redundancy schemes, but what about N+1 ones?

Cheers
Med 

-----Message d'origine-----
De : Cameron Byrne [mailto:[email-masked]] 
Envoyé : mercredi 25 novembre 2009 18:50
À : Xu Xiaohu
Cc : BOUCADAIR Mohamed NCPI/NAD/TIP; Brian E Carpenter; [email-masked]
Objet : Re: [BEHAVE] proprietary implementation v.s standardised protocols //re: draft-xu-behave-nat-state-sync-00

Cameron Byrne
Principal Engineer
T-Mobile USA

---

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 26 Nov 2009 09:10:39 +0100

I think we are basically exploring whether it is worth (and if it is 
possible) to standardize a  protocol to synchronize nat state.

The reason for doing that is that as you mention, there is a vlaue in 
allowing multiple boxes from multiple vendors to be used in synch i.e. 
interoperability.

The reasons for not doing so are less obvious i agree.

the main reason that i have encountered in a similar situation is that 
people that implement these boxes ar enot interested in doing so, so if 
we try to do this work, we end up without the required expertise to do 
it right.

You may ask then why vendors are not interested in doing this.
I guess that there are two lines of arguments here. First that in order 
to make a high availability system, the protocol needs to exchange state 
that is closely related to the particular implementation, so as it is 
very implementation dependent, it is not possible to do a standard 
protocol that works as fine as the proprietary one.
So, they claim that is one key differentiating factor, and that they 
don't want to loose it. Well, basically in general vendors preffer their 
own propiertary implementations, i guess. So, the real answer there is 
that it is likely to be because there is not enough pressure from the 
customers for this particular feature. If clients considered that having 
multiple interoperable boxes from different vendors is very important, 
maybe vendors would be more interested in doing this.

So, in summary, i personally don't have a position yet. However, it is 
clear to me that a very bad case is one where we end up with the WG 
chartered to do this and there is no vendor that has actual 
implementations of these boxes working and reviewing this work i.e. we 
don't have the right expertise for this. So, i would be in favour of 
doing this, only if we have a few vendors that are willing to work on this.

Hope it makes sense

Regards, marcelo

Joel M. Halpern escribió:
> Whether a given operator chooses to use multiple vendors, or chooses 
> to use one vendor, is a tradeoff made by that (enterprise, edge, or 
> core) operator.  One could have (and folks have) exactly the same 
> debate about routing protocols.
>
> Yours,
> Joel
>
>>> There is a possibility that some abnormal packets could cause the 
>>> two NAT boxes (using the same OS) to crash simultaneously due to a 
>>> bug in that OS.
>>

---

From: Simon Perreault <[email-masked]>
Date: Thu, 26 Nov 2009 07:52:41 -0500

> The reasons for not doing so are less obvious i agree.

My concern is that with the immense variation in behaviour among current NAT44
implementations, there is a very low chance of success for a standardized NAT44
state synchronization protocol.

By only focusing on NAT64, we would increase our chances.

Simon

---

From: <[email-masked]>
Date: Thu, 26 Nov 2009 14:37:55 +0100

Dear Simon, all,

Please see inline.

Cheers
Med, 

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de Simon Perreault
Envoyé : jeudi 26 novembre 2009 13:53
À : [email-masked]
Objet : Re: [BEHAVE] proprietary implementation v.s standardised protocols//re: draft-xu-behave-nat-state-sync-00

Med: Should be moderated, because we are currently in phase of studying how to introduce
DS-lite CGN (which is NAT44+IPv4-in-IPv6 encapsulation) for instance.

Simon

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 26 Nov 2009 09:43:14 -0800

> This may be true for 1+1 redundancy schemes, but what about N+1 ones?

In other threads i have proposed using multiple  NAT64 / PREF64 pairs
handed out by the DNS64, and having feedback from the NAT64 to the
DNS64 to communicate health of NAT64 box such that the PREF64 can be
taken out of rotation if needed by the DNS64.  This is how global site
load balancing works today for web sites and CDNs.  I do expect this
type of implementation to have NAT64 in 1+1 for local instant fault
tolerance and the DNS64 to be a more macro level recovery and load
sharing technique

If can't get it into the standard, i can get a vendor to do it.  And,
if i can't get a vendor to do it i can probably rig it myself with a
perl script to SNMP poll the NAT64 for health and load and reset the
DNS64 config automatically to redirect traffic from bad PREF64 /
NAT64s to good PREF64 NAT64s

I know the tricks that global site load balancing does are not pure,
but my perpective is that this is the reality of the internet, it is
the only way google, facebook, yahoo can scale.  It is the only way
NAT64 can scale linearly as a system of hardware building blocks and
survive at a macro level.

Cameron

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 26 Nov 2009 10:03:58 -0800

> This may be true for 1+1 redundancy schemes, but what about N+1 ones?

One more point, macro-level state sync is not solvable problem.  In
today's world of RSS and AJAX, the session growth is in many short
sessions.  Sessions that are so short they  last less than 1 second.
Even the locally connected state sync cannot really keep up with
pushing the state table (millions of connections) between a pair of
NAT boxes in real time.  State-sync will be much less effective across
a WAN, and much much less effective in 1 + N or N +N in a WAN.  The
vendors i have talked to remain cautious about their ability to scale
1+1 without impacting overall system capacity, and when i say i am
concerned about performance they tell me to turn off state sync.

State sync is a very large implementation challenge for vendors, the
specific challenges are box specific, and i do not think the IETF can
meaningful improve this situation with yet another draft / protocol.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Nov 2009 13:39:00 -0500

While it is true that state sync can not keep up with everything.  (As 
Cameron says, if the state changes faster than the synchronization, it 
just won't be synchronized.)
That does not mean that we should not address synchronization.  It just 
means we have to be clear about what we canaccomplish.  Part of the 
reason this is, to my mind, acceptable is that the same connections that 
are too fast for the synchronization are the ones that don't need it. 
They can cope with reestablishing.  It is the longer-lived 
communications which benefit from synchrony and which can be supported 
by it.

Given that it is valuable to support synchrony, it seems sensible to 
standardize it.

With regard to NAT44, I can see the argument that there are too many 
variations to handle everything.  So lets use the taxonomy and at least 
include the large piece that easily and naturally fits.  (Namely, full 
cone.)  Lets document what we have, and not go crazy trying to cover all 
possible cases.

Again, the point is to define something that improves interoperability 
for common, useful, cases.  There seems to be a fairly large, and easily 
addressed, space there.  And we can address it in a fashion that is not 
dependent upon whether it is 1+1, N+1, or N load-sharing.

Yours,
Joel

---

From: Dean Cheng <[email-masked]>
Date: Thu, 26 Nov 2009 11:15:33 -0800

Thanks for the information. From the data structure below,
I can roughly tell how the states are maintained (including
updates, timeouts, insertion, deletion, etc.). But what
I'm looking for is the mechanism (the protocol) used between
routers to exchange states, and particularly, with 
reliability for the synchronization. Can you help find
that information?

The proposal we made in this regard is to use SCSP (RFC2334)
where state synchronization is accomplished by a reliable
flooding mechanism, which has been used by all link-state
based routing protocols for years.

Cheers
Dean

Of
> Jan Melen
> Sent: Wednesday, November 25, 2009 11:57 AM
> To: Dean Cheng
> Cc: [email-masked]; [email-masked]
> >>> changed or falsified.

---

From: Christian Huitema <[email-masked]>
Date: Fri, 27 Nov 2009 03:31:21 +0000

> allowing multiple boxes from multiple vendors to be used in synch i.e. 

And the way to resolve that is to make sure that there are at least two vendors interested in developing a standard solution before starting the work. Clearly, Huawei is interested. Is there another major vendor on board?

-- Christian Huitema

---

From: <[email-masked]>
Date: Mon, 30 Nov 2009 13:54:30 +0100

Dear Cameron, 

The issue you are describing is more about load distribution rather than NAT state sync.

I agree with the requirement you have (BTW, similar solutions exist for the selection of the outbound proxy SIP) but this may be easy to implement with stateless NATxy rather than stateful one since the path MUST be symmetric (the same NAT device must be crossed) and appropriate routing actions should be undertaken to redirect the traffic to the appropriate NATxy device. If you have a fully distributed NATxy and all advertise the same IPv4 net, then the symmetry requirement is difficult to meet. Not to mention that the load should be controlled in both sides of the NAT.

Cheers,
Med

-----Message d'origine-----
De : Cameron Byrne [mailto:[email-masked]] 
Envoyé : jeudi 26 novembre 2009 18:43
À : BOUCADAIR Mohamed NCPI/NAD/TIP
Cc : Xu Xiaohu; Brian E Carpenter; [email-masked]
Objet : Re: [BEHAVE] proprietary implementation v.s standardised protocols //re: draft-xu-behave-nat-state-sync-00

Cameron

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 30 Nov 2009 09:19:15 -0800

> The issue you are describing is more about load distribution rather than NAT state sync.

Correct, i believe we can best provide service availability in the
macro network with DNS64 controlled load distribution.  NAT sync in
the macro network faces too many challenges to be useful

> I agree with the requirement you have (BTW, similar solutions exist for the selection of the outbound proxy SIP) but this may be easy to implement with stateless NATxy rather than stateful one sinc…

Unfortunately, i have yet to find an useful application for stateless
NATxy in my network.  The NAT44 i have deployed today require
multiplexing many users behind a small pool public IP address.
Changing the source from IPv4 to IPv6 will face the same limitation
that requires multiplexing many users behind a few IPv4 addresses and
consequently requires a stateful multiplexed solution.

Regarding symmetrical flows, I don't really understand your comment.
I don't support the idea of macro network state sync, and i do see
that people who try to deploy that will have a symmetry issue.

My expectation is that each stateful NAT64 (local 1+1) with a unique
PREF64 will advertise a unique IPv4 prefix, so the flows are always
symmetric.  The entire network has many stateful NAT64 (local 1+1) and
the translation load and resiliency is solved by distributing these
autonomous building blocks of capacity and availability throughout the
network.

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 01 Dec 2009 12:19:20 +0800

> 发送时间: 2009年12月1日 1:19
> 收件人: [email-masked]
> 抄送: [email-masked]; Xu Xiaohu

Do you want to use DNS64 to achieve NAT redundancy while realizing load-balancing? Do you mean offering the IPv6 host another prefix64 by DNS64 if the NAT box for the previous prefix crashes?

Xiaohu

> > I agree with the requirement you have (BTW, similar solutions exist for the
> selection of the outbound proxy SIP) but this may be easy to implement with
> stateless NATxy rather than stateful one since the path MUST be symmetric (the
> same NAT device must be crossed) and appropriate routing actions should be
> undertaken to redirect the traffic to the appropriate NATxy device. If you have
> a fully distributed NATxy and all advertise the same IPv4 net, then the symmetry
> requirement is difficult to meet. Not to mention that the load should be
> controlled in both sides of the NAT.
> >
> 
> >>>> If there isn't code and experience, since it is a quite complex design,
> I would
> >>>> be a bit worried.
> >>>>
> intended solely for the addressees.
> intended solely for the addressees.

---

From: <[email-masked]>
Date: Tue, 1 Dec 2009 08:18:29 +0100

Dear Cameron, 

Please see inline.

Cheers,
Med 

-----Message d'origine-----
De : Cameron Byrne [mailto:[email-masked]] 
Envoyé : lundi 30 novembre 2009 18:19
À : BOUCADAIR Mohamed NCPI/NAD/TIP
Cc : Xu Xiaohu; Brian E Carpenter; [email-masked]
Objet : Re: [BEHAVE] proprietary implementation v.s standardised protocols //re: draft-xu-behave-nat-state-sync-00

Med: Good point. I fully agree that 1:1 mapping can be used only if we have sufficient IPv4 addresses. Otherwise, shared IPv4 address (e.g., stateful NAT64, stateless a+p, etc.) should be used instead. 

Med: The comment is that for stateful mode, the same NAT device should be used for both flow directions otherwise communications will fail.
This requirement is difficult to implement if all NAT devices "represent" the same IPv4 address pool and a distributed mode is adopted (not located in the same zone/cluster).

symmetric.

Med: With these conditions, the symmetry requirement is met.

The entire network has many stateful NAT64 (local 1+1) and
the translation load and resiliency is solved by distributing these
autonomous building blocks of capacity and availability throughout the
network.

Med: This leads to 2*N model and not N+1. N+1 model can easily be deployed owing to 
(1) appropriate setting of routing weights
(2) all static mapping will be backuped
(3) states with short duration are not backuped. Only states with "long" lifetime will be secured.  

Does this model make sense?

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 30 Nov 2009 23:32:29 -0800

> Do you want to use DNS64 to achieve NAT redundancy while realizing load-balancing? Do you mean offering the IPv6 host another prefix64 by DNS64 if the NAT box for the previous prefix crashes?

Yes, I want DNS64 to facilitate NAT64 redundancy from a macro-network
perspective while also providing load-sharing. Load sharing and
redundancy are frequently coupled (web server farms, sip proxies,
anycast DNS servers ...).  This solutions is only focused on AAAA
query responses to an IPv6-only host from a DNS64, which in my
use-case is the most common scenario.  Other use-cases do not interest
me as much and may better fit a well know anycast prefix64.

Cameron