Subject: Re: [Softwires] Clarification of the stateles/stateful discussion

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 11:21:32 +0200

On 7/29/11 11:41 AM, Rémi Després wrote:
> Dear all,
>
> Dave rightly expresses in the Softwire meeting the need to separate/clarify discussion about:
> - Stateless vs stateful
> – Static vs dynamic port sets

Does this belong to softwires anymore? New WG maybe?

Cheers, Jan Zorz

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 11:26:20 +0200

On 7/30/11 3:38 PM, Satoru Matsushima wrote:
> On 2011/07/30, at 8:26, Peng Wu wrote:
>>>
>>> a) Stateful+Dynamic port sets: e.g. DS-Lite
>>> b) Stateful+Static port set: e.g. draft-cui-softwire-host-4over6-06
>>> c) Stateless + Static port set: e.g. 4rd, 4via6 translation
>>> d) Stateless + Dynamic port set: ??(Any candidate solution?)
>>
>> I would say that the last one makes no sense
>
> Agree. It is a problem space only on that logical frame.
> I think that the analysis work doesn't need to insist necessity of unfeasible area solution.

Well, A+P got enormous amounts of criticism because there was no dynamic 
allocations of additional ports.

Now we don't need that anymore, just because stateless solution can't 
handle it by design?

I love stateless a+p flavors, but imho we'll need both solutions, 
stateless and statefull and both of them with dynamic port ranges or 
sets, whatever you might call them.

Cheers, Jan Zorz

---

From: Ole Troan <[email-masked]>
Date: Mon, 1 Aug 2011 11:41:58 +0200

> Well, A+P got enormous amounts of criticism because there was no dynamic allocations of additional ports.
> 
> Now we don't need that anymore, just because stateless solution can't handle it by design?
> 
> I love stateless a+p flavors, but imho we'll need both solutions, stateless and statefull and both of them with dynamic port ranges or sets, whatever you might call them.

we can do static allocations for new port ranges.

cheers,
Ole

---

From: Qiong <[email-masked]>
Date: Mon, 1 Aug 2011 18:06:19 +0800

Hi, Satoru,

Yes, the 'draft-cui-softwire-host-4over6' doesn't mention IPv4 address
sharing. But 'lightweight 4over6' has mentioned IPv4 address sharing.
http://tools.ietf.org/id/draft-cui-softwire-b4-translated-ds-lite-01.txt

Thanks

Best wishes

Qiong Sun

On Sat, Jul 30, 2011 at 9:57 PM, Satoru Matsushima <
[email-masked]> wrote:

> One clarification.
>
> On 2011/07/29, at 10:18, GangChen wrote:
>
> > Before making such comparison (of course it should be as fair as
> possible),
> > I think we need to state what solution space we are targeting and what
> > category mode we should take care.
> > If I understand correctly, I would paraphrase as following categories.
> >
> AFAIK, the 'draft-cui-softwire-host-4over6' doesn't mention IPv4 address
> sharing. Is that correct?
>
> cheers,
> --satoru
> _______________________________________________
> Softwires mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/softwires
>

---

From: Satoru Matsushima <[email-masked]>
Date: Mon, 1 Aug 2011 19:59:47 +0900

Hi Qiong,

On 2011/08/01, at 19:06, Qiong wrote:

> Hi, Satoru,
> 
> Yes, the 'draft-cui-softwire-host-4over6' doesn't mention IPv4 address sharing. But 'lightweight 4over6' has mentioned IPv4 address sharing.

Thanks, I got it. Then I think that we should take 'draft-cui-softwire-b4-translated-ds-lite' as a solution for b).

cheers,
--satoru

> 
> Thanks
> 
> Best wishes
> 
> Qiong Sun
> 
> On Sat, Jul 30, 2011 at 9:57 PM, Satoru Matsushima <[email-masked]> wrote:
> One clarification.
> 
> > Before making such comparison (of course it should be as fair as possible),
> AFAIK, the 'draft-cui-softwire-host-4over6' doesn't mention IPv4 address sharing. Is that correct?
> 
> cheers,
> --satoru

---

From: Satoru Matsushima <[email-masked]>
Date: Mon, 1 Aug 2011 20:07:55 +0900

> I love stateless a+p flavors, but imho we'll need both solutions, stateless and statefull and both of them with dynamic port ranges or sets, whatever you might call them.

So my question is that how dynamic is dynamic, and how static is static.
The analogy of dynamic routing is that dynamic for updating routing information for prefixes but forwarding plane is stateless. If you imagine dynamic port ranges within stateless, it sounds like port range aware routing. I think that it would be controversial. 

cheers,
--satoru

---

From: Rémi Després <[email-masked]>
Date: Mon, 1 Aug 2011 14:14:57 +0200

> http://tools.ietf.org/id/draft-cui-softwire-b4-translated-ds-lite-01.txt

Thanks for the correction.

The confusion came from my original proposal (copied below with asterisks added), and carried forward in Gang Chen's answer, sorry for that.
<<
a) Stateful per transport connection (and also stateful per customer IPv6 prefix)
  e.g. DS-lite with CGN
b) Stateful per customer IPv6 prefix (but Stateless per transport connection) 
  e.g. *** draft-cui-softwire-host-4over6-06 
c) Stateless per customer IPv6 prefix (and also stateless per transport connection)
  - Hub-an-spoke    
    TBD
  - Direct CE-CE paths (mesh)
    . Encapsulation based 
      e.g. draft-murakami-softwire-4rd-00 alias
    . Translation based  
      e.g. draft-murakami-softwire-4v6-translation-00

Regards,
RD

---

From: Rémi Després <[email-masked]>
Date: Mon, 1 Aug 2011 14:30:17 +0200

> Well, A+P got enormous amounts of criticism because there was no dynamic allocations of additional ports.

These criticisms are not necessarily well placed.

Note that direct CE-CE paths (which some ISP's WANT to have), are incompatible with dynamic allocations of additional ports.

ISP's that want to offer dynamic allocations can always have these, loosing then the advantage of direct CE-CE paths.

> Now we don't need that anymore, just because stateless solution can't handle it by design?

We don't need it because that keeps networks simple and efficient.
The debate is AFAIK still open.

> I love stateless a+p flavors, but imho we'll need both solutions, stateless and statefull and both of them with dynamic port ranges or sets, whatever you might call them.

Stateless "per customer IPv6 prefix" is incompatible with dynamic port ranges assignable to customers.

Cheers,
RD

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 1 Aug 2011 08:36:57 -0500

Jan,

Interesting enough, the static port-set is one of the reasons why many
find 4v6 being so useful. 

Cheers,
Rajiv

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
On Behalf
> Of Jan Zorz @ go6.si
> Sent: Monday, August 01, 2011 5:26 AM
> To: [email-masked]
> Subject: Re: [Softwires] Clarification of the stateles/stateful
discussion

---

From: Rémi Després <[email-masked]>
Date: Mon, 1 Aug 2011 16:22:06 +0200

Le 1 août 2011 à 15:36, Rajiv Asati (rajiva) a écrit :

> ... Interesting enough, the static port-set is one of the reasons why many

Indeed: operation simplicity, scalability, possible direct CE-CE paths.
Very legitimate.

Regards,
RD

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 16:31:45 +0200

> Indeed: operation simplicity, scalability, possible direct CE-CE paths.

Let me repeat, what was "thrown" at us, when writing ymbk-aplusp:

"what happens, if customer behind CPE that shares IPv4 address runs out 
of ports? Does this mean that all his additional traffic from this point 
does not go anywhere? To end customer, this looks like broken internet 
connectivity."

Don't get me wrong, just repeating what we needed to solve in order to 
move on from that point. Basically, to solve that we needed to introduce 
states at some point. If we can solve this issue on stateless solution, 
I'm all for it.

Cheers, Jan

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 16:36:32 +0200

On 8/1/11 1:07 PM, Satoru Matsushima wrote:
> So my question is that how dynamic is dynamic, and how static is
> static. The analogy of dynamic routing is that dynamic for updating
> routing information for prefixes but forwarding plane is stateless.
> If you imagine dynamic port ranges within stateless, it sounds like
> port range aware routing. I think that it would be controversial.

yes, kind of.

Well, is short words, whatever number of ports you assign in 
port-set/range, end user can exhaust them.

You need then give the user additional ports in order to communicate or 
decide otherwise and send all additional traffic from this point on to 
big evil CGN in the core.

Any idea how to solve this with 4rd and don't use CGN?

Cheers, Jan Zorz

---

From: Simon Perreault <[email-masked]>
Date: Mon, 01 Aug 2011 10:45:13 -0400

Jan Zorz @ go6.si wrote, on 08/01/2011 10:36 AM:
> Well, is short words, whatever number of ports you assign in port-set/range, end
> user can exhaust them.

The fact that most ISPs have been successfully operating with 65536 ports per
subscriber demonstrates that it is possible to statically provision subscribers
with "enough" ports.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 1 Aug 2011 09:52:45 -0500

Jan,

That limitation is valid, and is well applicable to both stateless and stateful NAT.

In fact, many SPs deploying stateful NAT are already carving per-subscriber limit to solve another key problem in which one subscriber can starve other subscribers. Eureka. :-)

So, introducing state doesn't really solve the problem, considering the evolving operational practices.

Cheers,
Rajiv

> From: Jan Zorz @ go6.si [mailto:[email-masked]]
> Sent: Monday, August 01, 2011 10:32 AM
> To: Rémi Després
> Cc: Rajiv Asati (rajiva); [email-masked]
> Subject: Re: [Softwires] Clarification of the stateles/stateful discussion
>

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 1 Aug 2011 09:59:04 -0500

Indeed.

And we keep forgetting an important point that most of us have been
*just* fine with only 1000 (or 2000) ports since that's what most
CPE/home routers in our homes had with NAT44, by default.

Cheers,
Rajiv

On Behalf
> Of Simon Perreault
> Sent: Monday, August 01, 2011 10:45 AM
> To: [email-masked]
discussion
> > Well, is short words, whatever number of ports you assign in
port-set/range,
> end

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 1 Aug 2011 10:04:54 -0500

Jan,

The better solution would be to allocate more ports to the user,
depending on ISP policy.

The CPE/home router can log the exhaustion of the binding and this could
be notified by SNMP (at least in the managed CPE case).

Cheers,
Rajiv

On Behalf
> Of Jan Zorz @ go6.si
> Sent: Monday, August 01, 2011 10:37 AM
> To: Satoru Matsushima
> Cc: [email-masked]
discussion

---

From: Qiong <[email-masked]>
Date: Mon, 1 Aug 2011 23:04:54 +0800

Hi, Jan,

I guess this kind of port exhaustion problem will also exist in dynamic CGN.
Assume the average port number consumed by average subscribers are 2000
(around 32 users per public IPv4 address) and this CGN has 32 concurrent
users at that time with the only public IPv4 address in the address pool,
one can still find port exhaustion when applying for more 2500 concurrent
ports.

So, this is a problem about how to define appropriate port set for our
customers, or to define maximum concurrent subscribers for a given IPv4
address pool. Otherwise, there would either be a waste of resource, or
port exhaustion. Maybe we can even make some more flexible port-set rules
for different time-slot or different types of users. But anyway, this is the
problem we have to face in the further, and I also suggest that
port-consuming applications should upgrade to IPv6 directly.

Best regards

---

From: Rémi Després <[email-masked]>
Date: Mon, 1 Aug 2011 17:05:50 +0200

Le 1 août 2011 à 16:31, Jan Zorz @ go6.si a écrit :

> "what happens, if customer behind CPE that shares IPv4 address runs out of ports? Does this mean that all his additional traffic from this point does not go anywhere? To end customer, this looks like broken internet connectivity."

Thanks for the quotation.
What was "thrown" to you doesn't seem right, and needs to be challenged.

Answer to the question: 
- The customer doesn't run out of ports because it has an exclusive private IPv4 address.
- If the CPE NAT44 runs out of ports, is just does what it does today in the same situation.
In addition, the probability of the CPE running out of IPv4 ports in a dual-stack-service site having 4K ports needs not be higher than in an IPv4-only site having 64K ports.

> Don't get me wrong, just repeating what we needed to solve in order to move on from that point.

I think I don't get you wrong, and actually I believe your pioneer work on the subject has been quite useful.
It is just that what you felt obliged to solve isn't in reality needed in a significant number of legitimate use cases.

> Basically, to solve that we needed to introduce states at some point.
> If we can solve this issue on stateless solution, I'm all for it.

See above.

Cheers,
RD

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 1 Aug 2011 10:11:45 -0500

> subject has been quite useful.

Well said, Remi. +1. 

Cheers,
Rajiv

> From: Rémi Després [mailto:[email-masked]]
> Sent: Monday, August 01, 2011 11:06 AM
> To: Jan Zorz go6.si
> Cc: Softwires-wg; Rajiv Asati (rajiva)
> > "what happens, if customer behind CPE that shares IPv4 address runs out of
> ports? Does this mean that all his additional traffic from this point does not
> go anywhere? To end customer, this looks like broken internet connectivity."
> 
> In addition, the probability of the CPE running out of IPv4 ports in a dual-
> stack-service site having 4K ports needs not be higher than in an IPv4-only
> site having 64K ports.
> 
> 
> It is just that what you felt obliged to solve isn't in reality needed in a
> significant number of legitimate use cases.
> 
>

---

From: Ole Troan <[email-masked]>
Date: Mon, 1 Aug 2011 17:50:14 +0200

> subscriber demonstrates that it is possible to statically provision subscribers

many ISPs have deployed with much less than 64K ports. a typical limitation in CPE's NAT table is just 1024 ports. I've certainly returned CPEs to my ISPs with that limitation in the past.

cheers,
Ole

---

From: Rémi Després <[email-masked]>
Date: Mon, 1 Aug 2011 17:53:14 +0200

Le 1 août 2011 à 17:04, Qiong a écrit :

...
> So, this is a problem about how to define appropriate port set for our customers, or to define maximum concurrent subscribers for a given IPv4 address pool. Otherwise, there would either be a waste of resource, or port exhaustion.

> Maybe we can even make some more flexible port-set rules for different time-slot or different types of users.

There is AFAIK no need to make port-set rules themselves more flexible.

Two customer classes (and tariffs) with standard port-set sizes should IMHO be sufficient:   
- One for exclusive IOPv4 addresses
- One for shared addresses.
Their different port-set sizes are expressed by the difference between lengths of their IPv6 assigned prefixes. 

Three classes would give even more flexibility, but it doesn't seem to me necessary. (Keep it simple, stupid!)

> But anyway, this is the problem we have to face in the further, and

> I also suggest that port-consuming applications should upgrade to IPv6 directly.

Absolutely.
Actually, where native IPv6 addresses are already available, a significant part of the traffic is already IPv6.

Regards,
RD

---

From: Qiong <[email-masked]>
Date: Tue, 2 Aug 2011 00:01:11 +0800

Hi, Remi,

Agree. I only mean that for some extreme use cases where higher priority
user would like to apply for more ports. But in the foreseeable future, I
also think that two customer classes would be enough.

Thanks.

On Mon, Aug 1, 2011 at 11:53 PM, Rémi Després <[email-masked]>wrote:

> address pool. Otherwise, there would either be a waste of resource, or port
> exhaustion.
>

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:11:35 +0000

+1

On 7/30/11 9:26 AM, "Peng Wu" <[email-masked]> wrote:

>Hi Gang,

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:13:52 +0000

You are right. The current 'draft-cui-softwire-host-4over6' describes how
a server is provisioned a public IPv4 address over an IPv6-only network.
IPv4 address sharing wasn't discussed in the draft.

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:16:27 +0000

We didn't have time to discuss this in the WG meeting. We will need to see
how people accept this technology. I wish more discussions will come in
the mailing list and hopefully some operators express interests in it in
the mailing list.

On 8/1/11 6:59 AM, "Satoru Matsushima" <[email-masked]> wrote:

>Thanks, I got it. Then I think that we should take
>'draft-cui-softwire-b4-translated-ds-lite' as a solution for b).
>
>cheers,
>--satoru
>
>
>> 
>> Thanks
>> 
>> Best wishes
>> 
>> Qiong Sun
>>

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:18:06 +0000

In this case, 
http://tools.ietf.org/id/draft-cui-softwire-b4-translated-ds-lite-01.txt
couldn't be controversial because it will turn AFTR a PRR.

On 8/1/11 7:07 AM, "Satoru Matsushima" <[email-masked]> wrote:

>If you imagine dynamic port ranges within stateless, it sounds like port
>range aware routing. I think that it would be controversial.

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:29:52 +0000

Actually, Orange Lab did some tests on A+P. They published the results in
v6ops:

http://tools.ietf.org/html/draft-deng-v6ops-aplusp-experiment-results-01

In their tests, bittorrent seems use most ports (200). Other public apps
use no more than 100 ports. For a family of 5, I think 2000 ports should
be significant at any given time.

On 8/1/11 11:05 AM, "Rémi Després" <[email-masked]> wrote:

>In addition, the probability of the CPE running out of IPv4 ports in a
>dual-stack-service site having 4K ports needs not be higher than in an
>IPv4-only site having 64K ports.

---

From: "Lee, Yiu" <[email-masked]>
Date: Mon, 1 Aug 2011 17:36:09 +0000

Sorry typo: I mean this draft "could" be controversial.

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 22:14:18 +0200

On 8/1/11 5:04 PM, Qiong wrote:
> port exhaustion. Maybe we can even make some more flexible port-set
> rules for different time-slot or different types of users. But anyway,
> this is the problem we have to face in the further, and I also suggest
> that port-consuming applications should upgrade to IPv6 directly.

hi

assigning initial port range to every user and then dynamically 
additional port ranges (or sets) solves this problem. This was solved 
for statefull a+p. stateless still lacks this feature, requested by this 
wg years ago.

cheers, /jan

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 22:19:27 +0200

On 8/1/11 5:05 PM, Rémi Després wrote:
> Thanks for the quotation. What was "thrown" to you doesn't seem
> right, and needs to be challenged.

well, that changed significantly the face of a+p :)

...or maybe that was used to effectively delay the whole process :)

>
> Answer to the question: - The customer doesn't run out of ports
> because it has an exclusive private IPv4 address. - If the CPE NAT44
> runs out of ports, is just does what it does today in the same
> situation. In addition, the probability of the CPE running out of
> IPv4 ports in a dual-stack-service site having 4K ports needs not be
> higher than in an IPv4-only site having 64K ports.

that's defined in "overall architecture" as a+p-light.

> I think I don't get you wrong, and actually I believe your pioneer
> work on the subject has been quite useful. It is just that what you
> felt obliged to solve isn't in reality needed in a significant number
> of legitimate use cases.

thnx.

obligation came from legitimate questions from community.

cheers, Jan

---

From: "Jan Zorz @ go6.si" <[email-masked]>
Date: Mon, 01 Aug 2011 22:21:15 +0200

On 8/1/11 5:50 PM, Ole Troan wrote:
>> The fact that most ISPs have been successfully operating with 65536
>> ports per subscriber demonstrates that it is possible to statically
>> provision subscribers with "enough" ports.
>
> many ISPs have deployed with much less than 64K ports. a typical
> limitation in CPE's NAT table is just 1024 ports. I've certainly
> returned CPEs to my ISPs with that limitation in the past.

+1 :)

I would do the same :)

/jan

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 2 Aug 2011 11:41:45 +0900

>> returned CPEs to my ISPs with that limitation in the past.

Thanks, a clarification has made to clear a confusion of restricted port set/ranges and NAT session table limitation. Even if a CPE is allocated 256 ports, NAT session can be made over 900G sessions in theory. ('2^32'<Full 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) * 2^8<256 ports>.

cheers,
--satoru

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 2 Aug 2011 15:00:27 +0900

On 2011/08/02, at 12:10, Jacni Qin wrote:

> hi Jan,
> 
> On 8/1/2011 10:36 PM, Jan Zorz @ go6.si wrote:
>> Well, is short words, whatever number of ports you assign in port-set/range, end user can exhaust them. 
>> 
>> You need then give the user additional ports in order to communicate or decide otherwise and send all additional traffic from this point on to big evil CGN in the core. 
> No, this hurts. I'd rather choose one then insist on it. ;-)

Ok, Let's get back to analysis work as original discussion.

cheers,
--satoru

---

From: Qiong <[email-masked]>
Date: Tue, 2 Aug 2011 14:11:55 +0800

Hi Lee,

Thank you very much for your interests on 'lightweight 4over6'.

In our consideration, lightweight AFTR is not doing port-range routing. In
this lightweight AFTR, it would firstly lookup a mapping table (recording
[IPv6 address, IPv4 address, Port set]) for a downstream IPv4 packet. Then
after IPv4 packet has been encapsulated into IPv6 packet, it will do IPv6
routing based on different IPv6 addresses. So, lightweight AFTR does not
need to distribute port-set info into FIB and there is no impact on existing
routing architecture between B4 and AFTR.

Actually, the basic idea behind 'lightweight 4over6' is to include
addressing sharing for 'public 4over6', and to enhance DS-Lite for better
scalability. It is a lightweight solution which only keeps customer-based
state in AFTR compared to DS-Lite. And there is also no IPv6 address format
restriction and no IPv6 routing impact on existing network infrastructure.
We have implemented a prototype in our commercial network and find it easy
to deploy in practice.

With regard to so much stateful/stateless discussion, I think actually there
is no solution with totally stateless in the address sharing world. It is
only a matter of fact where to keep the states, either distributed in CPE or
centralized in the core CGN. Given the fact that most existing CPEs can
support NAT already,  our solution is to keep less states in the Core CGN,
while keeping the simplicity of IPv6 addressing and routing.

Sorry to make the discussion even more complicated. Your comments are more
than welcome.

Thanks in advance.

Best wishes

On Tue, Aug 2, 2011 at 1:18 AM, Lee, Yiu <[email-masked]> wrote:

> could be (modified here) controversial because it will turn AFTR a PRR.
>

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 2 Aug 2011 15:23:36 +0900

On 2011/08/02, at 5:14, Jan Zorz @ go6.si wrote:

> assigning initial port range to every user and then dynamically additional port ranges (or sets) solves this problem. This was solved for statefull a+p. stateless still lacks this feature, requested by this wg years ago.

I agree with Qiong.

This stateless/stateless and static/dynamic discussion assumes ipv4 over ipv6 environment. Let see Yahoo, Google and Facebook, have already been on ipv6. We don't need to take care about ports for these sites(hopefully iTunes in near future). I also agree with that it would be more significant to allocate appropriate port sets which are outside of default static port sets for customer demand than ports exhaustion in the v4 over v6 network.

cheers,
--satoru

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 2 Aug 2011 15:33:01 +0900

Hi Yiu,

One clarification.

> be significant at any given time.

Which do you mean '2000' is the number of port you need, or the number of NAT table sessions?

cheers,
--satoru

---

From: Jacni Qin <[email-masked]>
Date: Tue, 02 Aug 2011 11:10:56 +0800

hi Jan,

On 8/1/2011 10:36 PM, Jan Zorz @ go6.si wrote:
> You need then give the user additional ports in order to communicate 
> or decide otherwise and send all additional traffic from this point on 
> to big evil CGN in the core.

Cheers,
Jacni

---

From: Jacni Qin <[email-masked]>
Date: Tue, 02 Aug 2011 11:14:43 +0800

Hi Simon,

> subscriber demonstrates that it is possible to statically provision subscribers
Sorry, I don't catch your point here, could you?

Cheers,
Jacni

---

From: Simon Perreault <[email-masked]>
Date: Tue, 02 Aug 2011 09:24:42 -0400

Satoru Matsushima wrote, on 08/01/2011 10:41 PM:
> Thanks, a clarification has made to clear a confusion of restricted port
> set/ranges and NAT session table limitation. Even if a CPE is allocated 256
> ports, NAT session can be made over 900G sessions in theory. ('2^32'<Full
> 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) * 2^8<256 ports>.

This is only true for endpoint-dependent NATs, which are forbidden by the BEHAVE
RFCs (4787 and 5382). According to these RFCs, NATs MUST have
endpoint-independent mapping behaviour. This means that each NAT session will
consume one external port.

Simon

---

From: Simon Perreault <[email-masked]>
Date: Tue, 02 Aug 2011 09:33:05 -0400

> Sorry, I don't catch your point here, could you?

The original assertion was "there is no X such that X number of ports per user
is enough."

Past experience shows that "X=65536 is enough".

Therefore, the original assertion is false.

:)

Simon

---

From: Simon Perreault <[email-masked]>
Date: Tue, 02 Aug 2011 09:54:37 -0400

> consume one external port.

Sorry, I confused the terminology. Each NAT *binding* will consume one external
port. There may be multiple session to different destinations using the same
external port. The 900G figure is valid, as long as internal hosts reuse the
same source address+port for different destinations.

Simon

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Tue, 2 Aug 2011 17:39:12 -0500

Satoru-san,  

This is an important point that most of us forget that restricting to
"n" ports doesn't equate to just "n" NAT sessions rather many more than
n sessions. We must add that to the 4v6 motivation draft as well as to
the 4v6 comparison draft.

Cheers,
Rajiv

On Behalf
> Of Simon Perreault
> Sent: Tuesday, August 02, 2011 9:55 AM
> To: [email-masked]
discussion
> >> 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) *
2^8<256
> ports>.
> >
> > This is only true for endpoint-dependent NATs, which are forbidden
by the
> BEHAVE
> > endpoint-independent mapping behaviour. This means that each NAT
session
> will

---

From: Jacni Qin <[email-masked]>
Date: Wed, 03 Aug 2011 08:54:59 +0800

hi,

> Therefore, the original assertion is false.

Ok, Thanks.

Cheers,
Jacni

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 10:01:46 +0200

+1

RD

---

From: Simon Perreault <[email-masked]>
Date: Wed, 03 Aug 2011 09:05:56 -0400

>> the 4v6 comparison draft.

I think there is an important point missing from this discussion. It is
tricky but it has important practical consequences.

As I said, "The 900G figure is valid, *as long as internal hosts reuse
the same source address+port for different destinations*."

The "as long as ..." part is important. I don't know of any operating
system that behaves like that. That is, a different source port will be
used for each new outbound session, regardless of the destination.
Therefore, in practice, each session will require one NAT binding, which
will in turn consume one external port.

So the 900G figure is valid *in theory*, but *in practice* we're stuck
with a number of sessions roughly equal to the number of external ports
available on the NAT.

Now, if there is no NAT and the host itself is constrained to a given
port range, the OS will usually start reusing source ports for different
destinations when there is pressure to do so (i.e. when all source ports
are in use). Where a NAT would simply drop the session-creating packet,
in this case the OS is able to reuse a source port. So if there is no
NAT, the 900G figure is also valid in practice.

Simon

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 15:32:41 +0200

Hi, Simon,

See inline.

Le 3 août 2011 à 15:05, Simon Perreault a écrit :

> The "as long as ..." part is important.

Agreed.

> I don't know of any operating
> system that behaves like that.

The point is that this behavior concerns the NAT44 of a CE that supports address sharing (it doesn't concern a PC OS).
By permitting _this_ NAT to do endpoint-dependent mapping for TCP connections, the number of supported connections can be largely increased (if found useful).

Cheers,
RD

> That is, a different source port will be

---

From: Simon Perreault <[email-masked]>
Date: Wed, 03 Aug 2011 09:46:01 -0400

> By permitting _this_ NAT to do endpoint-dependent mapping for TCP connections, the number of supported connections can be largely increased (if found useful).

Agreed. IMHO, a NAT should be allowed to do endpoint-dependent mapping
for protocols that are known not to cause issues (e.g. HTTP and DNS). I
think this would address most scaling concerns people have with port
ranges. This was already discussed in the behave WG by the way...

Simon

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 16:14:02 +0200

> IMHO, a NAT should be allowed to do endpoint-dependent mapping
> for protocols that are known not to cause issues (e.g. HTTP and DNS).

Agreed.

> I
> ranges.

Note that, even without that, expressed concerns on this tread are in general excessive because this thread only concerns port sets used for residual IPv4 traffic where IPv6 has been deployed. Numbers of needed ports will quickly decrease.
 
Also, one should not forget that assigning full IPv4 addresses to DSL customers who need it remains possible with  stateless solutions (presumably at a price, but we know there is no free lunch).

Cheers, 
RD

> This was already discussed in the behave WG by the way...

---

From: GangChen <[email-masked]>
Date: Thu, 4 Aug 2011 00:04:42 +0800

2011/8/3, Rémi Després <[email-masked]>:
>
> Also, one should not forget that assigning full IPv4 addresses to DSL
> customers who need it remains possible with  stateless solutions (presumably
> at a price, but we know there is no free lunch).

In some sense, statelss solution could give such flexibilities to
DYNAMICALLY adjust how many ports are available to a sepecific CE.
Does this belong to static port behaviour or dynamic port behaviour?

BRs

Gang

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 18:42:33 +0200

> Does this belong to static port behaviour or dynamic port behaviour?

If stateless means "without per customer state" in BR's, as it should by default in this discussion (IMHO), no dynamic changes of port-set sizes are practicable.

But this permits IMHO realistic stateless deployments: as  already said on this list, DSL ISP's can propose two tariffs: one for exclusive IPv4 addresses, and a lower one for shared IPv4 addresses with the same number of ports for each.

More flexible approaches are clearly possible, but the price paid for this flexibility is having to maintain per-customer states in BR's, and impossible direct CE-CE paths.

Regards,
RD

---

From: Tetsuya Murakami <[email-masked]>
Date: Wed, 3 Aug 2011 13:44:04 -0700

Hi Simon,

> will in turn consume one external port.

From the developer point of view, almost operating systems such as netbsd, Linux can support the above function.

At first, the NAT function in these operating systems has already supported to specify the port-range. In terms of the port number allocation in the NAT, if there are unused port number in a given port-range, then the NAT function allocates a new port number from the unused port numbers. But if there is no unused port number in a given port-range, then the NAT function tries to reuse a port number which is used for another NAT session with the different destination address.

Hence, as Satoru-san mentioned, the existed operating system such as netbsd, linux, has already reuse the same port number on multiple NAT session with the different destination.

> with a number of sessions roughly equal to the number of external ports

As I mentioned above, the number of NAT session can be greater than the available port number in practice because the NAT function in these operating systems has already supported to reuse a port number which is used for another NAT session with the different destination. So, the 900G figure is valid today. In practice, there are another limitation from the memory size for keeping all NAT session and so the NAT function has a limitation of maximum number of NAT session. But this is totally regardless of the port-range functionality.

Thanks,
Tetsuya Murakami

---

From: Simon Perreault <[email-masked]>
Date: Thu, 04 Aug 2011 08:26:11 -0400

On 2011-08-03 16:44, Tetsuya Murakami wrote:
>> So the 900G figure is valid *in theory*, but *in practice* we're
>> stuck with a number of sessions roughly equal to the number of
>> external ports available on the NAT.
> 
> As I mentioned above, the number of NAT session can be greater than
> the available port number in practice because the NAT function in
> these operating systems has already supported to reuse a port number
> which is used for another NAT session with the different destination.

Yes, because these NATs are endpoint-dependent, which is forbidden by
the BEHAVE RFCs.

> So, the 900G figure is valid today. In practice, there are another
> limitation from the memory size for keeping all NAT session and so
> the NAT function has a limitation of maximum number of NAT session.
> But this is totally regardless of the port-range functionality.

Agreed.

Simon

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 4 Aug 2011 08:01:54 -0700

> Yes, because these NATs are endpoint-dependent, which is forbidden by

It is still very usefull and will be deployed regardless.

I understand you need to keep your documents consistent, but stretching
those ipv4 addresses further is a network and business reality of today's
big nat and future big nat.

Cb

---

From: Simon Perreault <[email-masked]>
Date: Thu, 04 Aug 2011 11:04:31 -0400

> It is still very usefull and will be deployed regardless.

Right. But the IETF needs consistency in the advice it provides.

> those ipv4 addresses further is a network and business reality of
> today's big nat and future big nat.

That's why there's a proposal floating around (no draft) that NATs MAY
apply endpoint-dependent mapping to protocols that are known to not
cause problems (e.g. HTTP and DNS).

Simon

---

From: Tetsuya Murakami <[email-masked]>
Date: Thu, 4 Aug 2011 12:37:46 -0700

Hi Simon,

> Yes, because these NATs are endpoint-dependent, which is forbidden by

If using endpoint-independent mapping, one NAT session can cover multiple destinations. For instance,

Internal : External : Destination
X:x -> E:e -> n/a:n/a

So, if a CPE has a limited set of the port number, the CPE can provide the access to many destination with one NAT session.

If using endpoint-dependent mapping, the same port number can be reused if the destination address is same. For instance,

Internal : External : Destination
X:x -> E:e1 -> Y1:n/a
X:x -> E:e2 -> Y2:n/a

In this case, since the destination is different, 2 NAT sessions can be created.

In terms of NAT implementation on the existed operating systems, the endpoint-independent mapping is not supported. In fact, we added the endpoint-independent mapping function in the existed operating system such as linux, netbsd, For instance, netbsd creates the following NAT sessions.

Internal : External : Destination
X:x -> E:e1 -> Y1:y1
X:x -> E:e2 -> Y1:y2
X:x -> E:e3 -> Y2:y3

If there are unused port number in its port-range, the NAT function allocates a new port number for each NAT session like the above. But if there is no unused port number, the NAT function tries to reuse a same port number which is used for another NAT session with the different destination address like X:x -> E:e1 -> Y4:y4. Hence, from the current implementation point of view, if a CPE has a limited set of the port number, the existed NAT function can provide the access to many destinations.

Thanks,
Tetsuya Murakami

---

From: Reinaldo Penno <[email-masked]>
Date: Thu, 4 Aug 2011 15:58:56 -0400

> cause problems (e.g. HTTP and DNS).

This was discussed in BEHAVE many times. There is a subtle, but yet
important, difference between 'recommending EIM behavior' and having it 'on'
all the time. The current RFCs do not really dwell into that space.

Simon and I will be updating the bis draft to clarify (amongst other new
things) this issue.

https://tools.ietf.org/html/draft-penno-behave-rfc4787-5382-5508-bis-00

---

From: Rémi Després <[email-masked]>
Date: Mon, 8 Aug 2011 07:38:10 +0200

Quiong,

Clarifying what is meant by "stateless" in different contexts is highly desirable.

The objective of the 4rd address mapping is
- no "per customer" state in provider nodes (BR, AFTR...)
- direct customer-customer paths made possible

The 'Lightweight 4over6' proposal has in my understanding a different objective: flexible port sets assignable to each individual customer.
It can therefore coexist with one or both "per-sustomer-stateless" solution(s) (with encapsulation and/or double-translation).

Hope it helps.

Regards,
RD

Le 2 août 2011 à 08:11, Qiong a écrit :

> In our consideration, lightweight AFTR is not doing port-range routing. In this lightweight AFTR, it would firstly lookup a mapping table (recording [IPv6 address, IPv4 address, Port set]) for a downstream IPv4 packet. Then after IPv4 packet has been encapsulated into IPv6 packet, it will do IPv6 routing based on different IPv6 addresses. So, lightweight AFTR does not need to distribute port-set info into FIB and there is no impact on existing routing architecture between B4 and AFTR.
> 
> Actually, the basic idea behind 'lightweight 4over6' is to include addressing sharing for 'public 4over6', and to enhance DS-Lite for better scalability. It is a lightweight solution which only keeps customer-based state in AFTR compared to DS-Lite. And there is also no IPv6 address format restriction and no IPv6 routing impact on existing network infrastructure. We have implemented a prototype in our commercial network and find it easy to deploy in practice. 
> 
> With regard to so much stateful/stateless discussion, I think actually there is no solution with totally stateless in the address sharing world. It is only a matter of fact where to keep the states, either distributed in CPE or centralized in the core CGN. Given the fact that most existing CPEs can support NAT already,  our solution is to keep less states in the Core CGN, while keeping the simplicity of IPv6 addressing and routing.
> 
> Sorry to make the discussion even more complicated. Your comments are more than welcome.
> 
> Thanks in advance.
> 
> Best wishes
> 
>

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 01:10:00 +0000

Hi Satoru,

I think you answered your question. As you said, it is possible to use 1
port (in theory) to reach 2000 sessions as long as each destination is
different. So given user 256 ports could create a much larger set of NAT
sessions in the CGN.

Cheers,
Yiu

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 01:11:38 +0000

Hi Qiong,

I see your point. So what is the difference between a lightweight AFTR and 4rd BR?

Cheers,
Yiu

From: Qiong <[email-masked]<mailto:[email-masked]>>
Date: Tue, 2 Aug 2011 14:11:55 +0800
To: "Yiu L. LEE" <[email-masked]<mailto:[email-masked]>>
Cc: Satoru Matsushima <[email-masked]<mailto:[email-masked]>>, "Jan Zorz @ go6.si" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Softwires] Clarification of the stateles/stateful discussion

---

From: Jacni Qin <[email-masked]>
Date: Tue, 09 Aug 2011 09:23:36 +0800

hi,

If I understand it correctly, a per user address/port mapping table is 
maintained on the LW AFTR, then no session table on it.

Cheers,
Jacni

On 8/9/2011 9:11 AM, Lee, Yiu wrote:
> From: Qiong <[email-masked] <mailto:[email-masked]>>
> Cc: Satoru Matsushima <[email-masked] 
> <mailto:[email-masked]>>, "Jan Zorz @ go6.si" <[email-masked] 
> <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" 
> <[email-masked] <mailto:[email-masked]>>
> In our consideration, lightweight AFTR is not doing port-range 
> routing. In this lightweight AFTR, it would firstly lookup a mapping 
> table (recording [IPv6 address, IPv4 address, Port set]) for a 
> downstream IPv4 packet. Then after IPv4 packet has been encapsulated 
> into IPv6 packet, it will do IPv6 routing based on different IPv6 
> addresses. So, lightweight AFTR does not need to distribute port-set 
> info into FIB and there is no impact on existing routing architecture 
> between B4 and AFTR.
>
>

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 01:25:43 +0000

So the only difference between 4rd BR and lightweight AFTR is just a static mapping rule for all users in the same domain (4rd BR case) and group rule or user specific rule for different users (LW AFTR)?

From: Jacni Qin <[email-masked]<mailto:[email-masked]>>
Date: Tue, 9 Aug 2011 09:23:36 +0800
To: "Yiu L. LEE" <[email-masked]<mailto:[email-masked]>>
Cc: Qiong <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Softwires] Clarification of the stateles/stateful discussion

hi,

If I understand it correctly, a per user address/port mapping table is maintained on the LW AFTR, then no session table on it.

Cheers,
Jacni

Cheers,
Yiu

_______________________________________________
Softwires mailing list
[email-masked]<mailto:[email-masked]>https://www.ietf.org/mailman/listinfo/softwires

---

From: Jacni Qin <[email-masked]>
Date: Tue, 09 Aug 2011 10:23:14 +0800

Re-,

Since the "rules" are "maintained" in different ways, then the 
forwarding behaviors, for incoming traffics and CE-CE etc., are 
different, IMHO.

Cheers,
Jacni

On 8/9/2011 9:25 AM, Lee, Yiu wrote:
> So the only difference between 4rd BR and lightweight AFTR is just a 
> static mapping rule for all users in the same domain (4rd BR case) and 
> group rule or user specific rule for different users (LW AFTR)?
>
> From: Jacni Qin <[email-masked] <mailto:[email-masked]>>
> Cc: Qiong <[email-masked] <mailto:[email-masked]>>, 
> "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
>> <mailto:[email-masked]>>, "Jan Zorz @ go6.si" 
>> <[email-masked] <mailto:[email-masked]>>, "[email-masked] 
>> <mailto:[email-masked]>" <[email-masked] 
>> <mailto:[email-masked]>>
>> [email-masked]://www.ietf.org/mailman/listinfo/softwires

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 02:28:00 +0000

Agree that the CE-CE communication will be possible for LW AFTR because the rules are not store in the CE but in the LW AFTR.

But my main question is both technical are so similar, can we have a session in 4rd to extend this special scenario rather than writing a new draft for it?

Date: Tue, 9 Aug 2011 10:23:14 +0800

Re-,

Since the "rules" are "maintained" in different ways, then the forwarding behaviors, for incoming traffics and CE-CE etc., are different, IMHO.

Cheers,
Jacni

---

From: Jacni Qin <[email-masked]>
Date: Tue, 09 Aug 2011 10:59:04 +0800

hi Yiu,

On 8/9/2011 10:28 AM, Lee, Yiu wrote:
> Agree that the CE-CE communication will be possible for LW AFTR 
> because the rules are not store in the CE but in the LW AFTR.
>
> But my main question is both technical are so similar, can we have a 
> session in 4rd to extend this special scenario rather than writing a 
> new draft for it?

Sorry, I don't follow and please correct me if I made mistakes,
In the LW AFTR approach, the CE gets its port range, and the LW AFTR 
maintain the binding info. per user. The CE-CE communications have to 
pass through LW AFTR. The LW AFTR forwards incoming traffics based on 
the binding table.
In 4rd, the rule is identical within one domain (on both CEs, and BR), 
then the CE can calculate Src/Dst of IPv6 encapsulation and route the 
packets to the other CEs.

Cheers,
Jacni

---

From: "Peng Wu" <[email-masked]>
Date: Tue, 9 Aug 2011 11:02:00 +0800

Hi Yiu and all,
>Agree that the CE-CE communication will be possible for LW AFTR because the rules are not store in the CE but in the LW AFTR.
                      Should be CE--LW AFTR--CE style, is that what you mean here?
>But my main question is both technical are so similar, can we have a session in 4rd to extend this special scenario rather than writing a new draft for it?
                      Not sure about it. The thing is that it's lightweight state while 4rd is stateless. Addressing is different.

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 03:03:55 +0000

Oops. Typo. I mean "won't be possible". Sorry for the confusion.

Date: Tue, 9 Aug 2011 10:59:04 +0800

hi Yiu,

Sorry, I don't follow and please correct me if I made mistakes,
In the LW AFTR approach, the CE gets its port range, and the LW AFTR maintain the binding info. per user. The CE-CE communications have to pass through LW AFTR. The LW AFTR forwards incoming traffics based on the binding table.
In 4rd, the rule is identical within one domain (on both CEs, and BR), then the CE can calculate Src/Dst of IPv6 encapsulation and route the packets to the other CEs.

Cheers,
Jacni

Re-,

Cheers,
Jacni

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 03:54:52 +0000

Couple thoughts.

1. The current draft doesn't specify the static mapping rule like what 4rd
does. So I guess we can't compare this to 4rd.

2. I keep thinking what are the difference of this and PRR. I guess
Qiong's PRR definition is the forwarding decision would be done in the
FIB. I don't disagree with this. My argument is the AFTR will need to use
the port-range to find out the next-hop (the b4's IPv6 address). The
difference between this and the PRR is the information is stored in the
NAT table rather than the routing table. But the general idea is the same.

On 8/8/11 11:02 PM, "Peng Wu" <[email-masked]> wrote:

>Hi Yiu and all,
>>Agree that the CE-CE communication will be possible for LW AFTR because
>>the rules are not store in the CE but in the LW AFTR.
>>session in 4rd to extend this special scenario rather than writing a new
>>draft for it?
>                      Not sure about it. The thing is that it's
>lightweight state while 4rd is stateless. Addressing is different.
>

---

From: "Peng Wu" <[email-masked]>
Date: Tue, 9 Aug 2011 12:04:16 +0800

Hi Yiu,

Agree, both.

---

From: Qiong <[email-masked]>
Date: Tue, 9 Aug 2011 14:43:23 +0800

Hi, Remi,

Thanks for your interests on 'Lightweight 4over6'.  Please see inline.

Clarifying what is meant by "stateless" in different contexts is highly
> desirable.
> The 'Lightweight 4over6' proposal has in my understanding a different
> objective: flexible port sets assignable to each individual customer.
>
[Qiong]: Yes. And in addition, its objective is also to achieve 'lightweight
addressing and routing' with no special requirements on IPv6 address format.

> It can therefore coexist with one or both "per-sustomer-stateless"
> solution(s) (with encapsulation and/or double-translation).
>
[Qiong]: Sorry, I'm not quite clear of 'per-customer-stateless' solution. Is
it for the ones where different customers have different IPv6 prefixes and
then embedded with IPv4 address?
For example:
user1_v6 = user1_prefix+user1_v4+suffix;
user2_v6 = user2_prefix+user2_v4+suffix;
...
where user1_prefix != user2_prefix

Is it right?

Thanks

Best regards.

Qiong Sun

---

From: Qiong <[email-masked]>
Date: Tue, 9 Aug 2011 14:48:30 +0800

Hi Yiu, Jacni,

On Tue, Aug 9, 2011 at 9:25 AM, Lee, Yiu <[email-masked]> wrote:

> static mapping rule for all users in the same domain (4rd BR case) and group
> rule or user specific rule for different users (LW AFTR)?
>

[Qiong]: Agree. And LW AFTR will not expose port-set information in IPv6
address directly.  So it would not introduce any specific requirements on
IPv6 address format. I think this is also a major difference between 4rd and
Lightweight 4over6.

Best regards.

Qiong Sun

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 9 Aug 2011 16:10:40 +0900

Hi Qiong,

in fact, 4rd doesn't require specific format to ipv6 address.
what do you find in 4rd?

--satoru sent from my iPad

On 2011/08/09, at 15:48, Qiong <[email-masked]> wrote:

> [Qiong]: Agree. And LW AFTR will not expose port-set information in IPv6 address directly.  So it would not introduce any specific requirements on IPv6 address format. I think this is also a major difference between 4rd and Lightweight 4over6.
> 
> Best regards.
> 
> Qiong Sun

---

From: Rémi Després <[email-masked]>
Date: Tue, 9 Aug 2011 09:16:48 +0200

Hi, Qiong,

Le 9 août 2011 à 08:43, Qiong a écrit :

> [Qiong]: Yes. And in addition, its objective is also to achieve 'lightweight addressing and routing' with no special requirements on IPv6 address format.  
> 
>  

> [Qiong]: Sorry, I'm not quite clear of 'per-customer-stateless' solution. Is it for the ones where different customers have different IPv6 prefixes and then embedded with IPv4 address?

It simply means that the number parameters of the address mapping algorithm is not related to the number of customer sites.

The number of these parameters generally depends on the number of disjoint IPv4 prefixes that the ISP has to support (typically just a few), while tens of thousands or millions of customers can be supported.

Regards,
RD

---

From: <[email-masked]>
Date: Tue, 9 Aug 2011 23:25:23 +0800

Hello Yiu, Qiong, all,

Interesting discussion, and hope am not jumping too late! I guess I'm still curing my jet lag. ;0

________________________________

	From: Qiong [mailto:[email-masked]] 
	Sent: Tuesday, August 09, 2011 2:49 PM
	To: Lee, Yiu
	Cc: [email-masked]
	Subject: Re: [Softwires] Clarification of the stateles/stateful discussion

	Hi Yiu, Jacni,

	[Qiong]: Agree.  
	 
	Xiaohong: If that's the only difference, don't see a need for two different solutions.
	 
	 And LW AFTR will not expose port-set information in IPv6 address directly.  So it would not introduce any specific requirements on IPv6 address format. I think this is also a major difference between 4rd and Lightweight 4over6. 
	 
	Xiaohong:  Could you explain more about your concern on  "specific requirements on IPv6 address format"? 

	IMHO, I don't see a issue in 4rd IPv6 address format, as 4rd authors have been carefully designing the port allocation algorithm for CE IPv6 prefix, which is efficiently encoded (less than 64 bits) at least to my eyes.  Open to hear oppsite opinions  if you have operation issues with your network.  ;) 

	As a matter of fact, I see a need when a  * incoming port *  request is out side 4rd port set, for example, UPnP 1.0 applications ask for a specific port and try consecutive ports, RTP/RTCP and some FTP client implementations ask for both even and odd ports pair, according to our A+P implementation, and analysis and investigation of it's impact on current practice since 2009.  

	How to deal with incoming port out side of stateless solution is something we have addressed by proposing a new port allocation algorithm and implemented a scattered port allocation A+P, documented in A+P experiments draft:

	https://datatracker.ietf.org/doc/draft-deng-v6ops-aplusp-experiment-results 

	In case you're courious about port consumptions of applications in the IPv4 address sharing context, you may like to have a look at:

	http://opensourcev6transtechnologies.weebly.com/experiments-results.html

	And it's also applicable to DS-Lite scenario, in case a deployment wants to allocate a bulk of incoming ports to be under control of customer, a deployment approach was documented in:

	DS-Lite AFTR NAT Bypass: Co-located B4 and NAT Model

	http://tools.ietf.org/html/draft-zhou-softwire-b4-nat-02 

	And the means of provisioning two parameters for this port allocation is via PCP, and was documented in

	http://tools.ietf.org/html/draft-tsou-pcp-natcoord-03 

	This deployment has been also demoed during IETF 81 PCP demo, and the website for this demo has been moved to (in case you're into ascii drawings ;)) :  

	http://opensourcev6transtechnologies.weebly.com/ietf-81-pcp-demo-site.html <http://opensourcev6transtechnologies.weebly.com/ietf-81-pcp-demo-site.html>  

	Thoughts are very much welcomed!

	P.S. I'm not saying this algorithm is the only way for 4rd to deal with incoming port restriction, and there are certainly ways to design 4rd port allocation while keep its IPv6 prefix encoding efficiently. The point is that there is a reason (current/past-to-be practice) for 4rd to take care of port allocation.

	                     _.._..,___,
	                    (   Cheers! )
	                    ]~,"-.-~     [ 
	                 .=] )  ' (;      ([ 
	                  | ]::   '          [   
	                  '=]):    .)     ([
	                    | : :             |
	                     ~~~-------    

	 Xiaohong

	 

	Best regards.

	Qiong Sun

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 16:36:59 +0000

You go it.

From: <[email-masked]<mailto:[email-masked]>>
Date: Tue, 9 Aug 2011 23:25:23 +0800
To: <[email-masked]<mailto:[email-masked]>>, "Yiu L. LEE" <[email-masked]<mailto:[email-masked]>>
Cc: <[email-masked]<mailto:[email-masked]>>
Subject: RE: [Softwires] Clarification of the stateles/stateful discussion

---

From: <[email-masked]>
Date: Wed, 10 Aug 2011 17:35:11 +0800

|-----Original Message-----
|From: Rémi Després [mailto:[email-masked]] 
|Sent: Wednesday, August 03, 2011 4:02 PM
|To: Rajiv Asati (rajiva)
|Cc: [email-masked]
|Subject: Re: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|
|Le 3 août 2011 à 00:39, Rajiv Asati (rajiva) a écrit :
|
|> Satoru-san,
|> 
|> This is an important point that most of us forget that 
|restricting to 
|> "n" ports doesn't equate to just "n" NAT sessions rather many more 
|> than n sessions. We must add that to the 4v6 motivation 
|draft as well 
|> as to the 4v6 comparison draft.
|
|+1

+1. Session number are usually many more than ports number.

We tested port and sessions assumptions of applications on the A+P NAT,
which has been documented in A+P experiments draft, to investigate how 
many ports and sessions they are costing on the NAT mapping table dynamically 
from the first NAT bindings being established to the last one being 
destroyed.

Some results below:

Usually, apps consume some more sessions than ports, and for the P2P apps, sessions
number could even be as a couple of times as ports number.

 For example, BitTorrent established five hundreds of sessions while the port consumption
 was under a hundred in the first minute of the communication, because when BitTorrent 
initiates a downloading, it first uses the same source port to connect to the different destinations 
(destination IP and port) therefore one source port multiplexing different sessions. Skype is
 another example that uses one source port to  multiplex different sessions thereby saving 
source port consumptions on NAT.

 For exact figures of ports/session number for apps, see my page: 
 http://opensourcev6transtechnologies.weebly.com/experiments-results.html
The session consumption comparison among the same set of applications is
 illustrated in Figure 4. 

Xiaohong

|
|RD
|
|> 
|>> port. There may be multiple session to different destinations using
|> the same
|>> external port. The 900G figure is valid, as long as internal hosts
|> reuse the
|>> same source address+port for different destinations.
|> 
|> Cheers,
|> Rajiv
|> 
|> 
|>> -----Original Message-----
|>> From: [email-masked] [mailto:[email-masked]]
|> On Behalf
|>> Of Simon Perreault
|>> Sent: Tuesday, August 02, 2011 9:55 AM
|>> To: [email-masked]
|>> Subject: Re: [Softwires] Clarification of the stateles/stateful
|> discussion
|>> 
|>> Simon Perreault wrote, on 08/02/2011 09:24 AM:
|>>> Satoru Matsushima wrote, on 08/01/2011 10:41 PM:
|>>>> Thanks, a clarification has made to clear a confusion of 
|restricted
|> port
|>>>> set/ranges and NAT session table limitation. Even if a CPE is
|> allocated 256
|>>>> ports, NAT session can be made over 900G sessions in theory.
|> ('2^32'<Full
|>>>> 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) *
|> 2^8<256
|>> ports>.
|>>> 
|>>> This is only true for endpoint-dependent NATs, which are forbidden
|> by the
|>> BEHAVE
|>>> RFCs (4787 and 5382). According to these RFCs, NATs MUST have 
|>>> endpoint-independent mapping behaviour. This means that each NAT
|> session
|>> will
|>>> consume one external port.
|>> 
|>> Sorry, I confused the terminology. Each NAT *binding* will consume 
|>> one external port. There may be multiple session to different 
|>> destinations using
|> the same
|>> external port. The 900G figure is valid, as long as internal hosts
|> reuse the
|>> same source address+port for different destinations.
|>> 
|>> Simon
|>> --
|>> DTN made easy, lean, and smart --> http://postellation.viagenie.ca
|>> NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
|>> STUN/TURN server               --> http://numb.viagenie.ca
|>> _______________________________________________
|>> Softwires mailing list
|>> [email-masked]
|>> https://www.ietf.org/mailman/listinfo/softwires
|> _______________________________________________
|> Softwires mailing list
|> [email-masked]
|> https://www.ietf.org/mailman/listinfo/softwires
|
|
|

---

From: "Lee, Yiu" <[email-masked]>
Date: Tue, 9 Aug 2011 16:36:59 +0000

You go it.

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Wed, 10 Aug 2011 07:38:44 -0500

Xiaohong,

Your work is very insightful. Thanks for pointing it out.

In your experimentation, did you use 'consecutive' port-set or 'scattered' port-set or both? If just the former, then is it possible to experiment using 'scattered' port-set?

Cheers,
Rajiv

> From: [email-masked] [mailto:xiaohong.deng@orange-
> ftgroup.com]
> Sent: Wednesday, August 10, 2011 5:35 AM
> To: [email-masked]; Rajiv Asati (rajiva)
> Cc: [email-masked]

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 10 Aug 2011 17:44:31 +0000

I also agree this is a very nice piece of work. The current title is
"Implementing AplusP in the provider's IPv6-only network". I think this
not only covers "Implementing AplusP in an IPv6-only network", but also
captures very important information about the port usage in various
applications. I would like to see the draft to expand to include more
popular applications and OS, and the radio between port usage and number
of sessions. By reading Fig. 3 and Fig. 4 in
http://opensourcev6transtechnologies.weebly.com/experiments-results.html,
the radio is roughly 1 to 4. These experiments give concrete results to
operators to plan for any port-sharing technology.

Cheers,
Yiu

On 8/10/11 8:38 AM, "Rajiv Asati (rajiva)" <[email-masked]> wrote:

>In your experimentation, did you use 'consecutive' port-set or
>'scattered' port-set or both? If just the former, then is it possible to
>experiment using 'scattered' port-set?
>
>Cheers,
>Rajiv
>
>
>> Usually, apps consume some more sessions than ports, and for the P2P
>>apps,
>> sessions
>>  For example, BitTorrent established five hundreds of sessions while
>>the port
>> consumption
>>  was under a hundred in the first minute of the communication, because
>>when
>> BitTorrent
>> initiates a downloading, it first uses the same source port to connect
>>to the
>> different destinations
>> (destination IP and port) therefore one source port multiplexing
>>different
>> sessions. Skype is
>>  another example that uses one source port to  multiplex different
>>sessions
>> thereby saving

---

From: <[email-masked]>
Date: Thu, 11 Aug 2011 18:44:13 +0800

Thanks Yiu, your suggestion makes a lot of sense, and I'm considering it.

Cheers,
Xiaohong 

|-----Original Message-----
|From: Lee, Yiu [mailto:[email-masked]] 
|Sent: Thursday, August 11, 2011 1:45 AM
|To: [email-masked]
|Subject: Re: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|I also agree this is a very nice piece of work. The current 
|title is "Implementing AplusP in the provider's IPv6-only 
|network". I think this not only covers "Implementing AplusP in 
|an IPv6-only network", but also captures very important 
|information about the port usage in various applications. I 
|would like to see the draft to expand to include more popular 
|applications and OS, and the radio between port usage and 
|number of sessions. By reading Fig. 3 and Fig. 4 in 
|http://opensourcev6transtechnologies.weebly.com/experiments-res
|ults.html,
|the radio is roughly 1 to 4. These experiments give concrete 
|results to operators to plan for any port-sharing technology.
|
|Cheers,
|Yiu
|
|
|On 8/10/11 8:38 AM, "Rajiv Asati (rajiva)" <[email-masked]> wrote:
|
|>Xiaohong,
|>
|>Your work is very insightful. Thanks for pointing it out.
|>
|>In your experimentation, did you use 'consecutive' port-set or 
|>'scattered' port-set or both? If just the former, then is it possible 
|>to experiment using 'scattered' port-set?
|>
|>Cheers,
|>Rajiv
|>
|>
|>> -----Original Message-----
|>> From: [email-masked] 
|[mailto:xiaohong.deng@orange-  
|>>ftgroup.com]
|>> Sent: Wednesday, August 10, 2011 5:35 AM
|>> To: [email-masked]; Rajiv Asati (rajiva)
|>> Cc: [email-masked]
|>> Subject: RE: [Softwires] Clarification of the stateles/stateful 
|>>discussion
|>> 
|>> 
|>> 
|>> |-----Original Message-----
|>> |From: Rémi Després [mailto:[email-masked]]
|>> |Sent: Wednesday, August 03, 2011 4:02 PM
|>> |To: Rajiv Asati (rajiva)
|>> |Cc: [email-masked]
|>> |Subject: Re: [Softwires] Clarification of the stateles/stateful 
|>> |discussion
|>> |
|>> |
|>> |Le 3 août 2011 à 00:39, Rajiv Asati (rajiva) a écrit :
|>> |
|>> |> Satoru-san,
|>> |>
|>> |> This is an important point that most of us forget that
|>> |restricting to
|>> |> "n" ports doesn't equate to just "n" NAT sessions rather 
|many more 
|>> |> than n sessions. We must add that to the 4v6 motivation
|>> |draft as well
|>> |> as to the 4v6 comparison draft.
|>> |
|>> |+1
|>> 
|>> +1. Session number are usually many more than ports number.
|>> 
|>> We tested port and sessions assumptions of applications on the A+P 
|>>NAT,  which has been documented in A+P experiments draft, to 
|>>investigate how  many ports and sessions they are costing on the NAT 
|>>mapping table dynamically  from the first NAT bindings being 
|>>established to the last one being  destroyed.
|>> 
|>> Some results below:
|>> 
|>> Usually, apps consume some more sessions than ports, and 
|for the P2P 
|>>apps,  sessions  number could even be as a couple of times as ports 
|>>number.
|>> 
|>>  For example, BitTorrent established five hundreds of 
|sessions while 
|>>the port  consumption
|>>  was under a hundred in the first minute of the communication, 
|>>because when  BitTorrent  initiates a downloading, it first uses the 
|>>same source port to connect to the  different destinations  
|>>(destination IP and port) therefore one source port multiplexing 
|>>different  sessions. Skype is
|>>  another example that uses one source port to  multiplex different 
|>>sessions  thereby saving  source port consumptions on NAT.
|>> 
|>>  For exact figures of ports/session number for apps, see my page:
|>>  
|>>http://opensourcev6transtechnologies.weebly.com/experiments-r
|esults.ht
|>>ml  The session consumption comparison among the same set of 
|>>applications is
|>>  illustrated in Figure 4.
|>> 
|>> Xiaohong
|>> 
|>> 
|>> |
|>> |RD
|>> |
|>> |>
|>> |>> port. There may be multiple session to different destinations 
|>> |>> using
|>> |> the same
|>> |>> external port. The 900G figure is valid, as long as internal 
|>> |>> hosts
|>> |> reuse the
|>> |>> same source address+port for different destinations.
|>> |>
|>> |> Cheers,
|>> |> Rajiv
|>> |>
|>> |>
|>> |>> -----Original Message-----
|>> |>> From: [email-masked] 
|>> |>> [mailto:[email-masked]]
|>> |> On Behalf
|>> |>> Of Simon Perreault
|>> |>> Sent: Tuesday, August 02, 2011 9:55 AM
|>> |>> To: [email-masked]
|>> |>> Subject: Re: [Softwires] Clarification of the stateles/stateful
|>> |> discussion
|>> |>>
|>> |>> Simon Perreault wrote, on 08/02/2011 09:24 AM:
|>> |>>> Satoru Matsushima wrote, on 08/01/2011 10:41 PM:
|>> |>>>> Thanks, a clarification has made to clear a confusion of
|>> |restricted
|>> |> port
|>> |>>>> set/ranges and NAT session table limitation. Even if a CPE is
|>> |> allocated 256
|>> |>>>> ports, NAT session can be made over 900G sessions in theory.
|>> |> ('2^32'<Full
|>> |>>>> 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) *
|>> |> 2^8<256
|>> |>> ports>.
|>> |>>>
|>> |>>> This is only true for endpoint-dependent NATs, which are 
|>> |>>> forbidden
|>> |> by the
|>> |>> BEHAVE
|>> |>>> RFCs (4787 and 5382). According to these RFCs, NATs MUST have 
|>> |>>> endpoint-independent mapping behaviour. This means 
|that each NAT
|>> |> session
|>> |>> will
|>> |>>> consume one external port.
|>> |>>
|>> |>> Sorry, I confused the terminology. Each NAT *binding* will 
|>> |>> consume one external port. There may be multiple session to 
|>> |>> different destinations using
|>> |> the same
|>> |>> external port. The 900G figure is valid, as long as internal 
|>> |>> hosts
|>> |> reuse the
|>> |>> same source address+port for different destinations.
|>> |>>
|>> |>> Simon
|>> |>> --
|>> |>> DTN made easy, lean, and smart --> 
|http://postellation.viagenie.ca
|>> |>> NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
|>> |>> STUN/TURN server               --> http://numb.viagenie.ca
|>> |>> _______________________________________________
|>> |>> Softwires mailing list
|>> |>> [email-masked]
|>> |>> https://www.ietf.org/mailman/listinfo/softwires
|>> |> _______________________________________________
|>> |> Softwires mailing list
|>> |> [email-masked]
|>> |> https://www.ietf.org/mailman/listinfo/softwires
|>> |
|>> |
|>> |
|>_______________________________________________
|>Softwires mailing list
|>[email-masked]
|>https://www.ietf.org/mailman/listinfo/softwires
|
|
|

---

From: <[email-masked]>
Date: Thu, 11 Aug 2011 19:01:58 +0800

Rajiv,

Thanks!

It's 'consecutive' port-set, and thanks for your proposal for testing  'scattered' port-set.

But I don't understand will the figure be different with 'scattered' port-set NAT? To my knowledge, it's something todo with app's behaviours and NAT type(EIM in our test).
Would you explain if I miss something? Thanks

Cheers,
Xiaohong

|-----Original Message-----
|From: Rajiv Asati (rajiva) [mailto:[email-masked]] 
|Sent: Wednesday, August 10, 2011 8:39 PM
|To: DENG Xiaohong ESP/PEK; [email-masked]
|Cc: [email-masked]
|Subject: RE: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|Xiaohong,
|
|Your work is very insightful. Thanks for pointing it out.
|
|In your experimentation, did you use 'consecutive' port-set or 
|'scattered' port-set or both? If just the former, then is it 
|possible to experiment using 'scattered' port-set?
|
|Cheers,
|Rajiv
|
|
|> -----Original Message-----
|> From: [email-masked] [mailto:xiaohong.deng@orange- 
|> ftgroup.com]
|> Sent: Wednesday, August 10, 2011 5:35 AM
|> To: [email-masked]; Rajiv Asati (rajiva)
|> Cc: [email-masked]
|> Subject: RE: [Softwires] Clarification of the stateles/stateful 
|> discussion
|> 
|> 
|> 
|> |-----Original Message-----
|> |From: Rémi Després [mailto:[email-masked]]
|> |Sent: Wednesday, August 03, 2011 4:02 PM
|> |To: Rajiv Asati (rajiva)
|> |Cc: [email-masked]
|> |Subject: Re: [Softwires] Clarification of the stateles/stateful 
|> |discussion
|> |
|> |
|> |Le 3 août 2011 à 00:39, Rajiv Asati (rajiva) a écrit :
|> |
|> |> Satoru-san,
|> |>
|> |> This is an important point that most of us forget that
|> |restricting to
|> |> "n" ports doesn't equate to just "n" NAT sessions rather 
|many more 
|> |> than n sessions. We must add that to the 4v6 motivation
|> |draft as well
|> |> as to the 4v6 comparison draft.
|> |
|> |+1
|> 
|> +1. Session number are usually many more than ports number.
|> 
|> We tested port and sessions assumptions of applications on the A+P 
|> NAT, which has been documented in A+P experiments draft, to 
|> investigate how many ports and sessions they are costing on the NAT 
|> mapping table dynamically from the first NAT bindings being 
|> established to the last one being destroyed.
|> 
|> Some results below:
|> 
|> Usually, apps consume some more sessions than ports, and for the P2P 
|> apps, sessions number could even be as a couple of times as ports 
|> number.
|> 
|>  For example, BitTorrent established five hundreds of sessions while 
|> the port consumption  was under a hundred in the first minute of the 
|> communication, because when BitTorrent initiates a downloading, it 
|> first uses the same source port to connect to the different 
|> destinations (destination IP and port) therefore one source port 
|> multiplexing different sessions. Skype is  another example that uses 
|> one source port to  multiplex different sessions thereby 
|saving source 
|> port consumptions on NAT.
|> 
|>  For exact figures of ports/session number for apps, see my page:
|>  
|> 
|http://opensourcev6transtechnologies.weebly.com/experiments-results.ht
|> ml The session consumption comparison among the same set of 
|> applications is  illustrated in Figure 4.
|> 
|> Xiaohong
|> 
|> 
|> |
|> |RD
|> |
|> |>
|> |>> port. There may be multiple session to different destinations 
|> |>> using
|> |> the same
|> |>> external port. The 900G figure is valid, as long as 
|internal hosts
|> |> reuse the
|> |>> same source address+port for different destinations.
|> |>
|> |> Cheers,
|> |> Rajiv
|> |>
|> |>
|> |>> -----Original Message-----
|> |>> From: [email-masked] 
|> |>> [mailto:[email-masked]]
|> |> On Behalf
|> |>> Of Simon Perreault
|> |>> Sent: Tuesday, August 02, 2011 9:55 AM
|> |>> To: [email-masked]
|> |>> Subject: Re: [Softwires] Clarification of the stateles/stateful
|> |> discussion
|> |>>
|> |>> Simon Perreault wrote, on 08/02/2011 09:24 AM:
|> |>>> Satoru Matsushima wrote, on 08/01/2011 10:41 PM:
|> |>>>> Thanks, a clarification has made to clear a confusion of
|> |restricted
|> |> port
|> |>>>> set/ranges and NAT session table limitation. Even if a CPE is
|> |> allocated 256
|> |>>>> ports, NAT session can be made over 900G sessions in theory.
|> |> ('2^32'<Full
|> |>>>> 32bits v4 address> - '2^29'<class-D/E> - '2^7'<0/8,127/8>) *
|> |> 2^8<256
|> |>> ports>.
|> |>>>
|> |>>> This is only true for endpoint-dependent NATs, which are 
|> |>>> forbidden
|> |> by the
|> |>> BEHAVE
|> |>>> RFCs (4787 and 5382). According to these RFCs, NATs MUST have 
|> |>>> endpoint-independent mapping behaviour. This means that each NAT
|> |> session
|> |>> will
|> |>>> consume one external port.
|> |>>
|> |>> Sorry, I confused the terminology. Each NAT *binding* 
|will consume 
|> |>> one external port. There may be multiple session to different 
|> |>> destinations using
|> |> the same
|> |>> external port. The 900G figure is valid, as long as 
|internal hosts
|> |> reuse the
|> |>> same source address+port for different destinations.
|> |>>
|> |>> Simon
|> |>> --
|> |>> DTN made easy, lean, and smart --> 
|http://postellation.viagenie.ca
|> |>> NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
|> |>> STUN/TURN server               --> http://numb.viagenie.ca
|> |>> _______________________________________________
|> |>> Softwires mailing list
|> |>> [email-masked]
|> |>> https://www.ietf.org/mailman/listinfo/softwires
|> |> _______________________________________________
|> |> Softwires mailing list
|> |> [email-masked]
|> |> https://www.ietf.org/mailman/listinfo/softwires
|> |
|> |
|> |
|

---

From: Simon Perreault <[email-masked]>
Date: Thu, 11 Aug 2011 08:37:55 -0400

On 2011-08-11 07:01, [email-masked] wrote:
> But I don't understand will the figure be different with 'scattered'
> port-set NAT? To my knowledge, it's something todo with app's
> behaviours and NAT type(EIM in our test). Would you explain if I miss
> something? Thanks

As it was discussed in this thread [*], an EIM NAT will, in theory, only
be able to support a number of sessions roughly equal to the number of
external ports. That is because internal hosts typically do not reuse
source port numbers for sessions to different destinations.

If your tests show otherwise, it would be very important for this
working group to understand exactly why theory does not match practice.

[*] http://www.ietf.org/mail-archive/web/softwires/current/msg02461.html

Simon

---

From: Nejc Škoberne <[email-masked]>
Date: Thu, 11 Aug 2011 21:00:18 +0100

Hello all,

I am working on a trade-off analysis of IPv4 address sharing mechanisms. I am 
replying to Remi's e-mail (sent before I subscribed to the list):

> I therefore worked out a way to present the range of solutions to be compared, 
> with the following taken in consideration:
> - The stateless/stateful IPv4 across IPv6 comparison isn't limited to IPv4 
>   shared addresses (applies also to exclusive IPv4 customer addresses).
> - If there is, in BR/AFBR's, no Customer state (i.e. no states referring to 
>   individual IPv6 prefixes), there can't be per-transport-connection state either.
> - CE-CE direct paths are possible only if IPv4/IPv6 mappings of BR/AFBR's don't 
>   depend on Customer state.
>
> The proposed document structure is as follows, with pros and cons for each section:
>    e.g. draft-cui-softwire-host-4over6-06

I think we really need such a comparison document. As a newcomer I spent a lot
of time digging up all the scattered documents here and it wasn't easy grasp the 
"big picture" of what has been proposed. I think we also need a trade-off analysis 
document, which I am writing at the moment.

Anyway, I am not sure I agree with the structure suggested. I think we should try
to be even more general and try to abstract all packet manipulations. I have prepared
a table which tries to document all so-far implemented combinations (IPv4 address sharing
mechanisms + 4via6 mechanisms, the intersection includes all but NAT444, which is
only IPv4 address sharing mechanism and Public 4over6 which is only a 4via6 mechanism). 
It would be useful to know if any of the missing combinations might be useful in 
practice as well, I hope this table will help in determining that.

I will be very happy to hear any comments on the comparison. It's only a 1-page table.

The PDF document: http://nejc.skoberne.net/transfer/mechanisms.pdf

Thanks,
Nejc

---

From: Reinaldo Penno <[email-masked]>
Date: Fri, 12 Aug 2011 01:56:40 -0400

I like it. Nice and clean diagram.

I believe NAT64 & 'provision of additional IP address and port ranges'
should be 'yes'. NAT64 uses a IPv4 NAT pool much like NAT444 and needs to
support all the bells and whistles...APP, EIM, HP, PB, etc

Same goes for " Requirements for provision of additional IPv4 addresses and
port ranges" and "NAT64". I believe it should be 'none' since it is
automatic.

Thanks,

Reinaldo

---

From: Nejc Škoberne <[email-masked]>
Date: Fri, 12 Aug 2011 08:26:20 +0100

Dear Reinaldo,

Thanks.

> I believe NAT64&  'provision of additional IP address and port ranges'

Of course, that's true, my mistake. I already fixed it and reuploaded it so 
there's an updaten version online already.

Thanks!

Nejc

---

From: <[email-masked]>
Date: Fri, 12 Aug 2011 20:32:26 +0800

|-----Original Message-----
|From: Simon Perreault [mailto:[email-masked]] 
|Sent: Thursday, August 11, 2011 8:38 PM
|To: DENG Xiaohong ESP/PEK
|Cc: [email-masked]; [email-masked]; [email-masked]
|Subject: Re: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|On 2011-08-11 07:01, [email-masked] wrote:
|> But I don't understand will the figure be different with 'scattered'
|> port-set NAT? To my knowledge, it's something todo with app's 
|> behaviours and NAT type(EIM in our test). Would you explain 
|if I miss 
|> something? Thanks
|
|As it was discussed in this thread [*], an EIM NAT will, in 

Understood EIM behaviors. My point is port-range
NAT or scattered port-set NAT is nothing to do with EIM.

|theory, only be able to support a number of sessions roughly 
|equal to the number of external ports. That is because 
|internal hosts typically do not reuse source port numbers for 
|sessions to different destinations.

Could you explain how this conclusion was drawed?

According to [a], port usage in figure 3 and session usage in figure 4 show that,
for Google Earth, port and session usage are roughly the same,
while firefox session usage is roughly 1.5 times as port usage.

|
|If your tests show otherwise, it would be very important for 
|this working group to understand exactly why theory does not 
|match practice.

BitTorrent and Skype are definitely cases that show othersiwse.

As the detailed analysis I stated in the previous email, but don't mind to
explain again:

When BT Client initiates a downloading it first uses the same 
source port to connect to the different destinations (destination IP and port) 
therefore a source port multiplexing different sessions. Data shows maxium
port usuage reaches 160, while session usuage continously stays above 450.

Skype is another example that uses a source port to multiplex 
different sessions thereby introducing less source port than sessions on NAT.
Data shows less than a couple of ports achieve more than a hundred sessions!

But, don't ask me why, as  niether did I write code for BT Client nor for Skype. ;0

[a] http://opensourcev6transtechnologies.weebly.com/experiments-results.html

Cheers,
Xiaohong
---
Open-source PCP Client : http://opensourcev6transtechnologies.weebly.com/
A+P experiments results: http://opensourcev6transtechnologies.weebly.com/experiments-results.html
IETF81 PCP demo page: http://opensourcev6transtechnologies.weebly.com/ietf-81-pcp-demo-site.html
|
|[*] 
|http://www.ietf.org/mail-archive/web/softwires/current/msg02461.html
|
|Simon
|--
|DTN made easy, lean, and smart --> http://postellation.viagenie.ca
|NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
|STUN/TURN server               --> http://numb.viagenie.ca
|

---

From: <[email-masked]>
Date: Fri, 12 Aug 2011 14:59:56 +0200

Dear Dave,

I would personally appreciate if you can draft your thoughts in form of an I-D before the meeting so that we can comment on them and discuss the points you are raising.

FWIW, draft-operators-softwire-stateless-4v6-motivation does not argue against stateful but it elaborates on a set of motivations to explain why stateless is also needed. -00 and -01 of the draft includes a set of open issues. That section has been removed because of the comments received in this list.

As for the point about failure modes, this is discussed in: draft-dec-stateless-4v6

Cheers,
Med

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de Dave Thaler
Envoyé : jeudi 28 juillet 2011 21:45
À : Satoru Matsushima; [email-masked] list
Objet : Re: [Softwires] Yesterday's slides

Attaching here since they don't seem to be posted yet.

-Dave

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Satoru Matsushima
> Sent: Thursday, July 28, 2011 1:29 PM
> To: [email-masked] list
> Subject: [Softwires] Yesterday's slides
> 
> Hi,
> 
> Could someone tell me where is Dave Thaler's slides?
> 
> Best regards,
> --satoru

---

From: Simon Perreault <[email-masked]>
Date: Fri, 12 Aug 2011 09:21:49 -0400

Initially I wrote a long email with a point-by-point reply, but then I
found what I think could be the source of our misunderstanding:

> Data shows less than a couple of ports achieve more than a hundred sessions!

In your experiments, do you measure port and session usage on the NAT,
on the internal host, or both? It's not clear to me when reading your
experiment results at
http://opensourcev6transtechnologies.weebly.com/experiments-results.html

Thanks,
Simon

---

From: <[email-masked]>
Date: Fri, 12 Aug 2011 21:25:03 +0800

|-----Original Message-----
|From: Simon Perreault [mailto:[email-masked]] 
|Sent: Friday, August 12, 2011 9:22 PM
|To: DENG Xiaohong ESP/PEK
|Cc: [email-masked]; [email-masked]; [email-masked]
|Subject: Re: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|Initially I wrote a long email with a point-by-point reply, 
|but then I found what I think could be the source of our 
|misunderstanding:
|
|On 2011-08-12 08:32, [email-masked] wrote:
|> BitTorrent and Skype are definitely cases that show othersiwse.
|> 
|> As the detailed analysis I stated in the previous email, but don't 
|> mind to explain again:
|> 
|> When BT Client initiates a downloading it first uses the same source 
|> port to connect to the different destinations (destination IP and 
|> port) therefore a source port multiplexing different sessions. Data 
|> shows maxium port usuage reaches 160, while session usuage 
|continously stays above 450.
|> 
|> Skype is another example that uses a source port to multiplex 
|> different sessions thereby introducing less source port than 
|sessions on NAT.
|> Data shows less than a couple of ports achieve more than a 
|hundred sessions!
|
|In your experiments, do you measure port and session usage on 
|the NAT, on the internal host, or both? It's not clear to me 
|when reading your experiment results at 
|http://opensourcev6transtechnologies.weebly.com/experiments-res
|ults.html

The data shows there are only the port and session usage on the NAT.

Cheers,
Xiaohong

|
|Thanks,
|Simon
|--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 12 Aug 2011 09:40:39 -0400

> The data shows there are only the port and session usage on the NAT.

Ok. Thanks.

Then I think we're missing an important piece of data: port and session
usage on the host. Here's why, with an example.

Suppose I run an app on my host that creates two sessions using two
different source ports:

192.168.0.10:65000 --> 1.2.3.4:80
192.168.0.10:65001 --> 5.6.7.8:80

An EIM NAT will always map these two sessions to two different external
ports, let's say like this:

4.4.4.4:65000 --> 1.2.3.4:80
4.4.4.4:65001 --> 5.6.7.8:80

A non-EIM NAT could instead conserve ports by reusing the same external
port for the two sessions:

4.4.4.4:65000 --> 1.2.3.4:80
4.4.4.4:65000 --> 5.6.7.8:80

------------------------------

Suppose now that a different app creates two sessions using the same
source port:

192.168.0.10:65000 --> 1.2.3.4:80
192.168.0.10:65000 --> 5.6.7.8:80

Both the EIM NAT and the non-EIM NAT would map them to the same external
port:

4.4.4.4:65000 --> 1.2.3.4:80
4.4.4.4:65000 --> 5.6.7.8:80

--------------------------------------

So, conclusion: unless your extremely certain of the NAT behaviour (we
rarely are, especially in edge cases), just by looking at the NAT port
and session usage says nothing about what happens on the host. We don't
know whether BitTorrent, Skype, or Firefox reuse source ports for
different destinations or whether they simply use additional internal
source ports.

Usually, if an application just uses the usual BSD sockets API to
initiate each new session (socket() then connect()), the OS will assign
a different source ports (for all common consumer OSes). It could be
possible that the application does it otherwise, but the straightforward
way to do it just consumes another port.

So your observations on the NAT, where significantly more sessions were
active than there were ports, mean that either of these is true:

a) Firefox, the BitTorrent client you tested, and Skype do not create
new sessions with the usual socket() then connect() sequence. (That
would be quite strange for Firefox and BitTorrent, not so much for Skype.)

b) The NAT you tested was not EIM. (This would not be surprising since
there are many non-EIM consumer grade NATs out there.)

Thoughts?

Simon

---

From: Simon Perreault <[email-masked]>
Date: Fri, 12 Aug 2011 10:01:18 -0400

> The PDF document: http://nejc.skoberne.net/transfer/mechanisms.pdf

VERY nice!

One comment I would have is that maybe colour could be used to indicate
good/bad instead of yes/no.

Then we just pick the column with the less red on it and we're done. ;)

Simon

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 12 Aug 2011 09:08:39 -0500

Nejc,

Good table indeed. 

If you could add few more service related impact due to other features such as UPNP inside home, DPI in the network, L4 QoS in the network, L4-aware routing etc., then it would be quite insightful. I may also help you with that.

Cheers,
Rajiv

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Simon Perreault
> Sent: Friday, August 12, 2011 10:01 AM
> To: [email-masked]

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 12 Aug 2011 09:20:49 -0500

Xiaohong,

It shouldn't, I agree. And your test could attest to that (as some may not agree otherwise).

Cheers,
Rajiv

> Sent: Thursday, August 11, 2011 7:02 AM
> To: Rajiv Asati (rajiva); [email-masked]
> Cc: [email-masked]
> But I don't understand will the figure be different with 'scattered' port-set
> NAT? To my knowledge, it's something todo with app's behaviours and NAT
> type(EIM in our test).

---

From: Reinaldo Penno <[email-masked]>
Date: Fri, 12 Aug 2011 12:25:32 -0400

This could be a separate table. The good thing about this table is that is
clean and focused.

On 8/12/11 7:08 AM, "Rajiv Asati (rajiva)" <[email-masked]> wrote:

> Nejc,
> 
> Good table indeed.
> 
> If you could add few more service related impact due to other features such as
> UPNP inside home, DPI in the network, L4 QoS in the network, L4-aware routing
> etc., then it would be quite insightful. I may also help you with that.
> 
> Cheers,
> Rajiv
> 
> 
>> Behalf

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 12 Aug 2011 11:27:39 -0500

Keeping it clean and focused would yield minimal benefit if the operators can't relate to the impact on their services and infrastructures.

Cheers,
Rajiv

> From: Reinaldo Penno [mailto:[email-masked]]
> Sent: Friday, August 12, 2011 12:26 PM
> To: Rajiv Asati (rajiva); Simon Perreault; [email-masked]

---

From: Reinaldo Penno <[email-masked]>
Date: Fri, 12 Aug 2011 12:33:33 -0400

Can you propose a list of all 'other features' you have in mind that might
impact all operators?

On 8/12/11 9:27 AM, "Rajiv Asati (rajiva)" <[email-masked]> wrote:

> Keeping it clean and focused would yield minimal benefit if the operators
> can't relate to the impact on their services and infrastructures.
> 
> Cheers,
> Rajiv
> 
>

---

From: Nejc Škoberne <[email-masked]>
Date: Sat, 13 Aug 2011 16:30:14 +0100

Dear Simon,

Thanks.

> Then we just pick the column with the less red on it and we're done. ;)

Well, I was thinking about this, but it's not really easy to make such
judgements. I mean, a lot of preassumptions are needed to make this
consistent. So perhaps I'll try to change red-green to something less
"judging". This table doesn't want to judge, it just want's to document
current status. At least that was my idea.

Nejc

---

From: Nejc Škoberne <[email-masked]>
Date: Sat, 13 Aug 2011 16:32:18 +0100

Dear Rajiv,

Thanks.

> If you could add few more service related impact due to other features such as UPNP inside home, DPI in the network, L4 QoS in the network, L4-aware routing etc., then it would be quite insightful.…

Yes, this is the general idea here. I will extend the table onto next 
page and compare all these aspects. I'd really appreciate any help.

Will send my first extended draft here and then you can comment/add
additional lines. You can also send me private mail, of course.

Thanks,
Nejc

---

From: Nejc Škoberne <[email-masked]>
Date: Sat, 13 Aug 2011 16:33:23 +0100

Dear Reinaldo,

> This could be a separate table. The good thing about this table is that is

I don't think that extending this table on another page will make it less 
clean and focused.

Nejc

---

From: Wojciech Dec <[email-masked]>
Date: Mon, 15 Aug 2011 10:07:32 +0200

Hello Nejc,

your comparison does not appear to include:
http://tools.ietf.org/html/draft-murakami-softwire-4v6-translation-00

Also, feel free to use/refer to the table in:
http://tools.ietf.org/html/draft-dec-stateless-4v6-02#section-4.1 , which
although being specific to the stateless variants, has material applicable
to the wider comparison that includes stateful.

Regards,
Wojciech.

2011/8/13 Nejc Škoberne <[email-masked]>

---

From: Nejc Škoberne <[email-masked]>
Date: Mon, 15 Aug 2011 11:06:28 +0100

Dear Wojciech,

> http://tools.ietf.org/html/draft-murakami-softwire-4v6-translation-00

Interesting. I guess the idea is the same as in draft-xli-behave-divi-03? Could
you please elaborate a bit on the differences? 

> http://tools.ietf.org/html/draft-dec-stateless-4v6-02#section-4.1 , 
> which although being specific to the stateless variants, has material 
> applicable to the wider comparison that includes stateful.

I am aware of this table, thanks. I think it's a great work! I am definitely
considering it.

Thanks,
Nejc

---

From: <[email-masked]>
Date: Mon, 15 Aug 2011 18:19:34 +0800

|-----Original Message-----
|From: Simon Perreault [mailto:[email-masked]] 
|Sent: Friday, August 12, 2011 9:41 PM
|To: DENG Xiaohong ESP/PEK
|Cc: [email-masked]; [email-masked]; [email-masked]
|Subject: Re: [Softwires] Clarification of the 
|stateles/stateful discussion
|
|On 2011-08-12 09:25, [email-masked] wrote:
|> |In your experiments, do you measure port and session usage on the 
|> |NAT, on the internal host, or both? It's not clear to me 
|when reading 
|> |your experiment results at 
|> |http://opensourcev6transtechnologies.weebly.com/experiments-res
|> |ults.html
|> 
|> The data shows there are only the port and session usage on the NAT.
|
|Ok. Thanks.
|
|Then I think we're missing an important piece of data: port 
|and session usage on the host. Here's why, with an example.

Nice examples. More below.

|
|
|Suppose I run an app on my host that creates two sessions 
|using two different source ports:
|
|192.168.0.10:65000 --> 1.2.3.4:80
|192.168.0.10:65001 --> 5.6.7.8:80
|
|An EIM NAT will always map these two sessions to two different 
|external ports, let's say like this:
|
|4.4.4.4:65000 --> 1.2.3.4:80
|4.4.4.4:65001 --> 5.6.7.8:80
|
|A non-EIM NAT could instead conserve ports by reusing the same 
|external port for the two sessions:
|
|4.4.4.4:65000 --> 1.2.3.4:80
|4.4.4.4:65000 --> 5.6.7.8:80
|
|------------------------------
|
|Suppose now that a different app creates two sessions using 
|the same source port:

Very perceptive example. We did run into this case during PCP-UPnP
Interworking work when we're struggling how PCP handle internal ports 
conflicts.

It's quiet common that different apps share the same source port
(at least ) on Windows XP (observed). 

This case *exactly* shows how comes more sessions than ports
even with EIM NAT, so I wonder if you have this vision why still 
surprised by the results?

|
|192.168.0.10:65000 --> 1.2.3.4:80
|192.168.0.10:65000 --> 5.6.7.8:80
|
|Both the EIM NAT and the non-EIM NAT would map them to the 
|same external
|port:
|
|4.4.4.4:65000 --> 1.2.3.4:80
|4.4.4.4:65000 --> 5.6.7.8:80
|
|--------------------------------------

|
|
|So, conclusion: unless your extremely certain of the NAT 
|behaviour (we rarely are, especially in edge cases), just by 
|looking at the NAT port and session usage says nothing about 
|what happens on the host. We don't know whether BitTorrent, 

1. Looking at the NAT usage indeed says something about what
happens on the host. More reasons below.

2. Port and session usages on NAT are exactly what we're concerning
about, are they?

As far as BitTorrent concerned, it is the case that incomming
sessions are established by different destinations connecting to
the same internal port (listening port), which leads to more sessions
than ports on the host.

That's one reason,  among others, why sessions are more than 
ports on both the host and the NAT for BT applications.

(b) Second, tcp and udp sharing the same internal port to establish
different sessions is another reason results in more sessions.  

(c) Third, as you listed above, different applications share the same
internal port to establish different sessions.

|Skype, or Firefox reuse source ports for different 
|destinations or whether they simply use additional internal 
|source ports.
|
|Usually, if an application just uses the usual BSD sockets API 
|to initiate each new session (socket() then connect()), the OS 
|will assign a different source ports (for all common consumer 
|OSes). It could be possible that the application does it 
|otherwise, but the straightforward way to do it just consumes 
|another port.

Consider how  socket RESUME option works. ;)

|
|So your observations on the NAT, where significantly more 

Just for BT and skype cases, it is significantly more. For firefox
and IE, I would say it's a bit more, given that http requests are 
always acommpany with DNS UDP queries, which hopefully 
could you explain why, according to reason (b) and (c) above,
If you also agree with (b), since you already agreed with (c).

|sessions were active than there were ports, mean that either 
|of these is true:

Niether, IMHO.

|
|a) Firefox, the BitTorrent client you tested, and Skype do not 
|create new sessions with the usual socket() then connect() 
|sequence. (That would be quite strange for Firefox and 
|BitTorrent, not so much for Skype.)

See above. Hope it's clear stated and be stay open to hear your
comments. 

|
|b) The NAT you tested was not EIM. (This would not be 
|surprising since there are many non-EIM consumer grade NATs out there.)

IMHO. A linux knernal 2.6.32 ,which we used for A+P NAT, provide an EIM, 
yet corret me if wrong.

|
|Thoughts?

All above.

Cheers,
Xiaohong

|
|Simon
|--

---

From: Rémi Després <[email-masked]>
Date: Tue, 16 Aug 2011 08:18:13 +0200

Hi Nejc,

Thanks for this contribution.
+1 to have it pursued.

Maybe it would be better to only cover specifications for which drafts are available.
(I found the two "DSlite?" columns confusing, with no document to check their validity.)

Also, some of us are currently working on separating the 4rd "stateless address mapping" from its possible application to various encapsulation and translation mechanisms. 
When done, hopefully soon, this could influence the way to structure your comparison table.

Regards,
RD

---

From: Nejc Škoberne <[email-masked]>
Date: Tue, 16 Aug 2011 10:23:59 +0100

Dear Rémi,

> (I found the two "DSlite?" columns confusing, with no document to check their validity.)

I considered this, but then decided to do it this way since as far as I 
understand, DS-Lite RFC doesn't say what IPv4 address and UDP/TCP port
allocation policy must be supported. The same goes for provisioning
additional IPv4 addresses and port ranges. Is this correct?

> When done, hopefully soon, this could influence the way to structure your comparison table.

This would indeed be very useful. I will certainly work on it as soon as the document is
publicly available.

Thanks,
Nejc

---

From: Rémi Després <[email-masked]>
Date: Tue, 16 Aug 2011 12:06:21 +0200

Le 16 août 2011 à 11:23, Nejc Škoberne a écrit :

> allocation policy must be supported.

I see.
Then "unspecified" in the DS-lite column might be another way to express it.

BTW, colors don't seem necessary.
Without them, one avoids having to decide whether "IPv4 address and UDP/TCP port allocation policy" is red or green ;-).

> The same goes for provisioning

Excellent.
Look forward to it.

RD