Subject: Re: [bmwg] WGLC on IPv6 Benchmarking - discussion review

From: "Poretsky, Scott" <[email-masked]>
Date: Tue, 6 Mar 2007 15:05:29 -0500

Completely agree.  For example, performance at 71 bytes has become a key benchmark because of VoIP.

Scott

----- Original Message -----
From: McQuaid, Jim <[email-masked]>
To: Chip Popoviciu (cpopovic) <[email-masked]>; [email-masked] <[email-masked]>
Sent: Mon Mar 05 08:42:56 2007
Subject: RE: [bmwg] WGLC on IPv6 Benchmarking - discussion review

A nice sumary of all the frame size issues, among other things.

I would suggest that the time has come to consider a reference document (not an appendix to another RFC) from BMWG that addressed the "frame sizes to test" issue itself.  This info RFC could repeat the accepted wisdom, contain new tables for new media or new variants and could also include guidelines for testing, such as those you suggest.

In other words, obsolete Appendix B of 2544 with a new document.

?

Jim McQuaid

NetQoS

________________________________

From: Chip Popoviciu (cpopovic) [mailto:[email-masked]] 
Sent: Monday, March 05, 2007 1:51 AM
To: [email-masked]
Subject: [bmwg] WGLC on IPv6 Benchmarking - discussion review

Dear BMWG members,

I would like to recap last week's discussion on the IPv6 Methodology draft Last Call:

1) Based on the comment from David Newman and in observance of RFC1242, we will change "maximum throughput" to "throughput" in Appendix A.1 and A.2.

2) Based on the David Newman's suggestion, we will round down the theoretical rates listed in Appendix A.1 and will include a note regarding this round down as well as a reference to the tolerance due to clock slip.

3) Minimum Frame size for SONET. The proposal on the table is 48 bytes which leaves room just for a basic IPv6 header, no IP payload. I can see this as a useful test if the perspective is security threats oriented however, on a practical network such packets would not be present as pointed out by Curtis. I also want to point out that in RFC2544, the minimum IPv4 packet listed was realistic (20 bytes main header plus 8 bytes UDP header). In the case of Ethernet that was still smaller than the payload for the minim Ethernet frame size. In the case of SONET, shouldn't we take the same approach? IPv6 main header 40 bytes + 8 bytes UDP header (ECHO) which will give us 56 bytes minimum SONET frame size. 

4) Maximum frame size for SONET. David Newman thinks it would be difficult to define the max frame size for SONET due to products supporting different values. In the Appendix we stopped at 4096. Should we make a note that "larger values might be supported by the various products. The maximum supported frame size should be used in benchmarking in addition to the recommended frame sizes."?

5) Maximum frame size for Ethernet. Dan Romascanu mentioned the increases in the Ethernet max frame size to 1522 in order to accommodate the IEEE 802.1Q VLAN header, and recently to 2000, as per IEEE 802.3as. The Appendix lists frame sizes up to 9216 bytes. Is the recommendation to include 2000 explicitly in order to have the data point for the size recommended by 802.3as?

We wanted to take the opportunity to provide through this draft some updates to RFC2544 such as changes in frame sizes and new, relevant media types. This is a good opportunity to do such updates so please send further comments on the points above or any other items related to the draft before the Last Call closes on March 20 (http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ipv6-meth-01.txt).

Thank you to all those who sent comments so far.

Best Regards,

Chip

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Tue, 6 Mar 2007 15:26:37 -0500

I have two questions on the topic, one practical and on
logistical/procedural:
 
1) Is it people's experience that a platform's performance varies
significantly between 64bytes and 71bytes? If I have the data point at
64 bytes, wouldn't that be a rather safe upper bound for 71bytes as
well?
 
2) How would a document focused on "frame sizes to test" be
maintained/updated? On one hand there would be the changes in the
technologies (the Ethernet example given by Dan) but the changes might
not be fast enough to be a major concern. On the other, if we are indeed
concerned about application specific sizes (as above) we could see more
rapid changes that the document would have to keep up with, not to
mention the number of scenarios and conditions that would have to be
covered.
 
Best Regards,
Chip

From: Poretsky, Scott [mailto:[email-masked]] 
Sent: Tuesday, March 06, 2007 3:05 PM
To: [email-masked]; Chip Popoviciu (cpopovic); [email-masked]
Subject: Re: [bmwg] WGLC on IPv6 Benchmarking - discussion review

Completely agree.  For example, performance at 71 bytes has become a key
benchmark because of VoIP.

Scott

----- Original Message -----
From: McQuaid, Jim <[email-masked]>
To: Chip Popoviciu (cpopovic) <[email-masked]>; [email-masked]
<[email-masked]>

I would suggest that the time has come to consider a reference document
(not an appendix to another RFC) from BMWG that addressed the "frame
sizes to test" issue itself.  This info RFC could repeat the accepted
wisdom, contain new tables for new media or new variants and could also
include guidelines for testing, such as those you suggest.

?

Jim McQuaid

NetQoS

Dear BMWG members,

I would like to recap last week's discussion on the IPv6 Methodology
draft Last Call:

1) Based on the comment from David Newman and in observance of RFC1242,
we will change "maximum throughput" to "throughput" in Appendix A.1 and
A.2.

2) Based on the David Newman's suggestion, we will round down the
theoretical rates listed in Appendix A.1 and will include a note
regarding this round down as well as a reference to the tolerance due to
clock slip.

3) Minimum Frame size for SONET. The proposal on the table is 48 bytes
which leaves room just for a basic IPv6 header, no IP payload. I can see
this as a useful test if the perspective is security threats oriented
however, on a practical network such packets would not be present as
pointed out by Curtis. I also want to point out that in RFC2544, the
minimum IPv4 packet listed was realistic (20 bytes main header plus 8
bytes UDP header). In the case of Ethernet that was still smaller than
the payload for the minim Ethernet frame size. In the case of SONET,
shouldn't we take the same approach? IPv6 main header 40 bytes + 8 bytes
UDP header (ECHO) which will give us 56 bytes minimum SONET frame size.

4) Maximum frame size for SONET. David Newman thinks it would be
difficult to define the max frame size for SONET due to products
supporting different values. In the Appendix we stopped at 4096. Should
we make a note that "larger values might be supported by the various
products. The maximum supported frame size should be used in
benchmarking in addition to the recommended frame sizes."?

5) Maximum frame size for Ethernet. Dan Romascanu mentioned the
increases in the Ethernet max frame size to 1522 in order to accommodate
the IEEE 802.1Q VLAN header, and recently to 2000, as per IEEE 802.3as.
The Appendix lists frame sizes up to 9216 bytes. Is the recommendation
to include 2000 explicitly in order to have the data point for the size
recommended by 802.3as?

We wanted to take the opportunity to provide through this draft some
updates to RFC2544 such as changes in frame sizes and new, relevant
media types. This is a good opportunity to do such updates so please
send further comments on the points above or any other items related to
the draft before the Last Call closes on March 20
(http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ipv6-meth-01.txt).

Best Regards,

Chip

---

From: "McQuaid, Jim" <[email-masked]>
Date: Tue, 6 Mar 2007 14:29:32 -0600

My answer to #2: it's the nature of the RFC system.  1944 was replaced
by 2544 (there was an error in the reserved IP address range, for one).
Few documents live forever.  Appendix B of 2544 is obsolescent and a
newer RFC would be a good step forward.  But hardly immuned to change -
it just locates the change in a more visible place for updating.

Jim

NetQoS

From: Chip Popoviciu (cpopovic) [mailto:[email-masked]] 
Sent: Tuesday, March 06, 2007 3:27 PM
To: Poretsky, Scott; McQuaid, Jim; [email-masked]
Subject: RE: [bmwg] WGLC on IPv6 Benchmarking - discussion review

Best Regards,

Chip

Scott

----- Original Message -----
From: McQuaid, Jim <[email-masked]>
To: Chip Popoviciu (cpopovic) <[email-masked]>; [email-masked]
<[email-masked]>

?

Jim McQuaid

NetQoS

Dear BMWG members,

Best Regards,

Chip

---

From: "Rajiv Asati \(rajiva\)" <[email-masked]>
Date: Tue, 6 Mar 2007 15:43:42 -0500

Hi Chip,

On #1, my take is that the performance doesn't quite vary between 64B and 71B. I agree to keeping 64B. 

On #2, we would need a living document (;-)) that could be referenced by other drafts.

Cheers,
Rajiv

> -----Original Message-----
> From: Chip Popoviciu (cpopovic) 
> To: Poretsky, Scott; [email-masked]; [email-masked]
> 1) Is it people's experience that a platform's performance 
> varies significantly between 64bytes and 71bytes? If I have 
> the data point at 64 bytes, wouldn't that be a rather safe 
> upper bound for 71bytes as well?
>  
> maintained/updated? On one hand there would be the changes in 
> the technologies (the Ethernet example given by Dan) but the 
> changes might not be fast enough to be a major concern. On 
> the other, if we are indeed concerned about application 
> specific sizes (as above) we could see more rapid changes 
> that the document would have to keep up with, not to mention 
> the number of scenarios and conditions that would have to be covered.
>  
> Best Regards,
> Chip
> 
> I would suggest that the time has come to consider a 
> reference document (not an appendix to another RFC) from BMWG 
> that addressed the "frame sizes to test" issue itself.  This 
> info RFC could repeat the accepted wisdom, contain new tables 
> for new media or new variants and could also include 
> guidelines for testing, such as those you suggest.
> 
> 
> 
> 1) Based on the comment from David Newman and in observance 
> of RFC1242, we will change "maximum throughput" to 
> "throughput" in Appendix A.1 and A.2.
> 
> 
> 
> 2) Based on the David Newman's suggestion, we will round down 
> the theoretical rates listed in Appendix A.1 and will include 
> a note regarding this round down as well as a reference to 
> the tolerance due to clock slip.
> 
> 
> 
> 3) Minimum Frame size for SONET. The proposal on the table is 
> 48 bytes which leaves room just for a basic IPv6 header, no 
> IP payload. I can see this as a useful test if the 
> perspective is security threats oriented however, on a 
> practical network such packets would not be present as 
> pointed out by Curtis. I also want to point out that in 
> RFC2544, the minimum IPv4 packet listed was realistic (20 
> bytes main header plus 8 bytes UDP header). In the case of 
> Ethernet that was still smaller than the payload for the 
> minim Ethernet frame size. In the case of SONET, shouldn't we 
> take the same approach? IPv6 main header 40 bytes + 8 bytes 
> 4) Maximum frame size for SONET. David Newman thinks it would 
> be difficult to define the max frame size for SONET due to 
> products supporting different values. In the Appendix we 
> stopped at 4096. Should we make a note that "larger values 
> might be supported by the various products. The maximum 
> supported frame size should be used in benchmarking in 
> addition to the recommended frame sizes."?
> 
> 
> 
> 5) Maximum frame size for Ethernet. Dan Romascanu mentioned 
> the increases in the Ethernet max frame size to 1522 in order 
> to accommodate the IEEE 802.1Q VLAN header, and recently to 
> 2000, as per IEEE 802.3as. The Appendix lists frame sizes up 
> to 9216 bytes. Is the recommendation to include 2000 
> explicitly in order to have the data point for the size 
> recommended by 802.3as?
> 
> 
> 
> We wanted to take the opportunity to provide through this 
> draft some updates to RFC2544 such as changes in frame sizes 
> and new, relevant media types. This is a good opportunity to 
> do such updates so please send further comments on the points 
> above or any other items related to the draft before the Last 
> Call closes on March 20

---

From: "McQuaid, Jim" <[email-masked]>
Date: Tue, 6 Mar 2007 14:46:09 -0600

Actually, I think the point on 71 v. 64 bytes is a good example of why
this subject needs a document of its own.

In testing specifically for VOIP, this might (?) be material in some
networks, whereas 64 is fine for general principles....   Such an issue
and discussion would have a natural home in the Choosing Test Frame
Sizes document.

cheers,

Jim McQuaid

NetQoS

-----Original Message-----
From: Rajiv Asati (rajiva) [mailto:[email-masked]] 
Sent: Tuesday, March 06, 2007 3:44 PM
To: Chip Popoviciu (cpopovic); Poretsky, Scott; McQuaid, Jim;
[email-masked]

Hi Chip,

On #1, my take is that the performance doesn't quite vary between 64B
and 71B. I agree to keeping 64B. 

On #2, we would need a living document (;-)) that could be referenced by
other drafts.

Cheers,
Rajiv

---

From: "Rajiv Asati \(rajiva\)" <[email-masked]>
Date: Tue, 6 Mar 2007 15:53:08 -0500

Hi Jim,

> this subject needs a document of its own.

Agreed.

> networks, whereas 64 is fine for general principles....   

Do we think that keeping 64B along with 71B (or vice versa) would provide an additional value (since the performance difference between 64B and 71B is questionable)?

Thanks.

Cheers,
Rajiv

> From: McQuaid, Jim [mailto:[email-masked]] 
> Sent: Tuesday, March 06, 2007 3:46 PM
> To: Rajiv Asati (rajiva); Chip Popoviciu (cpopovic); 
> Poretsky, Scott; [email-masked]

---

From: "McQuaid, Jim" <[email-masked]>
Date: Tue, 6 Mar 2007 16:29:16 -0600

I wouldn't claim that there is absolutely any additional value.
However, there could be a bias to select one or the other depending on
the testing goal.  I suppose in a 64-bit machine, the 71-byte backet
might be suboptimal under some circumstances....

But mainly, I think a standalone doc on the subject could explore such
an issue and provide a guideline.

I guess I might be willing to take on such an effort, as long as I can
lots of help!

cheers,

Jim McQuaid

NetQoS

-----Original Message-----
From: Rajiv Asati (rajiva) [mailto:[email-masked]] 
Sent: Tuesday, March 06, 2007 3:53 PM
To: McQuaid, Jim; Chip Popoviciu (cpopovic); Poretsky, Scott;
[email-masked]

Hi Jim,

Agreed.

> networks, whereas 64 is fine for general principles....   

Do we think that keeping 64B along with 71B (or vice versa) would
provide an additional value (since the performance difference between
64B and 71B is questionable)?

Thanks.

Cheers,
Rajiv

> Such an issue