Subject: Re: [bmwg] WGLC: draft-ietf-bmwg-igp-dataplane drafts

From: Al Morton <[email-masked]>
Date: Wed, 15 Jul 2009 15:47:54 -0400

Comments on meth-18, all nits,
Al (mostly as participant)

At 01:44 PM 7/14/2009, Al Morton wrote:
>...This message begins a Last call on the IGP-Dataplane Convergence
>Time Benchmarking drafts.
>
>http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-term-18
>http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-meth-18
>http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-app-17
>
>The Last Call will end on July 31, 2009.

Section 4, top of page 10,
s/At least one condition need to /At least one condition needs to /

5.7.  Measurement Accuracy
...  When packet jitter is much less than the convergence
    time, it is a negligible source of error and therefor it will be
s/therefor/therefore/

5.9 Tester Capabilities
Add "Also see section 6 for method-specific capabilities."

6.1.3.  Measurement Accuracy

    TBD
(can't leave this TBD, if there's no obvious material, delete section ?)

Section 7, Reporting Format

         Maximum Packet Delay Threshold      seconds
this is called "Forwarding Delay Threshold" in the term doc, right?
pick one for both...

Section 9, same as terms, use the standard BMWG paragraphs...

---

From: "McLendon, John" <[email-masked]>
Date: Wed, 15 Jul 2009 14:59:30 -0700

Hi all,
I have a comment on the following statement in Sec 4, top of pp. 9, in
meth-18:

"For Convergence Events caused by the Tester, such as an IGP cost
change, the Tester may start to drop all traffic received from the
Preferred Egress
Interface at the Convergence Event Instant to achieve the same result."

The word "drop" implies that the test device forwards or drops traffic.
In my experience, test devices neither forward or drop traffic. They may
generate or analyze traffic, but a test device dropping traffic doesn't
seem helpful to the desired measurement. Many test devices can bring a
link down at the physical layer.

J...

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Al Morton
Sent: Wednesday, July 15, 2009 3:48 PM
To: [email-masked]
Subject: Re: [bmwg] WGLC: draft-ietf-bmwg-igp-dataplane drafts

5.9 Tester Capabilities

    TBD

    

_______________________________________________
bmwg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bmwg

<DIV><FONT size="1">

E-mail confidentiality.
--------------------------------
This e-mail contains confidential and / or privileged information belonging to Spirent Communications plc, its affiliates and / or subsidiaries. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution and / or the taking of any action based upon reliance on the contents of this transmission is strictly forbidden. If you have received this message in error please notify the sender by return e-mail and delete it from your system. If you require assistance, please contact our IT department at [email-masked].

Spirent Communications plc
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom.
Tel No. +44 (0) 1293 767676
Fax No. +44 (0) 1293 767677

Registered in England Number 470893
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom.

Or if within the US,

Spirent Communications,
26750 Agoura Road, Calabasas, CA, 91302, USA.
Tel No. 1-818-676- 2300 

</FONT></DIV>

---

From: "Scott Poretsky" <[email-masked]>
Date: Thu, 16 Jul 2009 18:41:17 +0300

I agree with John.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
McLendon, John
Sent: Wednesday, July 15, 2009 6:00 PM
To: Al Morton; [email-masked]
Subject: [bmwg] Comment on meth-18.

Hi all,

J...

-----Original Message-----

5.9 Tester Capabilities

    TBD

    

<DIV><FONT size="1">

E-mail confidentiality.
--------------------------------
This e-mail contains confidential and / or privileged information
belonging to Spirent Communications plc, its affiliates and / or
subsidiaries. If you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution and / or the taking
of any action based upon reliance on the contents of this transmission
is strictly forbidden. If you have received this message in error please
notify the sender by return e-mail and delete it from your system. If
you require assistance, please contact our IT department at
[email-masked].

Spirent Communications plc
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United
Kingdom.
Tel No. +44 (0) 1293 767676
Fax No. +44 (0) 1293 767677

Registered in England Number 470893
Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10
9XN, United Kingdom.

Or if within the US,

Spirent Communications,

</FONT></DIV>

---

From: "Kris Michielsen" <[email-masked]>
Date: Fri, 17 Jul 2009 17:02:43 +0200

Al, 

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Al Morton
> Sent: 15 July 2009 21:48
> To: [email-masked]
>     time, it is a negligible source of error and therefor it 
> will be s/therefor/therefore/
> 
> 5.9 Tester Capabilities

Corrected all the above.

> (can't leave this TBD, if there's no obvious material, delete 

Oops, I had put it in as a placeholder. I will think about it some more.

> this is called "Forwarding Delay Threshold" in the term doc, right?

I'll stick to Forwarding Delay Threshold.
 
> Section 9, same as terms, use the standard BMWG paragraphs...

OK

Many thanks!
Kris

---

From: "Kris Michielsen" <[email-masked]>
Date: Fri, 17 Jul 2009 17:45:39 +0200

John,

Thanks for reviewing the draft!

What do you think of the following suggestion?
OLD:
For Convergence Events caused by the Tester, such as an IGP cost change, the Tester may start to drop all traffic received from the
Preferred Egress Interface at the Convergence Event Instant to achieve the same result.

NEW:
For Convergence Events caused by the Tester, such as an IGP cost change, the Tester may start to discard all traffic received from
the Preferred Egress Interface at the Convergence Event Instant, or may be able to separately observe packets received from the
Preferred Egress Interface prior to the Convergence Event Instant, to achieve the same result.

Thanks,
Kris

> Behalf Of McLendon, John
> Sent: 16 July 2009 00:00
> "For Convergence Events caused by the Tester, such as an IGP 
> cost change, the Tester may start to drop all traffic 
> received from the Preferred Egress Interface at the 
> Convergence Event Instant to achieve the same result."
> 
> In my experience, test devices neither forward or drop 
> traffic. They may generate or analyze traffic, but a test 
> device dropping traffic doesn't seem helpful to the desired 
> measurement. Many test devices can bring a link down at the 
> physical layer.
> 
> J...
> 
> 
> 
> This e-mail contains confidential and / or privileged 
> information belonging to Spirent Communications plc, its 
> affiliates and / or subsidiaries. If you are not the intended 
> recipient, you are hereby notified that any disclosure, 
> copying, distribution and / or the taking of any action based 
> upon reliance on the contents of this transmission is 
> strictly forbidden. If you have received this message in 
> error please notify the sender by return e-mail and delete it 
> from your system. If you require assistance, please contact 
> our IT department at [email-masked].
>

---

From: "Scott Poretsky" <[email-masked]>
Date: Fri, 17 Jul 2009 21:07:50 +0300

Kris,

Test equipment does not drop or discard since they do not route or
switch.  The test equipment is the last hop of the path for a packet
flow.  The test equipment receives the packets and increments counters.

Scott

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Kris Michielsen
Sent: Friday, July 17, 2009 11:46 AM
To: 'McLendon, John'; 'Al Morton'; [email-masked]
Subject: Re: [bmwg] Comment on meth-18.

John,

What do you think of the following suggestion?
OLD:
For Convergence Events caused by the Tester, such as an IGP cost change,
the Tester may start to drop all traffic received from the
Preferred Egress Interface at the Convergence Event Instant to achieve
the same result.

NEW:
For Convergence Events caused by the Tester, such as an IGP cost change,
the Tester may start to discard all traffic received from
the Preferred Egress Interface at the Convergence Event Instant, or may
be able to separately observe packets received from the
Preferred Egress Interface prior to the Convergence Event Instant, to
achieve the same result.

Thanks,
Kris

---

From: "McLendon, John" <[email-masked]>
Date: Fri, 17 Jul 2009 11:34:13 -0700

Hi Kris,
I'm still not sure what "discard" traffic means in terms of a test
device.
If the test device is monitoring both ports simultaneously (and all of
the testers I am knowledgeable about are capable of this), then my
understanding of the test scenario is that the test device either
detects/causes the switchover, captures the time of the Convergence
Event Instant onset, and can then apply the equation once full
convergence occurs. 

If this is correct, then nothing needs to be said about traffic received
by the tester on the Preferred Egress Interface unless one wishes to
measure the delta between the onset of the Convergence Event Instant and
the end of the Convergence Event, i.e. one wants to measure the time
between first packet loss and all traffic stopping on the Preferred
Egress Interface, i.e. the Ta/Tb/Ta'/Tb'/Ta"/Tb" timestamps in Figure 6.
In this case, one might want to state that the tester collects stats on
the both interfaces for the entire test, altho' this is implicit in the
equations and in Figure 6.

I also have one other comment - the current measurement period is
reported in seconds. The test devices I am aware of can capture the
timestamps with millisecond accuracy (presuming there is a minimum
transmission rate of at least two packets per millisecond for each
route). One might want to put in a MAY for this.

Make sense?
J...

-----Original Message-----
From: Kris Michielsen [mailto:[email-masked]] 
Sent: Friday, July 17, 2009 11:46 AM
To: McLendon, John; 'Al Morton'; [email-masked]
Subject: RE: [bmwg] Comment on meth-18.

John,

NEW:

Thanks,
Kris

<DIV><FONT size="1">

E-mail confidentiality.

Spirent Communications plc,
Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom.
Tel No. +44 (0) 1293 767676
Fax No. +44 (0) 1293 767677

Or if within the US,

Spirent Communications,

</FONT></DIV>

---

From: "Scott Poretsky" <[email-masked]>
Date: Fri, 17 Jul 2009 21:39:17 +0300

The magnitude may vary based upon the device under test (DUT) and the
Tester.  It was agreed by the BMWG to specify units 'seconds' so that
any measurement magnitude can be used (including even microseconds).  

Scott

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
McLendon, John
Sent: Friday, July 17, 2009 2:34 PM
To: Kris Michielsen; Al Morton; [email-masked]
Subject: Re: [bmwg] Comment on meth-18.

Hi Kris,

Make sense?
J...

-----Original Message-----
From: Kris Michielsen [mailto:[email-masked]] 
Sent: Friday, July 17, 2009 11:46 AM
To: McLendon, John; 'Al Morton'; [email-masked]
Subject: RE: [bmwg] Comment on meth-18.

John,

NEW:

Thanks,
Kris

<DIV><FONT size="1">

E-mail confidentiality.

Or if within the US,

Spirent Communications,

</FONT></DIV>

---

From: "Kris Michielsen" <[email-masked]>
Date: Mon, 20 Jul 2009 12:27:28 +0200

Hi John,

My view on it is as follows.

Let's take an example of an IGP adjacency loss. When looking at the Forwarding Rate over all routes, we derive the following graph
of Fwd Rate vs time. The Fwd Rate on Preferred Egress and Next-Best Egress are indicated. This is a general case since it doesn't
cause instantaneous traffic drop for all routes on the Convergence Event Instant (CEI).
At CEI the Tester stops rx/tx IGP hellos. At time Ta traffic starts dropping for the first route Rta (observed on Preferred Egress
Interface) and for this example traffic forwarding for Rta starts on Next-Best Egress Interface starts at time Ta'. Depending on the
DUT implementation this can be anywere between First Route Convergence Instant and Convergence Recovery Instant (see also discussion
in meth-18, Section 4).

     ^        
Fwd  |
Rate |-------------                    ............
     |             \                  .
     |              \                .
     |               \              .
     |                \            .
     |.................-.-.-.-.-.-.----------------
     +----+-------+---------------+----------------->
     ^    ^       ^               ^             time
     T0  CEI      Ta              Ta'

     Preferred Egress Interface: ---
     Next-Best Egress Interface: ...

This is also indicated in the graph below of the rx rate for route Rta only.

     ^        
rx   |
rate |-------------               .................
Rta  |            |               .
     |            |               .
     |.............-.-.-.-.-.-.-.-.----------------
     +----+-------+---------------+----------------->
     ^    ^       ^               ^             time
     T0  CEI      Ta              Ta'

     Preferred Egress Interface: ---
     Next-Best Egress Interface: ...

The goal is to derive the time between CEI and Ta' (convergence time of Rta). When only observing traffic on Next-Best Egress
Interface, we only have the dotted graph (...) above.
An obvious way to derive convergence time is to collect a timestamp at CEI and use that for Rate-Derived Method. For the
loss-derived methods, we need to measure the time between T0 and CEI and substract that from the duration of the loss period
observed on the Next-Best Egress Interface.
Another way is to start dropping packets at Convergence Event Instant. Then the convergence time is equal to the loss of
connectivity period duration and we can observe traffic on all DUT egress interfaces.
In the graph below the "=" curve is the 
There are To derived convergence time of route Rta from Next-Best Egress observation only, we can calculate Ta'need the duration of
the time period between T0 (start of traffic) and CEI.

         ^        
         | ====                            ............
Fwd Rate |    |                           .
         |    |                          .
         |    |                         .
         |    |                        .
         | ....=.=.=.=.=.=.=.=.=.=.=.=.================
         |                             
         +----+-----------------------+----------------->
              ^                       ^
             CEI                     Ta'

         ^
         | ====                       .................
rx rate  |    |                       .
Rta      |    |                       .
         | ...=.=.=.=.=.=.=.=.=.=.=.=.=================
         |
         +----+-----------------------+----------------->
              ^                       ^
             CEI                     Ta'

          ^
          |----------------                  ........................
rx rate   |               |                  .
Rtb       |               |                  .
          |................-.-.-.-.-.-.-.-.-.-----------------------
          |
          +---------------+------------------+----------------------->
              ^           ^                  ^
              |          Tb                 Tb'
       Convergence
       Event
       Instant

    =========                                     .......................
            |                                     .
            |                                     .
    ........=.=.=.=.=.=.=.=.=.=.=.=.=.=.=.=.=.=.=.=======================
            ^                                     ^
            |                                    Rtb'
       Convergence
       Event
       Instant

> From: McLendon, John [mailto:[email-masked]] 
> Sent: 17 July 2009 20:34
> I'm still not sure what "discard" traffic means in terms of a 
> test device.
> If the test device is monitoring both ports simultaneously 
> (and all of the testers I am knowledgeable about are capable 
> of this), then my understanding of the test scenario is that 
> the test device either detects/causes the switchover, 
> captures the time of the Convergence Event Instant onset, and 
> can then apply the equation once full convergence occurs. 
> 
> If this is correct, then nothing needs to be said about 
> traffic received by the tester on the Preferred Egress 
> Interface unless one wishes to measure the delta between the 
> onset of the Convergence Event Instant and the end of the 
> Convergence Event, i.e. one wants to measure the time between 
> first packet loss and all traffic stopping on the Preferred 
> In this case, one might want to state that the tester 
> collects stats on the both interfaces for the entire test, 
> altho' this is implicit in the equations and in Figure 6.
> 
> I also have one other comment - the current measurement 
> period is reported in seconds. The test devices I am aware of 
> can capture the timestamps with millisecond accuracy 
> (presuming there is a minimum transmission rate of at least 
> two packets per millisecond for each route). One might want 
> to put in a MAY for this.
> 
> Make sense?
> J...
> 
> For Convergence Events caused by the Tester, such as an IGP 
> Convergence Event Instant to achieve the same result.
> 
> NEW:
> For Convergence Events caused by the Tester, such as an IGP 
> cost change, the Tester may start to discard all traffic 
> Convergence Event Instant, or may be able to separately 
> observe packets received from the Preferred Egress Interface 
> prior to the Convergence Event Instant, to achieve the same result.
> 
> Thanks,
> Kris
> 
> > From: [email-masked] [mailto:[email-masked]] 
> On Behalf 
> > Of McLendon, John
> > Sent: 16 July 2009 00:00
> > the same result."
> traffic. They 
> > may generate or analyze traffic, but a test device dropping traffic 
> > doesn't seem helpful to the desired measurement. Many test 
> devices can 
> > bring a link down at the physical layer.
> > 
> > J...
> > 
> > 
> > 
> > From: [email-masked] [mailto:[email-masked]] 
> On Behalf 
> > Of Al Morton
> > (can't leave this TBD, if there's no obvious material, 
> delete section 
> > ?)
> > 
> > 
> > notified that any disclosure, copying, distribution and / or the 
> > taking of any action based upon reliance on the contents of this 
> > transmission is strictly forbidden. If you have received 
> this message 
> > in error please notify the sender by return e-mail and 
> delete it from 
> > your system. If you require assistance, please contact our IT 
> > department at [email-masked].
> >