Subject: Re: [Gen-art] Gen-ART review of draft-ietf-ippm-6man-pdm-option: Minor and editorial

From: <[email-masked]>
Date: Fri, 10 Mar 2017 14:44:06 +0000 (UTC)

Al,

I should have remembered that from RFC2330!   

Thanks for pointing that out.  My revised wording inline.

Nalini

--------------------------------------------
On Fri, 3/10/17, MORTON, ALFRED C (AL) <[email-masked]> wrote:

 Subject: RE: Gen-ART review of draft-ietf-ippm-6man-pdm-option:  Minor and editorial
 To: "[email-masked]" <[email-masked]>, "jouni.nospam" <[email-masked]>
 Cc: "General Area Review Team" <[email-masked]>, "[email-masked]" <[email-masked]>, "Michael Ackermann" <[email-masked]>, "Robert Hamilton" <[email-masked]>
 Date: Friday, March 10, 2017, 6:21 AM
 
 > -----Original Message-----
 > From: [email-masked]
 > [mailto:[email-masked]]
 > Sent: Friday, March 10, 2017 8:55 AM
 > To: [email-masked];
 jouni.nospam
 > Cc: General Area Review Team;
 draft-ietf-ippm-6man-pdm-
 > [email-masked];
 Michael Ackermann; Robert Hamilton
 > Subject: Re: Gen-ART review of
 draft-ietf-ippm-6man-pdm-option: Minor
 > and editorial
 > 
 > Jouni,
 > 
 > Our team has been discussing this.   So,
 please let us know what you
 > think about the wording below:
 > 
 > 
 > Wording to be added to section 3.5 Implementation
 Considerations (will
 > make the current considerations 3.5.1)
 >
 ************************************************************************
 > *************************************************
 > 
 > 3.5.2  PDM Timestamps
 > 
 > The PDM timestamps SHOULD be taken as close to the
 network interface as
 > possible.   This will ensure the most
 accurate measurements.
 > 
 > The ideal PDM reference plane or timestamp location
 is:
 > 
 > - Send direction: when the first bit of the first octet
 of the IP header
 > is received for output by the network interface
 > 
 > -  Receive direction when the first bit of the
 first octet of the IP
 > header is received for input by the network interface.
 > 
 > Not all implementations will be able to achieve the
 ideal level of
 > measurement.
 [ACM] 
 
 >Hi Nalini,
 
> It just occurred to me that your IPPM draft can reference
> the IPPM Framework RFC 2330, especially the section on
> clocks and Wiretime:
> https://tools.ietf.org/html/rfc2330#section-10.2
 
> IOW, we've been down this road before,
 
Revised Section 3.5.2
----------------------------

3.5.2  PDM Timestamps
 
The PDM timestamps are intended to isolate wire time from
server or host time.

RFC2330 [RFC2330] "Framework for IP Performance Metrics"
discusses measurement of wire time in section 10.2.

" +    For a given packet P, the 'wire arrival time' of P at H on L is
      the first time T at which any bit of P has appeared at H's
      observational position on L.

 +    For a given packet P, the 'wire exit time' of P at H on L is the
      first time T at which all the bits of P have appeared at H's
      observational position on L."

The timestamping methods used in a PDM implementation 
MUST follow the guidelines provided by RFC2330.

 > 
 > Thanks,
 > 
 > Nalini Elkins
 > Inside Products, Inc.
 > www.insidethestack.com
 > (831) 659-8360
 > 
 > --------------------------------------------
 > On Mon, 3/6/17, jouni.nospam <[email-masked]>
 wrote:
 > 
 >  Subject: Re: Gen-ART review of
 draft-ietf-ippm-6man-pdm-option:  Minor
 > and editorial
 >  To: [email-masked]
 >  Cc: "General Area Review Team" <[email-masked]>,
 "draft-ietf-ippm-
 > [email-masked]"
 <draft-ietf-ippm-6man-pdm-
 > [email-masked]>,
 "Michael Ackermann" <[email-masked]>,
 > "Robert Hamilton" <[email-masked]>
 >  Date: Monday, March 6, 2017, 11:15 AM
 > 
 >  Hi Nalini,
 > 
 >  Thank you. Accurate
 >  timestamping is a non-trivial goal to achieve.
 You should
 >  have the “ideal” reference plane described
 (this usually
 >  is located close or within the network interface
 PHY) and
 >  the exact packet location described what
 constitutes the
 >  point that timestamp refers to (*). While most
 existing
 >  systems plain have no access to required
 resources/functions
 >  to do the timestamping “properly” you try
 your best and
 >  adjust the timestamp with the time you predict it
 takes to
 >  reach the “ideal” reference plane. This of
 course adds
 >  certain amount of error (just check the typical
 operating
 >  system context switch times) but you can get
 around it just
 >  by acknowledging the fact or the trying to
 describe your
 >  acceptable maximum error tolerance.. like +-1ms
 from the
 >  ideal timestamp if the packet were timestamped at
 the PDM
 >  defined reference plane and at the PDM defined
 message
 >  timestamping point. The former is obviously
 easier
 >  approach.
 > 
 >  - JOuni
 > 
 >  (*) example: when the first
 >  bit of the first octet of the IP header (i.e.,
 the first bit
 >  of IP version field) gets output by the PHY to
 the physical
 >  link. And to the receive direction when the first
 bit of the
 >  IP header is received by the network interface
 PHY.. or
 >  something similar. It does not be this strict as
 long as
 >  there is clear understanding what is going on.
 > 
 > 
 >  > On Mar 6, 2017, at
 >  6:41 AM, <[email-masked]>
 >  <[email-masked]>
 >  wrote:
 >  >
 >  > Jouni,
 >  >
 >  > OK.   I
 >  will look.   We are also working with a very
 well
 >  respected FreeBSD developer to create a version
 of FreeBSD
 >  which has PDM in it.
 >  >
 >  > We need to know exactly what choices one
 >  has in the OS stack itself.
 >  >
 >  > Thanks,
 >  >
 >  > Nalini Elkins
 >  > Inside
 >  Products, Inc.
 >  >
 >  www.insidethestack.com
 >  > (831)
 >  659-8360
 >  >
 >  >
 >  > From: jouni.nospam <[email-masked]>
 >  > To: [email-masked]
 > 
 >  > Cc: General Area Review Team <[email-masked]>;
 >  "[email-masked]"
 >  <[email-masked]>;
 >  Michael Ackermann <[email-masked]>;
 >  Robert Hamilton <[email-masked]>
 >  > Sent: Sunday, March 5, 2017 4:54 PM
 >  > Subject: Re: Gen-ART review of
 >  draft-ietf-ippm-6man-pdm-option: Minor and
 editorial
 >  >
 >  > Hi,
 >  >
 >  > Thank you for the
 >  updates. I am fine with them.
 >  >
 >  > I still have one fundamental complaint.
 >  There is no discussion or definition of the
 message
 >  timestamp points. All this nanosecond or better
 accuracy,
 >  and the concern of truncation inaccuracy do not
 really
 >  matter if the message timestamp point varies node
 to node or
 >  even within the node. I suggest taking a look at
 standards
 >  like IEEE 802.1AS, IEEE 1588v2 or IEEE 1722 for
 discussions
 >  of this topic. You really need to have a good
 definition
 >  & discussion of the message timestamp point,
 or justify
 >  that in the text why it is not there.
 >  >
 > 
 >  > Regards,
 >  >
 >     Jouni
 >  >
 >  >
 >  >
 >  > > On Feb 28, 2017, at 7:04 AM, [email-masked]
 >  wrote:
 >  > >
 >  > >
 > 
 >  > >
 >  > >
 >  Jouni,
 >  > >
 >  > >
 >  We have posted a new version which addresses the
 minor and
 >  editorial comments. It is below.  I am waiting
 for your
 >  response to the major comments & then I will
 >  integrate.  As far as I know now, I have
 addressed all
 >  outstanding issues.  My comments to your
 editorial / minor
 >  is below.
 >  > >
 >  >
 >  > https://urldefense.proofpoint.com/v2/url?u=https-
 >
 3A__datatracker.ietf.org_doc_draft-2Dietf-2Dippm-2D6man-2Dpdm-
 > 2Doption_&d=DwIFaQ&c=LFYZ-
 >
 o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=cYzACikChPQGF4cm8f-
 wBWGYz8GduHbmLA55dmSpsuM&s=2eDvhYB5jPh66D2Mj4HHXH6UcQuJtuyffJfrrcNdiyw&e
 > =
 >  > >
 >  > >
 >  > > Nits/editorial comments:
 >  > >
 >  > >> 1)
 >  Section 1.4 numbered list: add missing full
 stops.
 >  > >
 >  > > Actually,
 >  I think it is better to remove the full stops
 from the two
 >  that have it.  These are sentence
 >  >
 >  > fragments.
 >  > >
 >  > >> 2) Section 3.2: remove
 >  > >
 >  > >>
 >  "The 5-tuple consists of
 >  > >
 >  > >>  the source and destination IP
 >  addresses, the source and destination
 >  >
 >  >
 >  > >>  ports, and the upper
 >  layer protocol (ex. TCP, ICMP, etc)."
 >  > >
 >  > >> since
 >  this is unnecessary repetition.
 >  > >
 > 
 >  > >
 >  > > This
 >  is in the section below:
 >  > >
 >  > > Old
 >  > > ----
 > 
 >  > > Packet Sequence Number This
 >  Packet (PSNTP)
 >  > >
 >  > > 16-bit unsigned integer.  This field
 >  will wrap. It is intended for
 >  > > use
 >  while analyzing packet traces.
 >  > >
 > 
 >  > > Initialized at a random number
 >  and incremented monotonically for each
 >  >
 >  > packet of the session flow of the 5-tuple. 
 The
 >  5-tuple consists of
 >  > > the source
 >  and destination IP addresses, the source and
 destination
 >  > > ports, and the upper layer protocol
 >  (ex. TCP, ICMP, etc).  The
 >  > >
 >  random number initialization is intended to make
 it harder
 >  to spoof
 >  > > and insert such packets.
 > 
 >  > >
 >  > > New
 > 
 >  > > ----
 >  > >
 >  Packet Sequence Number This Packet (PSNTP)
 >  > >
 >  > > 16-bit
 >  unsigned integer.  This field will wrap. It is
 intended for
 > 
 >  > > use while analyzing packet
 >  traces.
 >  > >
 >  >
 >  > Initialized at a random number and
 incremented
 >  monotonically for each
 >  > > packet of
 >  the session flow of the 5-tuple.
 >  > >
 > 
 >  > >
 >  > >> 3)
 >  Section 3.2: remove
 >  > >
 >  > >> "Operating systems MUST NOT
 >  implement a single
 >  > >
 >  > >> counter for all
 >  connections."
 >  > >
 >  > >> Seems again like unnecessary
 >  repetition to previous sentence.
 >  > >
 > 
 >  > >
 >  > > This
 >  is in the section on "Packet Sequence Number
 This
 >  Packet (PSNTP)"
 >  > >
 >  > > Old
 >  > > ---
 > 
 >  > > Operating systems MUST implement
 >  a separate packet sequence number
 >  > >
 >  counter per 5-tuple. Operating systems MUST NOT
 implement a
 >  single
 >  > > counter for all
 >  connections.
 >  > >
 >  > > New
 >  > > ---
 > 
 >  > > Operating systems MUST implement
 >  a separate packet sequence number
 >  > >
 >  counter per 5-tuple.
 >  > >
 >  > >
 >  > >> 4)
 >  Section 3.2 again unnecessary repetition of IPv6
 basics that
 >  can be read from RFC2460. Suggest strongly to
 remove:
 >  > >
 >  > >>
 >  "This indicates the following processing
 requirements:
 > 
 >  > >
 >  > >>
 >  00 - skip over this option and continue
 processing the
 >  header.
 >  > >
 >  >
 >  >>  RFC2460 [RFC2460] defines other values
 for the
 >  Option Type field.
 >  > >
 >  > >> These MUST NOT be used in the
 >  PDM."
 >  > >
 >  >
 >  >> and
 >  > >
 >  > >> "The possible values are as
 >  follows:
 >  > >
 >  >
 >  >> 0 - Option Data does not change
 en-route
 >  > >
 >  > >>  1 -
 >  Option Data may change en-route
 >  > >
 > 
 >  > >>  The three high-order bits
 >  described above are to be treated as part
 >  > >
 >  > >>  of
 >  the Option Type, not independent of the Option
 Type.  That
 >  is, a
 >  > >
 >  >
 >  >> particular option is identified by a
 full 8-bit
 >  Option Type, not just
 >  > >
 >  > >> the low-order 5 bits of an Option
 >  Type."
 >  > >
 >  >
 >  >
 >  > > Old
 >  >
 >  > ----
 >  > > Option Type
 >  > >
 >  > > The two
 >  highest-order bits of the Option Type field are
 encoded to
 > 
 >  > > indicate specific processing of
 >  the option; for the PDM destination
 >  >
 >  > option, these two bits MUST be set to 00.
 This
 >  indicates the
 >  > > following
 >  processing requirements:
 >  > >
 >  > > 00 - skip over this option and
 >  continue processing the header.
 >  > >
 > 
 >  > > RFC2460 [RFC2460] defines other
 >  values for the Option Type field.
 >  > >
 >  These MUST NOT be used in the PDM.
 >  >
 >  >
 >  > > In keeping with RFC2460
 >  [RFC2460], the third-highest-order bit of the
 >  > > Option Type specifies whether or not
 >  the Option Data of that option
 >  > >
 >  can change en-route to the packet's final
 destination.
 > 
 >  > >
 >  > > In the
 >  PDM, the value of the third-highest-order bit
 MUST be 0.
 >  The
 >  > > possible values are as
 >  follows:
 >  > >
 >  >
 >  > 0 - Option Data does not change en-route
 >  > >
 >  > > 1 - Option
 >  Data may change en-route
 >  > >
 >  > > The three high-order bits described
 >  above are to be treated as part
 >  > >
 >  of the Option Type, not independent of the Option
 Type.
 >  That is, a
 >  > > particular option is
 >  identified by a full 8-bit Option Type, not just
 >  > > the low-order 5 bits of an Option
 >  Type.
 >  > >
 >  > >
 > 
 >  > >
 >  > > New
 > 
 >  > > ----
 >  > >
 >  Option Type
 >  > >
 >  >
 >  > In keeping with RFC2460[RFC2460], the two
 high order
 >  bits of the Option Type field are encoded to
 >  > > indicate specific processing of the
 >  option; for the PDM destination
 >  > >
 >  option, these two bits MUST be set to 00.
 >  > >
 >  > > The third
 >  high order bit of the Option Type specifies
 whether or not
 >  the Option Data of that option
 >  > >
 >  can change en-route to the packet's final
 destination.
 > 
 >  > >
 >  > > In the
 >  PDM, the value of the third high order bit MUST
 be 0.
 >  > >
 >  > >
 >  > >
 >  > > 5) Section
 >  3.3 same as in comment 4). Suggest strongly
 removing:
 >  > >
 >  > >>
 >  "This follows the order defined in RFC2460
 [RFC2460]
 > 
 >  > >
 >  > >>
 >              IPv6 header
 >  > >
 >  > >>              Hop-by-Hop
 >  Options header
 >  > >
 >  > >>            Destination
 >  Options header  <--------
 >  > >
 >  > >>            Routing header
 > 
 >  > >
 >  > >>
 >          Fragment header
 >  > >
 >  > >>          Authentication
 >  header
 >  > >
 >  >
 >  >>        Encapsulating Security
 Payload header
 > 
 >  > >
 >  > >>
 >        Destination Options header
 <------------
 >  > >
 >  > >>
 >    upper-layer header"
 >  > >
 >  > >
 >  > >
 >  > >> 6) Suggest removing entire
 >  Section 3.4 and moving the following text to
 Section 3.3:
 > 
 >  > >
 >  > >>
 >  "PDM MUST be placed before the ESP header in
 >  > >
 >  > >> order
 >  to work.  If placed before the ESP header, the
 PDM header
 >  will
 >  > >
 >  >
 >  >> flow in the clear over the network thus
 allowing
 >  gathering of
 >  > >
 >  > >> performance and diagnostic data
 >  without sacrificing security."
 >  >
 >  >
 >  > >
 >  > >
 >  ESP processing with PDM was changed because of
 comments from
 >  security reviewer.
 >  > >
 >  > >
 >  > >
 >  > >> 7) Section 3.6 suggest removing
 >  the following text. I see no value it would add
 to what has
 >  already been said:
 >  > >
 >  > >> "As with all other
 >  destination options extension headers, the PDM
 is
 >  > >
 >  > >>  for
 >  destination nodes only. As specified above,
 intermediate
 >  devices
 >  > >
 >  >
 >  >>  MUST neither set nor modify this
 field."
 >  > >
 >  > >
 >  > > Done
 >  > >
 >  > >
 >  > >> 8)
 >  Section 3.6 suggest removing the following
 5-tuple text as
 >  it has already been described earlier in Section
 2:
 >  > >
 >  > >>
 >  "The 5-tuple is:
 >  > >
 >  > >>  SADDR : IP address of the
 >  sender SPORT : Port for sender DADDR : IP
 >  > >
 >  > >>
 >  address of the destination DPORT : Port for
 destination
 >  PROTC :
 >  > >
 >  >
 >  >>  Protocol for upper layer (ex. TCP,
 UDP,
 >  ICMP)"
 >  > >
 >  >
 >  > Done
 >  > >
 >  >
 >  >
 >  > >> 9) Sections 4.2 and 4.3
 >  suggest removing them entirely. I see what value
 these
 >  sections add. I acknowledge they are good
 >  > >> to know information of timer
 >  hardware implementation difference but do not
 really add
 >  value on the on-wire encoding of the PDM option.
 >  > >
 >  > >> 10)
 >  Section 4.4 suggest removing the entire section.
 Time Base
 >  was already described in detail enough in Section
 3.2.
 >  > >
 >  > >> 11)
 >  Section 4.5 time base for picoseconds is 11 not
 00.
 >  > >
 >  > >> 12)
 >  Section 4.5 suggest removing the following text,
 since it
 >  does not add any more clarity to what has already
 been said
 >  in my opinion. This is because all the examples
 follow nice
 >  nybble increment in scaling:
 >  > >
 >  > >> "Sample binary values (high
 >  order 16 bits taken)
 >  > >
 >  > >>  1 psec            1
 > 
 >    0001
 >  > >
 >  >
 >  >>  1 nsec          3E8
 >                      0011 1110 1000
 >  > >
 >  > >>  1
 >  usec        F4240
 >  1111 0100 0010 0100 0000
 >  > >
 >  > >>  1 msec    3B9ACA00
 >    0011 1011 1001 1010 1100 1010 0000 0000
 >  > >
 >  > >>  1
 >  sec    E8D4A51000 1110 1000 1101 0100 1010 0101
 0001 0000
 >  0000 0000"
 >  > >
 >  > >> 12) Section 4.6 I do not
 >  understand why this section is here. I strongly
 suggest
 >  removing it. Sections 4.5 and 3.2 already
 describe how I
 >  would encode the delta time using scaling as a
 separate
 >  fields not embedded (option fields ScaleDTLR and
 ScaleDTLS).
 >  Did I misunderstand something here?
 >  >
 >  >
 >  > >
 >  > >
 >  Points 9 - 12 refer to the timing calculations. 
 The
 >  timebase has been fixed to be attoseconds.
 >  > > The entire timing section has been
 >  re-written.  It has also been moved to an
 appendix.
 >  > > Please let us know if it this
 >  addresses your concerns.
 >  > >
 >  > >
 >  > >
 >  > >> 13) Section 5 suggest removing
 >  the following text because of it repeating what
 has already
 >  been said earlier:
 >  > >
 >  > >> "Each packet, in addition to
 >  the PDM contains information on the
 >  >
 >  >
 >  > >> sender and receiver. As
 >  discussed before, a 5-tuple consists of:
 >  > >
 >  > >>
 >  SADDR : IP address of the sender
 >  > >
 > 
 >  > >>  SPORT : Port for sender
 >  > >
 >  > >>
 >  DADDR : IP address of the destination
 >  >
 >  >
 >  > >>  DPORT : Port for
 >  destination
 >  > >
 >  >
 >  >>  PROTC : Protocol for upper layer (ex.
 TCP, UDP,
 >  ICMP)
 >  > >
 >  >
 >  >> It should be understood that the packet
 >  identification information is
 >  > >
 >  > >> in each packet. We will not
 >  repeat that in each of the following
 >  >
 >  >
 >  > >> steps."
 >  > >
 >  > >
 >  > > This is now in Appendix B.  It has
 >  been removed from that section.
 >  > >
 > 
 >  > >
 >  > >
 >  > >> 14) Section 5.3 suggest merging
 >  the following text into one example and do
 necessary
 >  rewording. There is no need to do the same
 >  > >>  calculation twice on almost
 >  adjacent lines:
 >  > >
 >  > >> "Sending time : packet 2 -
 >  receive time : packet 1
 >  > >
 >  > >> We will call the result of this
 >  calculation: Delta Time Last Received
 >  >
 >  >
 >  > >> (DELTATLR)
 >  > >
 >  > >> That
 >  is:
 >  > >
 >  >
 >  >> Delta Time Last Received = (Sending
 time: packet 2
 >  - receive time: packet 1)"
 >  > >
 > 
 >  > > Done
 >  > >
 > 
 >  > >
 >  > >>
 >  15) Expand RTT and PSN on their first use.
 >  > >
 >  > > Done
 >  > >
 >  > > Thanks,
 >  > >
 >  > > Nalini
 >  Elkins
 >  > > Inside Products, Inc.
 >  > > www.insidethestack.com
 >  > > (831) 659-8360
 >  >
 > 
 >  >
 >

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Fri, 10 Mar 2017 15:05:12 +0000

Hi Nalini,

Let me suggest a few quick clarifications in a top-post:

I liked your original first sentence, with the SHOULD,
that's really all we can ask. The last sentence was 
quite practical, too.

Also, wiretime is a concept/notion, the ideal we try
to get close to achieving in practice. And we need to 
say what H and L are, so, here's my proposal...

Re-Revised Section 3.5.2

3.5.2  PDM Timestamps
 
The PDM timestamps SHOULD be taken as close to the
network interface as possible.
The PDM timestamps are intended to isolate wire time from
server or host time, but may necessarily attribute some
host processing time to network latency.

RFC2330 [RFC2330] "Framework for IP Performance Metrics"
describes two notions of wire time in section 10.2.
These notions are only defined in terms of an Internet host H 
observing an Internet link L at a particular location:

 +    For a given packet P, the 'wire arrival time' of P at H on L is
      the first time T at which any bit of P has appeared at H's
      observational position on L.

Not all implementations will be able to achieve the
ideal level of measurement.

Al

> Sent: Friday, March 10, 2017 9:44 AM
> To: [email-masked]; jouni.nospam; MORTON, ALFRED C
> (AL)
> Cc: General Area Review Team; draft-ietf-ippm-6man-pdm-
> [email-masked]; Michael Ackermann; Robert Hamilton
>  To: "[email-masked]"
> <[email-masked]>, "jouni.nospam"
> <[email-masked]>
>  Cc: "General Area Review Team" <[email-masked]>, "draft-ietf-ippm-
> [email-masked]" <draft-ietf-ippm-6man-pdm-
> [email-masked]>, "Michael Ackermann" <[email-masked]>,
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_rfc2330-23section-2D10.2&d=DwIFaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=IPrnGy4K3nr5XaQh24815q0ZAYu
> LInXigPyegsEXaM0&s=3hPio_KWWSEZw3NhtTfAbT1mXSXhz_I2yP-WEL58K8I&e=
>

---

From: "Ackermann, Michael" <[email-masked]>
Date: Fri, 10 Mar 2017 15:31:57 +0000

Well said Al!
Especially the last sentence that essentially means that some of this will be implementation dependent.  
So I think these guidelines and intentions are probably about as far as we can go in this doc.   
Thanks
Mike

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Friday, March 10, 2017 10:05 AM
To: [email-masked]; jouni.nospam <[email-masked]>
Cc: General Area Review Team <[email-masked]>; [email-masked]; Ackermann, Michael <[email-masked]>; Robert Hamilton <[email-masked]>
Subject: RE: Gen-ART review of draft-ietf-ippm-6man-pdm-option: Minor and editorial

Hi Nalini,

I liked your original first sentence, with the SHOULD, that's really all we can ask. The last sentence was quite practical, too.

Also, wiretime is a concept/notion, the ideal we try to get close to achieving in practice. And we need to say what H and L are, so, here's my proposal...

Re-Revised Section 3.5.2

3.5.2  PDM Timestamps
 
The PDM timestamps SHOULD be taken as close to the network interface as possible.
The PDM timestamps are intended to isolate wire time from server or host time, but may necessarily attribute some host processing time to network latency.

RFC2330 [RFC2330] "Framework for IP Performance Metrics"
describes two notions of wire time in section 10.2.
These notions are only defined in terms of an Internet host H observing an Internet link L at a particular location:

Not all implementations will be able to achieve the ideal level of measurement.

Al

>  > From: [email-masked]  > 
>  > Sent: Friday, March 10, 2017 8:55 AM  > To: 
> [email-masked];  jouni.nospam  > Cc: General Area 
> Review Team;
>  draft-ietf-ippm-6man-pdm-option: Minor  > and editorial  >  > Jouni,  
> >  > Our team has been discussing this.   So,  please let us know what 
> you  > think about the wording below:
>  >
>  >
>  > Wording to be added to section 3.5 Implementation  Considerations 
> (will  > make the current considerations 3.5.1)  >
> 
> **********************************************************************
> **  > *************************************************
>  >
>  > 3.5.2  PDM Timestamps
>  >
>  > The PDM timestamps SHOULD be taken as close to the  network 
> interface as  > possible.   This will ensure the most  accurate 
> measurements.
>  >
>  > - Send direction: when the first bit of the first octet  of the IP 
> header  > is received for output by the network interface  >  > -  
> Receive direction when the first bit of the  first octet of the IP  > 
>  > Not all implementations will be able to achieve the  ideal level of  
> > measurement.
>  >
>  [ACM]
> 
>  >Hi Nalini,
> 
> > It just occurred to me that your IPPM draft can reference the IPPM 
> > Framework RFC 2330, especially the section on clocks and Wiretime:
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=IPrnGy4K3nr5XaQh24815q0ZA
> Yu LInXigPyegsEXaM0&s=3hPio_KWWSEZw3NhtTfAbT1mXSXhz_I2yP-WEL58K8I&e=
> 
> The PDM timestamps are intended to isolate wire time from server or 
> host time.
> 
> The timestamping methods used in a PDM implementation MUST follow the 
> guidelines provided by RFC2330.
> 
> 
> 
>  >
>  > Thanks,
>  >
>  > Nalini Elkins
>  > Inside Products, Inc.
>  > www.insidethestack.com
>  > (831) 659-8360
>  >
>  draft-ietf-ippm-6man-pdm-option:  Minor  > and editorial  >  To: 
> [email-masked]  >  Cc: "General Area Review Team" 
> <[email-masked]>,
>  "draft-ietf-ippm-
>  "Michael Ackermann" <[email-masked]>,  > "Robert Hamilton" 
> <[email-masked]>  >  Date: Monday, March 6, 2017, 11:15 AM  >  >  
> Hi Nalini,  >  >  Thank you. Accurate  >  timestamping is a 
> non-trivial goal to achieve.
>  You should
>  >  have the “ideal” reference plane described  (this usually  >  is 
> located close or within the network interface
>  PHY) and
>  >  the exact packet location described what  constitutes the  >  
> point that timestamp refers to (*). While most  existing  >  systems 
> plain have no access to required  resources/functions  >  to do the 
> timestamping “properly” you try  your best and  >  adjust the 
> timestamp with the time you predict it  takes to  >  reach the “ideal” 
> reference plane. This of  course adds  >  certain amount of error 
> (just check the typical  operating  >  system context switch times) 
> but you can get  around it just  >  by acknowledging the fact or the 
> trying to  describe your  >  acceptable maximum error tolerance.. like 
> +-1ms  from the  >  ideal timestamp if the packet were timestamped at  
> the PDM  >  defined reference plane and at the PDM defined  message  >  
> timestamping point. The former is obviously  easier  >  approach.
>  >
>  >  - JOuni
>  >
>  >  bit of the first octet of the IP header (i.e.,  the first bit  >  
> of IP version field) gets output by the PHY to  the physical  >  link. 
> And to the receive direction when the first  bit of the  >  IP header 
> is received by the network interface  PHY.. or  >  something similar. 
> It does not be this strict as  long as  >  there is clear 
> understanding what is going on.
>  >
>  >
>  >  > On Mar 6, 2017, at
>  >  will look.   We are also working with a very  well  >  respected 
> FreeBSD developer to create a version  of FreeBSD  >  which has PDM in 
> it.
>  >  >
>  >  > We need to know exactly what choices one  >  has in the OS stack 
> itself.
>  >  >
>  >  > Thanks,
>  >  >
>  >  > Nalini Elkins
>  >  > Inside
>  >  Products, Inc.
>  >  >
>  >  www.insidethestack.com
>  >  > (831)
>  >  659-8360
>  >  >
>  >  >
>  >  > From: jouni.nospam <[email-masked]>  >  > To: 
> [email-masked]  >  >  > Cc: General Area Review Team 
> <[email-masked]>;  >  "[email-masked]"
>  >  Michael Ackermann <[email-masked]>;  >  Robert Hamilton 
> <[email-masked]>  >  > Sent: Sunday, March 5, 2017 4:54 PM  >  > 
> Subject: Re: Gen-ART review of  >  draft-ietf-ippm-6man-pdm-option: 
> Minor and  editorial  >  >  >  > Hi,  >  >  >  > Thank you for the  >  
>  >  There is no discussion or definition of the  message  >  timestamp 
> points. All this nanosecond or better  accuracy,  >  and the concern 
> of truncation inaccuracy do not  really  >  matter if the message 
> timestamp point varies node  to node or  >  even within the node. I 
> suggest taking a look at  standards  >  like IEEE 802.1AS, IEEE 1588v2 
> or IEEE 1722 for  discussions  >  of this topic. You really need to 
> have a good  definition  >  & discussion of the message timestamp 
> point,  or justify  >  that in the text why it is not there.
>  >  >
>  >
>  >  > Regards,
>  >  >
>  >     Jouni
>  >  >
>  >  >
>  >  >
>  >  We have posted a new version which addresses the  minor and  >  
> editorial comments. It is below.  I am waiting  for your  >  response 
> to the major comments & then I will  >  integrate.  As far as I know 
> now, I have  addressed all  >  outstanding issues.  My comments to 
> your  editorial / minor  >  is below.
>  >  > >
>  >  >
>  >  Section 1.4 numbered list: add missing full  stops.
>  >  > >
>  >  > > Actually,
>  >  I think it is better to remove the full stops  from the two  >  
> that have it.  These are sentence  >  >  >  > fragments.
>  >  > >
>  >  > >>  the source and destination IP  >  addresses, the source and 
> destination  >  >  >  >  >  > >>  ports, and the upper  >  layer 
> protocol (ex. TCP, ICMP, etc)."
>  >  > >
>  >  > >> since
>  >  > > 16-bit unsigned integer.  This field  >  will wrap. It is 
> intended for  >  > > use  >  while analyzing packet traces.
>  >  > >
>  >
>  >  > > Initialized at a random number  >  and incremented 
> monotonically for each  >  >  >  > packet of the session flow of the 
> 5-tuple.
>  The
>  >  5-tuple consists of
>  >  > > the source
>  >  and destination IP addresses, the source and  destination  >  > > 
> ports, and the upper layer protocol  >  (ex. TCP, ICMP, etc).  The  >  
> > >  >  random number initialization is intended to make  it harder  >  
> to spoof  >  > > and insert such packets.
>  >
>  >  > >
>  >  > > New
>  >
>  >  > > ----
>  >  > >
>  >  Packet Sequence Number This Packet (PSNTP)  >  > >  >  > > 16-bit  
> >  unsigned integer.  This field will wrap. It is  intended for  >  >  
> > > use while analyzing packet  >  traces.
>  >  > >
>  >  >
>  >  > Initialized at a random number and  incremented  >  
> monotonically for each  >  > > packet of  >  the session flow of the 
> 5-tuple.
>  >  > >
>  >
>  >  > >
>  >  > >> 3)
>  >  Section 3.2: remove
>  >  > >
>  >  is in the section on "Packet Sequence Number  This  >  Packet 
> (PSNTP)"
>  >  > >
>  >  > > Old
>  >  > > ---
>  >
>  >  > > Operating systems MUST implement  >  a separate packet 
> sequence number  >  > >  >  counter per 5-tuple. Operating systems 
> MUST NOT  implement a  >  single  >  > > counter for all  >  
> connections.
>  >  > >
>  >  > > New
>  >  > > ---
>  >
>  >  > > Operating systems MUST implement  >  a separate packet 
> sequence number  >  > >  >  counter per 5-tuple.
>  >  > >
>  >  > >
>  >  > >> 4)
>  >  Section 3.2 again unnecessary repetition of IPv6  basics that  >  
>  >  00 - skip over this option and continue  processing the  >  
> header.
>  >  > >
>  >  >
>  >  >>  RFC2460 [RFC2460] defines other values  for the  >  Option 
> Type field.
>  >  > >
>  >  > >> These MUST NOT be used in the  >  PDM."
>  >  > >
>  >  >
>  >  >> and
>  >  > >
>  >  >> 0 - Option Data does not change  en-route  >  > >  >  > >>  1 -  
> >  Option Data may change en-route  >  > >  >  >  > >>  The three 
> high-order bits  >  described above are to be treated as part  >  > >  
> >  > >>  of  >  the Option Type, not independent of the Option  Type.  
> That  >  is, a  >  > >  >  >  >  >> particular option is identified by 
> a  full 8-bit  >  Option Type, not just  >  > >  >  > >> the low-order 
> 5 bits of an Option  >  Type."
>  >  > >
>  >  >
>  >  >
>  >  > > Old
>  >  >
>  >  > ----
>  >  > > Option Type
>  >  > >
>  >  > > The two
>  >  highest-order bits of the Option Type field are  encoded to  >  >  
> > > indicate specific processing of  >  the option; for the PDM 
> destination  >  >  >  > option, these two bits MUST be set to 00.
>  This
>  >  indicates the
>  >  > > following
>  >  processing requirements:
>  >  > >
>  >  > > 00 - skip over this option and  >  continue processing the 
> header.
>  >  > >
>  >
>  >  > > RFC2460 [RFC2460] defines other  >  values for the Option Type 
> field.
>  >  > >
>  >  [RFC2460], the third-highest-order bit of the  >  > > Option Type 
> specifies whether or not  >  the Option Data of that option  >  > >  >  
> can change en-route to the packet's final  destination.
>  >
>  >  > >
>  >  > > In the
>  >  PDM, the value of the third-highest-order bit  MUST be 0.
>  >  The
>  >  > 0 - Option Data does not change en-route  >  > >  >  > > 1 - 
> Option  >  Data may change en-route  >  > >  >  > > The three 
> high-order bits described  >  above are to be treated as part  >  > >  
> >  of the Option Type, not independent of the Option  Type.
>  >  That is, a
>  >  > > particular option is
>  >  identified by a full 8-bit Option Type, not just  >  > > the 
> low-order 5 bits of an Option  >  Type.
>  >  > >
>  >  > >
>  >
>  >  > >
>  >  > > New
>  >
>  >  > > ----
>  >  > >
>  >  Option Type
>  >  > >
>  >  >
>  >  > In keeping with RFC2460[RFC2460], the two  high order  >  bits 
> of the Option Type field are encoded to  >  > > indicate specific 
> processing of the  >  option; for the PDM destination  >  > >  >  
>  >  high order bit of the Option Type specifies  whether or not  >  
> the Option Data of that option  >  > >  >  can change en-route to the 
> packet's final  destination.
>  >
>  >  > >
>  >  > > In the
>  >  PDM, the value of the third high order bit MUST  be 0.
>  >  > >
>  >  > >
>  >  > >
>  >  > > 5) Section
>  >  "This follows the order defined in RFC2460  [RFC2460]  >  >  > >  
> >  > >>  >              IPv6 header  >  > >  >  > >>              
> Hop-by-Hop  >  Options header  >  > >  >  > >>            Destination  
> >  Options header  <--------  >  > >  >  > >>            Routing 
> header  >  >  > >  >  > >>  >          Fragment header  >  > >  >  > 
> >>          Authentication  >  header  >  > >  >  >  >  >>        
> Encapsulating Security  Payload header  >  >  > >  >  > >>  >        
>  >  Section 3.4 and moving the following text to  Section 3.3:
>  >
>  >  > >
>  >  > >>
>  >  "PDM MUST be placed before the ESP header in  >  > >  >  > >> 
> order  >  to work.  If placed before the ESP header, the  PDM header  
> >  will  >  > >  >  >  >  >> flow in the clear over the network thus  
> allowing  >  gathering of  >  > >  >  > >> performance and diagnostic 
> data  >  without sacrificing security."
>  >  >
>  >  >
>  >  > >
>  >  > >
>  >  ESP processing with PDM was changed because of  comments from  >  
> security reviewer.
>  >  > >
>  >  > >
>  >  > >
>  >  > >> 7) Section 3.6 suggest removing  >  the following text. I see 
> no value it would add  to what has  >  already been said:
>  >  > >
>  >  > >> "As with all other
>  >  destination options extension headers, the PDM  is  >  > >  >  > 
> >>  for  >  destination nodes only. As specified above,  intermediate  
> >  devices  >  > >  >  >  >  >>  MUST neither set nor modify this  
> field."
>  >  > >
>  >  > >
>  >  > > Done
>  >  > >
>  >  > >
>  >  > >> 8)
>  >  Section 3.6 suggest removing the following  5-tuple text as  >  it 
> has already been described earlier in Section
>  2:
>  >  > >
>  >  > >>
>  >  "The 5-tuple is:
>  >  > >
>  >  sender SPORT : Port for sender DADDR : IP  >  > >  >  > >>  >  
> address of the destination DPORT : Port for  destination  >  PROTC :
>  >  > >
>  >  >
>  >  >>  Protocol for upper layer (ex. TCP,  UDP,  >  ICMP)"
>  >  > >
>  >  >
>  >  > Done
>  >  > >
>  >  >
>  >  >
>  >  suggest removing them entirely. I see what value  these  >  
> sections add. I acknowledge they are good  >  > >> to know information 
> of timer  >  hardware implementation difference but do not  really add  
>  >  was already described in detail enough in Section  3.2.
>  >  > >
>  >  > >> 11)
>  >  Section 4.5 time base for picoseconds is 11 not  00.
>  >  > >
>  >  > >> 12)
>  >  Section 4.5 suggest removing the following text,  since it  >  
> does not add any more clarity to what has already  been said  >  in my 
> opinion. This is because all the examples  follow nice  >  nybble 
> increment in scaling:
>  >  > >
>  >                      0011 1110 1000  >  > >  >  > >>  1  >  usec        
> F4240  >  1111 0100 0010 0100 0000  >  > >  >  > >>  1 msec    
> 3B9ACA00  >    0011 1011 1001 1010 1100 1010 0000 0000  >  > >  >  > 
> >>  1  >  sec    E8D4A51000 1110 1000 1101 0100 1010 0101
>  0001 0000
>  >  0000 0000"
>  >  > >
>  >  understand why this section is here. I strongly  suggest  >  
> removing it. Sections 4.5 and 3.2 already  describe how I  >  would 
> encode the delta time using scaling as a  separate  >  fields not 
> embedded (option fields ScaleDTLR and  ScaleDTLS).
>  >  > > The entire timing section has been  >  re-written.  It has 
> also been moved to an  appendix.
>  >  > >> 13) Section 5 suggest removing  >  the following text because 
> of it repeating what  has already  >  been said earlier:
>  >  > >
>  >  the PDM contains information on the  >  >  >  >  >  > >> sender 
> and receiver. As  >  discussed before, a 5-tuple consists of:
>  >  > >
>  >  > >>
>  >  DADDR : IP address of the destination  >  >  >  >  >  > >>  DPORT 
> : Port for  >  destination  >  > >  >  >  >  >>  PROTC : Protocol for 
> upper layer (ex.
>  TCP, UDP,
>  >  ICMP)
>  >  > >
>  >  >
>  >  >> It should be understood that the packet  >  identification 
> information is  >  > >  >  > >> in each packet. We will not  >  repeat 
> that in each of the following  >  >  >  >  >  > >> steps."
>  >  > >
>  >  > >
>  >  > > This is now in Appendix B.  It has  >  been removed from that 
> section.
>  >  > >
>  >
>  >  > >
>  >  > >
>  >  > >> 14) Section 5.3 suggest merging  >  the following text into 
> one example and do  necessary  >  rewording. There is no need to do 
> the same  >  > >>  calculation twice on almost  >  adjacent lines:
>  >  > >
>  >  > >> We will call the result of this  >  calculation: Delta Time 
> Last Received  >  >  >  >  >  > >> (DELTATLR)  >  > >  >  > >> That  >  
> is:
>  >  > >
>  >  >

The information contained in this communication is highly confidential and is intended solely for the use of the individual(s) to whom this communication is directed. If you are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or distribution of this information is prohibited. Please notify the sender, by electronic mail or telephone, of any unintended receipt and delete the original message without making any copies.
 
 Blue Cross Blue Shield of Michigan and Blue Care Network of Michigan are nonprofit corporations and independent licensees of the Blue Cross and Blue Shield Association.

---

From: "Hamilton, Robert" <[email-masked]>
Date: Fri, 10 Mar 2017 15:46:54 +0000

Agreed. I think this is as precise and concise as we can be.
This is excellent!

R;

>From: Ackermann, Michael [mailto:[email-masked]]
>Sent: Friday, March 10, 2017 10:32 AM
>To: MORTON, ALFRED C (AL); [email-masked];
>jouni.nospam
>[email-masked]; Hamilton, Robert
>Cc: General Area Review Team <[email-masked]>; draft-ietf-ippm-6man-
>[email-masked]; Ackermann, Michael <[email-masked]>;
>I liked your original first sentence, with the SHOULD, that's really all we can
>ask. The last sentence was quite practical, too.
>
>Also, wiretime is a concept/notion, the ideal we try to get close to achieving in
>practice. And we need to say what H and L are, so, here's my proposal...
>
>Re-Revised Section 3.5.2
>The PDM timestamps are intended to isolate wire time from server or host
>time, but may necessarily attribute some host processing time to network
>latency.
>
>These notions are only defined in terms of an Internet host H observing an
>Internet link L at a particular location:
>
>> <[email-masked]>  >  Date: Monday, March 6, 2017, 11:15 AM  >  > Hi
>> Nalini,  >  >  Thank you. Accurate  >  timestamping is a non-trivial
>> goal to achieve.
>>  You should
>>  >  the exact packet location described what  constitutes the  > point
>> that timestamp refers to (*). While most  existing  >  systems plain
>> have no access to required  resources/functions  >  to do the
>wBWGYz8GduHbmLA55dmSpsuM&s=2eDvhYB5jPh66D2Mj4HHXH6UcQu
>JtuyffJfrrcNdiyw
>> &e
>>  > =
>>  >  > >
>>  >  > >
>>  >  > Initialized at a random number and  incremented  > monotonically
>> for each  >  > > packet of  >  the session flow of the 5-tuple.
>>  >  > >
>>  >
>>  >  > >
>>  >  > >> 3)
>>  >  Section 3.2: remove
>>  >  > >
>>  >  00 - skip over this option and continue  processing the  > header.
>>  >  > >
>>  >  >
>>  >  high order bit of the Option Type specifies  whether or not  > the
>> Option Data of that option  >  > >  >  can change en-route to the
>>  >  Section 4.5 suggest removing the following text,  since it  > does
>> not add any more clarity to what has already  been said  >  in my
>The information contained in this communication is highly confidential and
>is intended solely for the use of the individual(s) to whom this
>communication is directed. If you are not the intended recipient, you are
>hereby notified that any viewing, copying, disclosure or distribution of this
>information is prohibited. Please notify the sender, by electronic mail or
>telephone, of any unintended receipt and delete the original message
>without making any copies.
>
> Blue Cross Blue Shield of Michigan and Blue Care Network of Michigan are
>nonprofit corporations and independent licensees of the Blue Cross and
>Blue Shield Association.

Confidentiality Notice: This electronic message transmission, including any attachment(s), may contain confidential, proprietary, or privileged information from Chemical Abstracts Service ("CAS"), a division of the American Chemical Society ("ACS"). If you have received this transmission in error, be advised that any disclosure, copying, distribution, or use of the contents of this information is strictly prohibited. Please destroy all copies of the message and contact the sender immediately by either replying to this message or calling 614-447-3600.