Subject: Re: [AVTCORE] WGLC for draft-ietf-avtext-framemarking

From: Magnus Westerlund <[email-masked]>
Date: Tue, 2 Jan 2018 15:34:03 +0100

Hi,

I have reviewed the document and have the below comments. My view is 
that the document needs to address these issues before it is ready to be 
published.

1. Section 1: I think an informational reference to 
https://datatracker.ietf.org/doc/draft-ietf-perc-private-media-framework/ 
would make sense to point to something that provides more information 
than this sentence does:

or extensions that provide participants with private media
    via end-to-end encryption where the switch has no access to media
    decryption keys.

2. Reference to RFC 5285.

This can be updated to use the newly (Nov 2017) published RFC 8285.

3. Section 3:

Regarding source RTP streams only. I wonder if an assumption should be 
made explicit. That it has been assumed that redundancy can be applied 
hop-by-hop to ensure that the amount of errors propagated is kept down.

Maybe not worth noting but, any end-to-end redundancy can run into 
issues unless the source ensures that it breaks the source blocks at any 
Intra frame boundaries.

4. Section 3.

Spelling of Redundancy: Exists two wrongly spelled instances: 
Redunadancy, Redudancy

5. Section 3.1 and 3.2: What is a frame from RTP perspective?

    o  S: Start of Frame (1 bit) - MUST be 1 in the first packet in a
       frame; otherwise MUST be 0.
    o  E: End of Frame (1 bit) - MUST be 1 in the last packet in a frame;
       otherwise MUST be 0.

What defines  a frame, and what underlying assumption hides in this 
definition? Is a frame defined as having a specific RTP timestamp value, 
or are they all the RTP packets in sequential order from a S=1 until an 
packet with E=1? The later would not be compatible with certain RTP 
Payload formats that supports interleaved mode. For example the HEVC 
payload format (RFC7798) in cases where there are multiple NAL units for 
a single frame (access unit). I am not certain that the first definition 
would work either in all cases. But it would at least mean much fewer 
restrictions on how the stream is encoded and sent.

If these bits are going to be useful in most cases, the RTP Packets with 
content for a particular frame needs to be determinable from RTP header 
+ frame marker header extension. So you need to define how the middlebox 
determines which packets are part of the same frame.

6. Section 3.1:

o  The remaining (4 bits) - MUST be 0 for non-scalable streams.

I think this needs to be clarified that the remaining bits are reserved 
for future extensions. Thus they MUST be set to 0, and MUST be
ignored on reception.

7.  Section 3.2:

    o  LID: Layer ID (8 bits) - Identifies the spatial and quality layer
       encoded.  If no scalability is used, this MUST be 0 or omitted.
       When omitted, TL0PICIDX MUST also be omitted.

What is the meaning of "omitted"? I first interpreted this to mean to 
not be sent, i.e. the byte is not included. But, as the same section 
says that L=2, i.e. three bytes of data, it can't be skipped to be sent.

Can you please clarify the meaning of omitted here?

8. Section 3.2:

    o  TL0PICIDX: Temporal Layer 0 Picture Index (8 bits) - Running index
       of base temporal layer 0 frames when TID is 0.  When TID is not 0,
       this indicates a dependency on the given index.  If no scalability
       is used, this MUST be 0 or omitted.  When omitted, LID MUST also
       be omitted.

Isn't value 0 part of the runing index for base temporal layer 0? It is 
not clear to me if one can identify the difference of being dependent on 
the frame with index 0 and when 0 means not used. Can you please clarify.

9. Section 7.1:

I don't see how RFC 5285/8285 is not an normative reference for this 
document.

In addition all the RTP payload formats referenced in Section 
3.2.1.1-3.2.1.4 need to be normative references.

Cheers

Magnus

Den 2017-12-12 kl. 20:30, skrev Jonathan Lennox:
> Hello, all!
>
> This is to announce the Working Group Last Call for draft-ietf-avtext-framemarking-06.
>
> Because of the upcoming winter holidays, this will be an extended last call.
>
> Please review this document and send comments to this mailing list by *Wednesday, January 3, 2018*.
>
> If you review the document and have nothing to add, please let the list know that as well.
>
> Thank you!
>
> Jonathan Lennox
> AVTCORE chair
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

-- 

Magnus Westerlund

----------------------------------------------------------------------
Media Technologies, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 5 Jan 2018 16:01:39 -0500

Here are my comments (as an individual) on draft-ietf-avtext-framemarking.  (I’m not bothering to repeat any issues that Magnus raised; his are all good points.)

Section 3.1:

I think this should say MUST be set to 0 by senders and MUST be ignored by receivers, for the sake of future extensibility (such as Roni’s proposal).

Section 3.2:

   o  I: Independent Frame (1 bit) - MUST be 1 for frames that can be
      decoded independent of temporally prior frames, e.g. intra-frame,
      VPX keyframe, H.264 IDR [RFC6184], H.265 IDR/CRA/BLA/RAP
      [RFC7798]; otherwise MUST be 0.  Note that this bit only signals
      temporal independence, so it can be 1 in spatial or quality
      enhancement layers that depend on temporally co-located layers but
      not temporally prior frames.

One issue that might arise — in a base spatial layer refresh, i.e. this coding structure (Figure 2 from draft-ietf-avtext-lrr):

        ... <--  S1  <--  S1  <--  S1  <--  S1  <-- ...
                  |        |        |        |
                 \/       \/       \/       \/
        ... <--  S0  <--  S0       S0  <--  S0  <-- ...

                  1        2        3        4
the S0 packets of Frame 3 will have the I bit set, even though this is not a true IDR / keyframe.  In order for a receiver to detect a true IDR, it must look for an I bit set on every spatial layer of a frame.  Is this definitely what we want?  If so, Section 3.4 is wrong, and this procedure should be spelled out there.

Alternately, if I=1 and LID=0 means true IDR, this should be stated explicitly; it simplifies IDR detection but would mean that the Figure 2 coding structure cannot be described by frame marking.

   o  D: Discardable Frame (1 bit) - MUST be 1 for frames that can be
      discarded, and still provide a decodable media stream; otherwise
      MUST be 0.

I feel “MUST” for the first half of this definition is too strong here — there are scenarios where this information might not be fully available to the entity creating the frame marking, and so it might want to set the value to 0 in scenarios where it’s uncertain.  I think this should be permitted.

   o  B: Base Layer Sync (1 bit) - MUST be 1 if this frame only depends
      on the base layer; otherwise MUST be 0.  If no scalability is
      used, this MUST be 0.

It should be made clear whether “on the base layer” refers to the base temporal layer, the base spatial layer, or both.  I think the intention is that it mean the base temporal layer, but I’m not sure.

This is another case where the entity creating the frame marking might not have full information, and so might want to set the value to 0 if it’s uncertain. 

Additionally, it’s weird that “If no scalability is used, this MUST be 0”, since conceptually a stream that consists entirely of base layer frames would set this to 1.  In general, I think all the “If no scalability is used” clauses in Section 3.2 should be removed, since they’re redundant with Section 3.1.

How do we want TL0PICIDX values to work across IDR frames?  Some codecs (H.264 SVC) reset their TL0PICID on IDR, others (VP8) insist on continuous values across them.

The former requires an IDRPICID (which I don’t think we want to add) for full correctness, but the latter can make splicing annoying (requiring a splicer to rewrite TL0PICIDX values forever).

Section 3.2.1.5:

I think this section needs to call out the requirement in section 3.2 that LID=0 always indicates the base layer — i.e., it’s not valid for a future LID mapping to be defined in a way that that breaks that invariant.  Otherwise, receivers have no way to tell whether they’ve received the first spatial layer of a frame.

Also, I think it’s necessary — either here, or somewhere else — to require that frame marking only be used with RTP payload formats that follow the usual marker bit rule for video, that a marker bit indicates the last packet of the picture/access unit/temporal unit/whatever.  Otherwise, there’s no way for a receiver to tell whether it’s received the *last* spatial layer of a frame.

Section 3.4.1:

This should mention that because frame marking can only be used with temporally-nested streams, temporal-layer LRR refreshes are unnecessary for frame-marked streams.

Other refreshes can be detected based on the I bit being set for the spatial layers in question (modulo the decision on my point about the I bit, above).

---

From: "Roni Even (A)" <[email-masked]>
Date: Sun, 7 Jan 2018 10:26:34 +0000

Hi,
I agree with Magnus comments.
Some of my own:

1. I think that the names "extension for non-scalable streams" is not correct since it can be used also for scalable streams so suggest changing  to short frame marking and long frame marking extensions.

2. In section 3.2 for omitting LID and TL0PICIDX it looks to me that this is the case where also TID= 0 and B=0 and it looks like the document should recommend to use the short frame marking format (non-scalable streams) extension format in this case (this is also the reason for the name change I suggested above).

Roni Even

-----Original Message-----
From: avt [mailto:[email-masked]] On Behalf Of Jonathan Lennox
Sent: Tuesday, December 12, 2017 9:30 PM
To: IETF AVTCore WG
Subject: [AVTCORE] WGLC for draft-ietf-avtext-framemarking

Hello, all!

Thank you!

Jonathan Lennox
AVTCORE chair

---

From: "Roni Even (A)" <[email-masked]>
Date: Sun, 7 Jan 2018 11:00:18 +0000

Hi,

In section 3.2.13 (H.264 AVC) what is the TID in relation to sub-sequence? RFC6184 does not provide any information and H.264 has sub-seq-layer-num and sub-seq-id but they are not 3 bits (8 and 16)?

Roni Even

-----Original Message-----
From: avt [mailto:[email-masked]] On Behalf Of Jonathan Lennox
Sent: Tuesday, December 12, 2017 9:30 PM
To: IETF AVTCore WG
Subject: [AVTCORE] WGLC for draft-ietf-avtext-framemarking

Hello, all!

Thank you!

Jonathan Lennox
AVTCORE chair