Subject: [manet] Mirja Kühlewind's Discuss on draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 08 May 2017 16:28:11 -0700

Mirja Kühlewind has entered the following ballot position for
draft-ietf-manet-olsrv2-multipath-12: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-manet-olsrv2-multipath/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

The following text in section 4 seems to indicate that scheduling is done
on a per-packet basis:
"When there is a
   datagram to be sent to a destination, the source router acquires a
   path from the Multi-path Routing Set (MAY be Round-Robin, or other
   scheduling algorithms)."
This seems not appropriate as e.g. TCP packets routed on links with
largely different delays may suffer performance. ECMP usually hashes the
5-tuple or 6-tuple (incl. DiffServ Codepoint) to setup state and routes
all packets belonging to the same flow on the same route. I recommend to
apply the same here.

Also related is this text in section 8.4 that should explain Round-Robin
on a per flow basis instead. Further this should only be an example
scheduling alogirthm while text belong seems to assume that Round-Robin
is always used.
"If a matching Multi-path Routing Tuple is obtained, the Path Tuples
   of the Multi-path Routing Tuple are applied to the datagrams using
   Round-robin scheduling.  For example, there are 2 path Tuples
   (Path-1, Path-2) for destination router D. A series of datagrams
   (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent router D.
   Path-1 is then chosen for Packet-1, Path-2 for Packet-2, Path-1 for
   Packet 3, etc.  Other path scheduling mechanisms are also possible
   and will not impact the interoperability of different
   implementations."

Related is this text in section 8.4.:
"If datagrams without source routing header need to be forwarded using
   multiple paths (for example, based on the information of DiffServ
   Code Point [RFC2474])"
RFC2474 does not specify any application requirements on multipath use
and as such the DiffServe field should not be used to determine if the
flow can be routed on multiple paths. The ability to profit from
multipath routing depends not only on the application and protocols used
but also on the characteristics of the multipath link(s); so it's hard to
make any implicit assumptions here. However, if routing would only be
recommended on a per-flow basis this problem does not occur and the
brackets above could be remove. Further, if routed on a per flow basis
would be done, DiffServ could actually be used to decide which path to
use, if e.g. one path has a lower delay, but that seem to need further
discussion as well.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Minor comments/questions:

1) section 8.4: this sentence is not clear:
"It is RECOMMENDED to use MTU sizes considering the source routing
   header to avoid fragmentation."
MAYBE
"It is NOT RECOMMENDED to fragment the IP packet if the packet with the
source routing header would  exceed the minimum MTU along the path. In
this case source routing and therefore the additional path calculated by
MP-OLSRv2 SHOULD NOT be used."

2) section 9:
"For IPv6 networks, it MUST
      be set to 0, i.e., no constraint on maximum number of hops."
Why is that?

3) Not sure why section 12.1. is there? Can this be removed?

---

From: Jiazi Yi <[email-masked]>
Date: Wed, 10 May 2017 00:51:17 +0200

Dear Mirja, 

Thanks very much for your review and comments. Please find the reply inline: 

>   implementations.”

In fact, the per-packet scheduling is an intentional choice because:

1. The aimed scenario is mobile ad hoc networks, which have high packet loss by nature. One of the most important reasons is route failure due to mobility — in such case, if a per-flow scheduling is applied, the whole flow will lost. On the other hand, if we send the packets in disjoint paths (not necessarily equal cost), we can still make use partial information of the flow, or even reconstruct the flow with erasure coding. This is especially interesting for video/audio streaming. 

2. In such kind of lossy networks, the traditional TCP performs very poor [1]. So as far as I know, TCP is not very popular in ad hoc networks. On the other hand, based on our study, the multi-path routing can actually reduce the overall jitter [2]. 

We understand your concern on this issue (and you can imagine that you are not the first one who raises this ;). It is also something that we want to have further experience from this *experimental* draft, as we stated in the “experiments to be conducted” section:

	• Different path-selection schedulers. By default, round-robin scheduling is used to select a path to be used for datagrams. In some scenarios, weighted scheduling can be considered: for example, the paths with lower metrics (i.e., higher quality) can transfer more datagrams compared to paths with higher metrics. 
	• The impacts of the delay variation due to multipath routing. [RFC2991] brings out some concerns of multipath routing, especially variable latencies. Although current experiment results show that multipath routing can reduce the jitter in dynamic scenarios, some transport protocols or applications may be sensitive to the datagram re-ordering. 
	• The disjoint multipath protocol has interesting application with erasure coding, especially for services like video/audio streaming [WPMC11]. The combination of erasure coding mechanisms and this extension is thus encouraged.

[1] Performance Evaluation of TCP over Mobile Ad-hoc Networks https://arxiv.org/pdf/1002.2189.pdf <networkshttps://arxiv.org/pdf/1002.2189.pdf> 
[2] Multipath optimized link state routing for mobile ad hoc networks", In Elsevier Ad Hoc Journal, vol.9, n. 1, 28-47, January, 2011.

> use, if e.g. one path has a lower delay, but that seem to need further

As we mentioned before, the multipath routing has special interests in audio/video streaming applications. For applications that requires strict ordering, single path can be used. For streaming and time-critical applications, the multi-path application might be more interesting, which can be identified by the DiffServe information. 

> MP-OLSRv2 SHOULD NOT be used.”

Fixed. 

>      be set to 0, i.e., no constraint on maximum number of hops."

Because the current RFC6554 supports only IPv6 strict source routing, we thus need to keep all the path information in the source routing header, in which case we don’t know the number of hops. 

On the other hand, we noticed that the IPv6 segment routing header is being discussed at 6man, we thus have the following text in the “Experiments to be conducted” section:

	• Use of IPv6 loose source routing. In the current specification, only strict source routing is used for IPv6 based on [RFC6554]. In [I‑D.ietf‑6man‑segment‑routing‑header], the use of loose source routing is also proposed in IPv6. In scenarios where the length of the source routing header is critical, the loose source routing can be considered.

> 3) Not sure why section 12.1. is there? Can this be removed?

Removed. 

Agains, we appreciate your valuable comments and hope that our reply addresses your concern. 

regards

Jiazi

> 
> 
> _______________________________________________
> manet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/manet

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Mon, 22 May 2017 12:17:25 +0200

Hi Jiazi,

sorry for my very late reply. Please see below.

> Am 10.05.2017 um 00:51 schrieb Jiazi Yi <[email-masked]>:
> 
> Dear Mirja, 
> 

This is only true for unreliable or partially reliable traffic. I see your use case. However, will if you have TCP traffic, you can probably assume that the traffic is reliable and should not route on a per-packet basis. This case is not covered in your draft. Also for other non-TCP you often might not know much about the traffic characteristics and therefore cannot know if per-packet scheduling is good or bad. Therefore default should be per-flow.

> 2. In such kind of lossy networks, the traditional TCP performs very poor [1]. So as far as I know, TCP is not very popular in ad hoc networks. On the other hand, based on our study, the multi-path…

I didn’t have time to read the whole paper but on a brief look, it looks like you take the delay into account for TCP traffic. Which is what I said above: either you have two links which have more or less the same delay or re-ordering will be wrongly interpreted as congestion. The other problem is TCP reduces its sending rate due to loss. So if both links are congested and losses occurred (in a non-synchronized way), TCP will react to both signals and reduce its sending rate more than necessary. That's why I would recommend to schedule TCP as well as other reliable and congestion-controlled traffic on a per-flow base. Further for UDP traffic there is no good way to actually know if the traffic is reliably transmitted and congestion control, so it’s hard to make such a decision in the network in a general case.

> 	• The disjoint multipath protocol has interesting application with erasure coding, especially for services like video/audio streaming [WPMC11]. The combination of erasure coding mechanisms and thi…

That’s fine but then you cannot assume in the rest of the document that per-packet scheduling is used.

There are two options to handle this: either you remove the text that is cited above and do not talk about scheduling at all other than in the experimentation part, or you explain carefully when potentially per-packet scheduling could be used and make clear that by default and especially for TCP per-flow scheduling should be used.

> 
> 
> [1] Performance Evaluation of TCP over Mobile Ad-hoc Networks https://arxiv.org/pdf/1002.2189.pdf 

Which DiffServ codepoints are you taking about? Some of the code points require low delay but they either say nothing about reordering or require a bounded jitter as well which mean forwarding on a per-packet basis over two links which highly different delays should not be done.

Mirja

---

From: Jiazi Yi <[email-masked]>
Date: Tue, 23 May 2017 13:49:35 +0200

Dear Mirja, 

Thank very much for your reply. 

We understand your concern on the performance, especially the issue of packet reordering when the multipath protocol is applied to transport protocol like TCP. Therefore, we propose to have some guidelines to indicate when per-flow scheduling is recommended, and when per-packet scheduling is recommended. 

More specifically, in the draft, we propose the text:

==========
In section 1.1, Experiments to be conducted:

	• Different path-selection schedulers. Depending on the application type and transport layer type either per-flow scheduler or per-datagram scheduler is applied. By default, round-robin scheduling is used to select a path to be used. In some scenarios, weighted scheduling can be considered: for example, the paths with lower metrics (i.e., higher quality) can transfer more datagrams or flows compared to paths with higher metrics.

In section 8.4, Datagram Processing at the MP-OLSRv2 Originator

If a matching Multipath Routing Tuple is obtained, the Path Tuples of the Multipath Routing Tuple are applied to the datagrams using round-robin scheduling. The scheduling policy can be either per-flow or per-datagram, depending on the transport layer protocol and the application used: 

	- In per-flow scheduling, the datagrams of the same flow is transmitted through the same path. Different flows are assigned to different paths according to round-robin scheduling. For example, there are 2 path Tuples (Path-1, Path-2) for the destination router D. A series of flows (Flow-1, Flow-2, Flow-3, ... etc.) are to be sent to router D. Path-1 is then chosen for Flow-1, Path-2 for Flow-2, Path-1 for Flow 3, etc. 

	- In per-datagram scheduling, different datagrams are transmitted through different paths according to round-robin scheduling. For example, there are 2 path Tuples (Path-1, Path-2) for the destination router D. A series of datagrams (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent to router D. Path-1 is then chosen for Packet-1, Path-2 for Packet-2, Path-1 for Packet 3, etc. 

Per-flow scheduling is recommended for transport layer protocols that require strict ordering of the datagrams such as TCP. By default, per-flow scheduling is used. Per-Datagram scheduling is recommended for transport layer protocols that have less constraint on datagram arriving order such as UDP. In lossy networks, per-datagram scheduling is also recommended.  
Other path scheduling mechanisms are also possible and will not impact the interoperability of different implementations.

===========

If it looks good for you, we will make the modification in the next revision of the draft. 

regards

Jiazi

> On 22 May 2017, at 12:17, Mirja Kuehlewind (IETF) <[email-masked]> wrote:
> 
> Hi Jiazi,
>

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Tue, 23 May 2017 14:10:11 +0200

Hi Jiazi,

unfortunately, the following sentence is not correct, as basically any transport can be encapsulated over UDP:

> Per-Datagram scheduling is recommended for transport layer protocols that have less constraint on datagram arriving order such as UDP.

I would propose the following sentence:

> Per-Datagram scheduling is only recommended if it is known that the traffic is not sensitive to reordering, e.g. for non-reliable transmission of media traffic.

In this case I would further recommend to add the following sentence:

> Note that the use of paths with highly different delays can still have a negative impact on these kind of transmissions, as packets that arrive too late may be ignored and therefore have been transmitted unnecessarily.

Further I also don’t agree to this sentence:

> In lossy networks, per-datagram scheduling is also recommended.

Even in lossy networks, per-datagram scheduling is still not recommended for TCP (if the delay of the different paths are too diverse).

Finally, I would further recommend you to not say that round-robin needs to be used in section 8.4. I also don’t think you need to have the two bullet points that explain what per-datagram and per-flow means. It’s enough to say that per-flow scheduling means that a decision is made on the first packet which path to use and all subsequent packets of the same flow (e.g. identified by the 5- or 6-tuple) need to use the same path.

Further please also check section 4 as this also makes assumptions on the path selection.

Thanks,
Mirja

> Am 23.05.2017 um 13:49 schrieb Jiazi Yi <[email-masked]>:
> 
> Dear Mirja, 
>

---

From: Jiazi Yi <[email-masked]>
Date: Wed, 24 May 2017 00:26:07 +0200

Hi Mirja, 

Thanks very much for your input. Now we have the text for section 8.4:

> If a matching Multipath Routing Tuple is obtained, the Path Tuples of the Multipath Routing Tuple are applied to the datagrams using either per-flow scheduling or per-datagram scheduling, depending on the transport layer protocol and the application used. By default, per-flow scheduling is used, especially for the transport protocols that are sensitive to reordering, such as TCP. The path selection decision is made on the first datagram and all subsequent datagrams of the same flow use the same path. If the path is detected broken before the flow is closed, another path with the most similar metric is used. Per-datagram scheduling is recommended if the traffic is insensitive to reordering such as non-reliable transmission of media traffic, or when erasure coding is applied. In such case, each datagram selects their paths independently. 
> 
> By default, the traffic load is equally distributed in multiple paths. Other path scheduling mechanisms (e.g., assigning more traffic over better paths) are also possible and will not impact the interoperability of different implementations.

And we will also rephrase the text in section 4. 

Let us know if it’s OK for you. 

best

Jiazi

> On 23 May 2017, at 14:10, Mirja Kuehlewind (IETF) <[email-masked]> wrote:
> 
> Hi Jiazi,
>

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Wed, 24 May 2017 13:22:48 +0200

Thanks! That’s much better!

Two nits below.

Mirja

> Am 24.05.2017 um 00:26 schrieb Jiazi Yi <[email-masked]>:
> 
> Hi Mirja, 
> 

s/each datagram selects their paths/each datagram selects its paths/

>> 
>> By default, the traffic load is equally distributed in multiple paths. Other path scheduling

Maybe
s/the traffic load is equally distributed/the traffic load should be equally distributed/

>>  mechanisms (e.g., assigning more traffic over better paths) are also possible and will not impact the interoperability of different implementations.
> 
>

---

From: Jiazi Yi <[email-masked]>
Date: Wed, 24 May 2017 14:32:56 +0200

Dear Mirja: 

Thanks very much! We just submitted a new revision based on our discussion: 

> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ietf-manet-olsrv2-multipath/ <https://datatracker.ietf.org/doc/draft-ietf-manet-olsrv2-multipath/>
> 
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-ietf-manet-olsrv2-multipath-15 <https://tools.ietf.org/html/draft-ietf-manet-olsrv2-multipath-15>
> https://datatracker.ietf.org/doc/html/draft-ietf-manet-olsrv2-multipath-15 <https://datatracker.ietf.org/doc/html/draft-ietf-manet-olsrv2-multipath-15>
> 
> A diff from the previous version is available at:
> https://www.ietf.org/rfcdiff?url2=draft-ietf-manet-olsrv2-multipath-15 <https://www.ietf.org/rfcdiff?url2=draft-ietf-manet-olsrv2-multipath-15>
Hopefully this revision have addressed your concern. 

best

Jiazi

> On 24 May 2017, at 13:22, Mirja Kuehlewind (IETF) <[email-masked]> wrote:
>

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 26 May 2017 17:52:02 +0200

Hi Mirja, and IESG,

You first mentioned TCP or reliability in your first message and authors
replying too, but IMO, in MANET WG drafts under IETF we still have no
routing through TCP because our MANET-Packet as in RFC5444, is only having
as mentioned in RFC5498 the IP number 138 and the UDP port number. So our
OLSRv2 routers don't do any TCP ports. Regarding reliability, I don't think
it is right to mention the reliability in MANET/this-draft, because it may
need many other considerations.

I think that we need to clarify in this  draft the manet packet rfc5444,
and that it takes the manet messages including the multipaths. So in the
draft mentions datagram, but not manet packet which is 5444 packet, is our
source routing at ip layer or transport layer do we need ip number or
transport port number.

We need to add the below to section 12 as it was done in rfc7181 and 7722:

Expert Review: Evaluation Guidelines:
   For the registry where an Expert Review is required, the designated
   expert SHOULD take the same general recommendations into
   consideration as are specified by [RFC5444], [RFC5498], [RFC7631]
and [RFC7722].

This draft is using the same header type of rfc6554 which I may disagree.
Furthermore, in RFC6554 routing has mentioned in its IANA section the
assigned ipv6 number is 3 for RPL source routing (this draft is not for
RPL), so for this multipath-protocol, we need to assign an IPv4 number for
the manet-source-routing, and  we should mention it in section 12 as using
138 for IPv6 for manet packets 5444. As we have done in MANET DSR source
routing RFC4728 assigned it for IPv4 number 48.

In section 12 we need to update the table -1 to have a title similar to
7722 which is:

Type 7 Message TLV Type Extensions

however, and that we add the references and type extensions of 0 and 1
which were allocated already. Add references in table 1 as 7181 and 7722.

take care,
AB

On Wed, May 24, 2017 at 1:22 PM, Mirja Kuehlewind (IETF) <
[email-masked]> wrote:

> >> If a matching Multipath Routing Tuple is obtained, the Path Tuples of
> the Multipath Routing Tuple are applied to the datagrams using either
> per-flow scheduling or per-datagram scheduling, depending on the transport
> layer protocol and the application used. By default, per-flow scheduling is
> used, especially for the transport protocols that are sensitive to
> reordering, such as TCP. The path selection decision is made on the first
> datagram and all subsequent datagrams of the same flow use the same path.
> If the path is detected broken before the flow is closed, another path with
> the most similar metric is used. Per-datagram scheduling is recommended if
> the traffic is insensitive to reordering such as non-reliable transmission
> of media traffic, or when erasure coding is applied. In such case, each
> datagram selects their paths independently.
>
> >>  mechanisms (e.g., assigning more traffic over better paths) are also
> possible and will not impact the interoperability of different
> implementations.
> >
> >
> >>> Per-Datagram scheduling is recommended for transport layer protocols
> that have less constraint on datagram arriving order such as UDP.
> >>
> >>> Per-Datagram scheduling is only recommended if it is known that the
> traffic is not sensitive to reordering, e.g. for non-reliable transmission
> of media traffic.
> >>
> >>> Note that the use of paths with highly different delays can still have
> a negative impact on these kind of transmissions, as packets that arrive
> too late may be ignored and therefore have been transmitted unnecessarily.
> >>
> >> Even in lossy networks, per-datagram scheduling is still not
> recommended for TCP (if the delay of the different paths are too diverse).
> >>
> >> Finally, I would further recommend you to not say that round-robin
> needs to be used in section 8.4. I also don’t think you need to have the
> two bullet points that explain what per-datagram and per-flow means. It’s
> enough to say that per-flow scheduling means that a decision is made on the
> first packet which path to use and all subsequent packets of the same flow
> (e.g. identified by the 5- or 6-tuple) need to use the same path.
> >>
> >>> We understand your concern on the performance, especially the issue of
> packet reordering when the multipath protocol is applied to transport
> protocol like TCP. Therefore, we propose to have some guidelines to
> indicate when per-flow scheduling is recommended, and when per-packet
> scheduling is recommended.
> >>>
> >>>     • Different path-selection schedulers. Depending on the
> application type and transport layer type either per-flow scheduler or
> per-datagram scheduler is applied. By default, round-robin scheduling is
> used to select a path to be used. In some scenarios, weighted scheduling
> can be considered: for example, the paths with lower metrics (i.e., higher
> quality) can transfer more datagrams or flows compared to paths with higher
> metrics.
> >>>
> >>>
> >>> If a matching Multipath Routing Tuple is obtained, the Path Tuples of
> the Multipath Routing Tuple are applied to the datagrams using round-robin
> scheduling. The scheduling policy can be either per-flow or per-datagram,
> depending on the transport layer protocol and the application used:
> >>>
> >>>     - In per-flow scheduling, the datagrams of the same flow is
> transmitted through the same path. Different flows are assigned to
> different paths according to round-robin scheduling. For example, there are
> 2 path Tuples (Path-1, Path-2) for the destination router D. A series of
> flows (Flow-1, Flow-2, Flow-3, ... etc.) are to be sent to router D. Path-1
> is then chosen for Flow-1, Path-2 for Flow-2, Path-1 for Flow 3, etc.
> >>>
> >>>     - In per-datagram scheduling, different datagrams are transmitted
> through different paths according to round-robin scheduling. For example,
> there are 2 path Tuples (Path-1, Path-2) for the destination router D. A
> series of datagrams (Packet-1, Packet-2, Packet-3, ... etc.) are to be sent
> to router D. Path-1 is then chosen for Packet-1, Path-2 for Packet-2,
> Path-1 for Packet 3, etc.
> >>>
> >>> Per-flow scheduling is recommended for transport layer protocols that
> require strict ordering of the datagrams such as TCP. By default, per-flow
> scheduling is used. Per-Datagram scheduling is recommended for transport
> layer protocols that have less constraint on datagram arriving order such
> as UDP. In lossy networks, per-datagram scheduling is also recommended.
> >>> Other path scheduling mechanisms are also possible and will not impact
> the interoperability of different implementations.
> >>>
> >>> ===========
> >>>
> >>>>> 1. The aimed scenario is mobile ad hoc networks, which have high
> packet loss by nature. One of the most important reasons is route failure
> due to mobility — in such case, if a per-flow scheduling is applied, the
> whole flow will lost. On the other hand, if we send the packets in disjoint
> paths (not necessarily equal cost), we can still make use partial
> information of the flow, or even reconstruct the flow with erasure coding.
> This is especially interesting for video/audio streaming.
> >>>>
> >>>> This is only true for unreliable or partially reliable traffic. I see
> your use case. However, will if you have TCP traffic, you can probably
> assume that the traffic is reliable and should not route on a per-packet
> basis. This case is not covered in your draft. Also for other non-TCP you
> often might not know much about the traffic characteristics and therefore
> cannot know if per-packet scheduling is good or bad. Therefore default
> should be per-flow.
> >>>>
> >>>>>
> >>>>> 2. In such kind of lossy networks, the traditional TCP performs very
> poor [1]. So as far as I know, TCP is not very popular in ad hoc networks.
> On the other hand, based on our study, the multi-path routing can actually
> reduce the overall jitter [2].
> >>>>
> >>>> I didn’t have time to read the whole paper but on a brief look, it
> looks like you take the delay into account for TCP traffic. Which is what I
> said above: either you have two links which have more or less the same
> delay or re-ordering will be wrongly interpreted as congestion. The other
> problem is TCP reduces its sending rate due to loss. So if both links are
> congested and losses occurred (in a non-synchronized way), TCP will react
> to both signals and reduce its sending rate more than necessary. That's why
> I would recommend to schedule TCP as well as other reliable and
> congestion-controlled traffic on a per-flow base. Further for UDP traffic
> there is no good way to actually know if the traffic is reliably
> transmitted and congestion control, so it’s hard to make such a decision in
> the network in a general case.
> >>>>
> >>>>>
> >>>>> We understand your concern on this issue (and you can imagine that
> you are not the first one who raises this ;). It is also something that we
> want to have further experience from this *experimental* draft, as we
> stated in the “experiments to be conducted” section:
> >>>>>
> >>>>>   • Different path-selection schedulers. By default, round-robin
> scheduling is used to select a path to be used for datagrams. In some
> scenarios, weighted scheduling can be considered: for example, the paths
> with lower metrics (i.e., higher quality) can transfer more datagrams
> compared to paths with higher metrics.
> >>>>>   • The impacts of the delay variation due to multipath routing.
> [RFC2991] brings out some concerns of multipath routing, especially
> variable latencies. Although current experiment results show that multipath
> routing can reduce the jitter in dynamic scenarios, some transport
> protocols or applications may be sensitive to the datagram re-ordering.
> >>>>>   • The disjoint multipath protocol has interesting application with
> erasure coding, especially for services like video/audio streaming
> [WPMC11]. The combination of erasure coding mechanisms and this extension
> is thus encouraged.
> >>>>
> >>>> There are two options to handle this: either you remove the text that
> is cited above and do not talk about scheduling at all other than in the
> experimentation part, or you explain carefully when potentially per-packet
> scheduling could be used and make clear that by default and especially for
> TCP per-flow scheduling should be used.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>> [2] Multipath optimized link state routing for mobile ad hoc
> networks", In Elsevier Ad Hoc Journal, vol.9, n. 1, 28-47, January, 2011.
> >>>>>
> >>>>>>
> >>>>> As we mentioned before, the multipath routing has special interests
> in audio/video streaming applications. For applications that requires
> strict ordering, single path can be used. For streaming and time-critical
> applications, the multi-path application might be more interesting, which
> can be identified by the DiffServe information.
> >>>>
> >>>> Which DiffServ codepoints are you taking about? Some of the code
> points require low delay but they either say nothing about reordering or
> require a bounded jitter as well which mean forwarding on a per-packet
> basis over two links which highly different delays should not be done.
> >>>>
> >>>> Mirja
> >>>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>
> >>>>> Because the current RFC6554 supports only IPv6 strict source
> routing, we thus need to keep all the path information in the source
> routing header, in which case we don’t know the number of hops.
> >>>>>
> >>>>> On the other hand, we noticed that the IPv6 segment routing header
> is being discussed at 6man, we thus have the following text in the
> “Experiments to be conducted” section:
> >>>>>
> >>>>>   • Use of IPv6 loose source routing. In the current specification,
> only strict source routing is used for IPv6 based on [RFC6554]. In
> [I‑D.ietf‑6man‑segment‑routing‑header], the use of loose source routing
> is also proposed in IPv6. In scenarios where the length of the source
> routing header is critical, the loose source routing can be considered.
> >>>>>
> >>>>>>

---

From: "Ratliff, Stanley" <[email-masked]>
Date: Fri, 26 May 2017 16:11:00 +0000

AB,

The text you reference in Section 8.4 is talking about the data being routed; not about the control plane (OLSR) traffic. For example, if regular browser traffic were to be carried by an OLSR network with the multipath enhancement, said browser traffic (which is TCP-based) MUST be delivered by the network in-sequence. The restriction stated in Section 8.4 preserves the ordering, since the traffic follows the same path.

As to RFC5444 – IIRC, the WG decision at the time was to allow for both operation using UDP as a transport between routers, as well as for Layer 3 communication (as OSPF does). The decision of whether to support UDP communication or Layer 3 communication is implementation-dependent. AFAIK, there are no Layer 3 implementations presently, but if anyone on the list knows of one, please correct me. However, that is the reason both a UDP port number and an IP number were allocated.

Regards,
Stan

From: manet [mailto:[email-masked]] On Behalf Of Abdussalam Baryun
Sent: Friday, May 26, 2017 11:52 AM
To: Mirja Kuehlewind (IETF) <[email-masked]>
Cc: manet <[email-masked]>; The IESG <[email-masked]>; [email-masked]
Subject: Re: [manet] Mirja Kühlewind's Discuss on draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)

Hi Mirja, and IESG,

You first mentioned TCP or reliability in your first message and authors replying too, but IMO, in MANET WG drafts under IETF we still have no routing through TCP because our MANET-Packet as in RFC5444, is only having as mentioned in RFC5498 the IP number 138 and the UDP port number. So our OLSRv2 routers don't do any TCP ports. Regarding reliability, I don't think it is right to mention the reliability in MANET/this-draft, because it may need many other considerations.

I think that we need to clarify in this  draft the manet packet rfc5444, and that it takes the manet messages including the multipaths. So in the draft mentions datagram, but not manet packet which is 5444 packet, is our source routing at ip layer or transport layer do we need ip number or transport port number.

Expert Review: Evaluation Guidelines:
   For the registry where an Expert Review is required, the designated
   expert SHOULD take the same general recommendations into
   consideration as are specified by [RFC5444], [RFC5498], [RFC7631] and [RFC7722].

This draft is using the same header type of rfc6554 which I may disagree. Furthermore, in RFC6554 routing has mentioned in its IANA section the assigned ipv6 number is 3 for RPL source routing (this draft is not for RPL), so for this multipath-protocol, we need to assign an IPv4 number for the manet-source-routing, and  we should mention it in section 12 as using 138 for IPv6 for manet packets 5444. As we have done in MANET DSR source routing RFC4728 assigned it for IPv4 number 48.

In section 12 we need to update the table -1 to have a title similar to 7722 which is:

however, and that we add the references and type extensions of 0 and 1 which were allocated already. Add references in table 1 as 7181 and 7722.

take care,
AB

On Wed, May 24, 2017 at 1:22 PM, Mirja Kuehlewind (IETF) <[email-masked]<mailto:[email-masked]>> wrote:
Thanks! That’s much better!

Two nits below.

Mirja

> Am 24.05.2017 um 00:26 schrieb Jiazi Yi <[email-masked]<mailto:[email-masked]>>:
>
> Hi Mirja,
>

Maybe

>> On 23 May 2017, at 14:10, Mirja Kuehlewind (IETF) <[email-masked]<mailto:[email-masked]>> wrote:
>>
>> Hi Jiazi,
>>
>>> Am 23.05.2017 um 13:49 schrieb Jiazi Yi <[email-masked]<mailto:[email-masked]>>:
>>>
>>> Dear Mirja,
>>>
>>>> On 22 May 2017, at 12:17, Mirja Kuehlewind (IETF) <[email-masked]<mailto:[email-masked]>> wrote:
>>>>
>>>> Hi Jiazi,
>>>>
>>>>> Am 10.05.2017 um 00:51 schrieb Jiazi Yi <[email-masked]<mailto:[email-masked]>>:
>>>>>
>>>>> Dear Mirja,
>>>>>
>>>>> [1] Performance Evaluation of TCP over Mobile Ad-hoc Networks https://arxiv.org/pdf/1002.2189.pdf<https://urldefense.proofpoint.com/v2/url?u=https-3A__arxiv.org_pdf_1002.2189.pdf&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=kVrk59fKkcL1V7Qr7L9glnGQ9yChwWSXhj0aCNFn2do&e=>
>>>>>> [email-masked]<mailto:[email-masked]>
>>>>>> https://www.ietf.org/mailman/listinfo/manet<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>
>>>
>>
>
>

_______________________________________________
manet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/manet<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>

============================================================================================================================================
This electronic message and any files transmitted with it contains
information from iDirect, which may be privileged, proprietary
and/or confidential. It is intended solely for the use of the individual
or entity to whom they are addressed. If you are not the original
recipient or the person responsible for delivering the email to the
intended recipient, be advised that you have received this email
in error, and that any use, dissemination, forwarding, printing, or
copying of this email is strictly prohibited. If you received this email
in error, please delete it and immediately notify the sender.

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 26 May 2017 18:37:31 +0200

I did not reference 8.4, just reply to the discussing about TCP, so we
don't do routing on TCP ports, yes we can have data plane using manet paths
but not reliable, so the discussion about reliability was not related. I
think the multipath work needs clear assignments for IPv4 and IPv6 not just
following RPL 6554 .

I should not think about what is implemented/used-RFC now, but we may think
of what if all our RFCs become implemented over night, so it should be
perfect or workable otherwise our review failed. My attention was that we
have allocated issues and I am talking as if thy are implemented (MANET
protocols usually are used, but no one will tell you he implemented it if
he/she did, they just use it).

AB

On Fri, May 26, 2017 at 6:11 PM, Ratliff, Stanley <[email-masked]>
wrote:

> AB,
>
>
>
> The text you reference in Section 8.4 is talking about the data being
> routed; not about the control plane (OLSR) traffic. For example, if regular
> browser traffic were to be carried by an OLSR network with the multipath
> enhancement, said browser traffic (which is TCP-based) MUST be delivered by
> the network in-sequence. The restriction stated in Section 8.4 preserves
> the ordering, since the traffic follows the same path.
>
>
>
> As to RFC5444 – IIRC, the WG decision at the time was to allow for both
> operation using UDP as a transport between routers, as well as for Layer 3
> communication (as OSPF does). The decision of whether to support UDP
> communication or Layer 3 communication is implementation-dependent. AFAIK,
> there are no Layer 3 implementations presently, but if anyone on the list
> knows of one, please correct me. However, that is the reason both a UDP
> port number and an IP number were allocated.
>
>
>
> Regards,
>
> Stan
>
>
>
> *From:* manet [mailto:[email-masked]] *On Behalf Of *Abdussalam
> Baryun
> *Sent:* Friday, May 26, 2017 11:52 AM
> *To:* Mirja Kuehlewind (IETF) <[email-masked]>
> *Cc:* manet <[email-masked]>; The IESG <[email-masked]>;
> [email-masked]
> *Subject:* Re: [manet] Mirja Kühlewind's Discuss on
> draft-ietf-manet-olsrv2-multipath-12: (with DISCUSS and COMMENT)
>
>
>
> Hi Mirja, and IESG,
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__arxiv.org_pdf_1002.2189.pdf&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=kVrk59fKkcL1V7Qr7L9glnGQ9yChwWSXhj0aCNFn2do&e=>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>
> >>>
> >>
> >
> >
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_manet&d=DwMFaQ&c=LlUvidvlhStGUSxQl0giOA&r=QzgCKlK-eUsbLW5r8KZAL9L_yqNON1dbKtMa-Wbxna8&m=dje7mQYIiFDjgbJK8mVpwiL96zg2m99Lqs3mVQGpEfk&s=Jhd9pV0Cw_wXKbnlFstVZICKvmTHz_Fz5G1AyjYvksI&e=>
>
>
> ------------------------------