Subject: [lp-wan] fragmentation for variable MTU

From: Laurent Toutain <[email-masked]>
Date: Thu, 4 Oct 2018 08:42:25 +0200

Hi,

Some technologies have variable MTU, LoRaWAN is the best example. Nicolas
ask the question few weeks ago on the mailing list.

One of the hidden beauties of Ack on Error it that we can solve the MTU
variation quite easily.

It comes from the fact that we can number a large number of fragments using
the extended window/FCN  field. A byte can be enough to number a 1500
packet cut into 8 byte long fragments. The ack containing the bitmap is
also small and can fit small MTUs.

The idea for variable MTU is to align on the shortest MTU offered by the
technology (minimal MTU or mMTU) and number all the fragments based on this
unit.

When the conditions allow it, the sender can send a larger Fragment which
must contain a multiple of mMTU. The receiver will set all the bits to 1 in
the window and continuous windows.

The magic comes from the fact that Ack on Error do not send ack when the
bitmap for a window is full. Therefore, the cost of this tiny fragmentation
is almost null, only ack are sent when some fragments are missing, not if
they are received.

In my opinion there is no change to the state machine, the reception of the
large fragment of n mMTU is processes as the reception of n fragments of
mMTU. The only change is that if we allow feedback at all-0, in that case
we allow feedback when the sending of a fragment creates a window number
change.

If the MTU change when retransmitting a fragment, the sender can give the
position of the first element through the W/FCN field and the message size
gives the length. Ack mechanism will make that all the missing part will be
resent and knowing the bitmap the sender can adapt the retransmission size.

I’m very impressed by what all the team did for Ack on Error, it was a lot
of discussions but we have a very efficient protocol. In optimal situation
we can have a single ack for a packet, and it just increases with the
number of transmission errors. Padding also is quite amazing since the cost
is only 3.5 bits. I think we have achieve a great fragmentation protocol
and we don’t yet see all its possibilities :-)

Laurent

---

From: <[email-masked]>
Date: Thu, 4 Oct 2018 08:46:51 +0000

Hello Laurent,

Thanks for this proposal.
Does it work for MTUs that can take any values (not just powers of 2)?
I can grasp how it works for MTUs that are chosen among, say, 128, 64, 32 and 16 bytes.
But can you make it work with MTUs to be chosen among, say, 55, 21, 17 and 13?
Since they are co-prime numbers, does it mean that the mMTU is one byte in this example ? So we have one Bitmap bit per byte?
Thanks

Dominique

De : lp-wan <[email-masked]<mailto:[email-masked]>> on behalf of Laurent Toutain <[email-masked]<mailto:[email-masked]>>
Date : Thursday 4 October 2018 08:42
À : lp-wan <[email-masked]<mailto:[email-masked]>>
Cc : Nicolas Sornin <[email-masked]<mailto:[email-masked]>>
Objet : [lp-wan] fragmentation for variable MTU

Hi,

Some technologies have variable MTU, LoRaWAN is the best example. Nicolas ask the question few weeks ago on the mailing list.

One of the hidden beauties of Ack on Error it that we can solve the MTU variation quite easily.

It comes from the fact that we can number a large number of fragments using the extended window/FCN  field. A byte can be enough to number a 1500 packet cut into 8 byte long fragments. The ack containing the bitmap is also small and can fit small MTUs.

The idea for variable MTU is to align on the shortest MTU offered by the technology (minimal MTU or mMTU) and number all the fragments based on this unit.

When the conditions allow it, the sender can send a larger Fragment which must contain a multiple of mMTU. The receiver will set all the bits to 1 in the window and continuous windows.

The magic comes from the fact that Ack on Error do not send ack when the bitmap for a window is full. Therefore, the cost of this tiny fragmentation is almost null, only ack are sent when some fragments are missing, not if they are received..

In my opinion there is no change to the state machine, the reception of the large fragment of n mMTU is processes as the reception of n fragments of mMTU. The only change is that if we allow feedback at all-0, in that case we allow feedback when the sending of a fragment creates a window number change.

If the MTU change when retransmitting a fragment, the sender can give the position of the first element through the W/FCN field and the message size gives the length. Ack mechanism will make that all the missing part will be resent and knowing the bitmap the sender can adapt the retransmission size.

I’m very impressed by what all the team did for Ack on Error, it was a lot of discussions but we have a very efficient protocol. In optimal situation we can have a single ack for a packet, and it just increases with the number of transmission errors. Padding also is quite amazing since the cost is only 3..5 bits. I think we have achieve a great fragmentation protocol and we don’t yet see all its possibilities •••

Laurent

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Laurent Toutain <[email-masked]>
Date: Thu, 4 Oct 2018 10:51:44 +0200

Hi Dominique

I don't understand completely the question, but I will try to answer. The
only constraint is that we need to send a multiple of mMTU, so for instance
if the mMTU is 8, and the maximum MTU at a time is 200, we can only send a
196 byte long payload.

Laurent

On Thu, Oct 4, 2018 at 10:47 AM <[email-masked]> wrote:

> Since they are co-prime numbers, does it mean that the mMTU is one byte in
> this example ? So we have one Bitmap bit per byte?
> Thanks
>
> Dominique
>
> De : lp-wan <[email-masked]> on behalf of Laurent Toutain <
> [email-masked]>
> À : lp-wan <[email-masked]>
> Cc : Nicolas Sornin <[email-masked]>
> It comes from the fact that we can number a large number of fragments
> using the extended window/FCN  field. A byte can be enough to number a
> 1500 packet cut into 8 byte long fragments. The ack containing the bitmap
> is also small and can fit small MTUs.
>
>
> they are received..
> In my opinion there is no change to the state machine, the reception of
> the large fragment of n mMTU is processes as the reception of n fragments
> of mMTU. The only change is that if we allow feedback at all-0, in that
> case we allow feedback when the sending of a fragment creates a window
> number change.
>
>
> is only 3..5 bits. I think we have achieve a great fragmentation protocol

---

From: <[email-masked]>
Date: Thu, 4 Oct 2018 09:28:19 +0000

So in essence you're saying that in my stupid example of MTUs to be chosen among 55, 21, 17 and 13 bytes, I have the following possible strategies :

  *   Decide that mMTU is 1 bytes, and then I can send fragment payloads of 55, 21, 17 and 13 bytes, but I consume 1 bit in the Bitmap per byte (so 55 bits maximum in the bitmap)
  *   Decide that mMTU is 4 bytes, and then I consume 1 bit in the Bitmap for every 4 bytes (so 13 bits maximum in the bitmap) but I can only send payloads of 52, 20, 16 and 12 bytes, not quite filling up my available MTUs
  *   Decide that mMTU is 8 bytes, and then I consume 1 bit in the Bitmap for every 8 bytes (so 6 bits maximum in the bitmap) but I can only send payloads of 48, 16, 16 and 8 bytes, not using a significant fraction of my available MTUs
  *   Or even decide that mMTU is 5 bytes, and then I consume 1 bit in the Bitmap for every 5 bytes (so 11 bits maximum in the bitmap) but I can only send payloads of 55, 20, 15 and 10 bytes,
  *   Decide that mMTU is 3 bytes, <<<left as an exercise to the reader>>>
  *   Etc.
  *   Etc.

Do this match what you have in mind ?
Dominique

De : Laurent Toutain <[email-masked]<mailto:[email-masked]>>
Date : Thursday 4 October 2018 10:51
À : Dominique Barthel <[email-masked]<mailto:[email-masked]>>
Cc : lp-wan <[email-masked]<mailto:[email-masked]>>, Nicolas Sornin <[email-masked]<mailto:[email-masked]>>
Objet : Re: [lp-wan] fragmentation for variable MTU

Hi Dominique

I don't understand completely the question, but I will try to answer. The only constraint is that we need to send a multiple of mMTU, so for instance if the mMTU is 8, and the maximum MTU at a time is 200, we can only send a 196 byte long payload.

Laurent

On Thu, Oct 4, 2018 at 10:47 AM <[email-masked]<mailto:[email-masked]>> wrote:
Hello Laurent,

Dominique

Hi,

Laurent

---

From: Laurent Toutain <[email-masked]>
Date: Thu, 4 Oct 2018 11:57:43 +0200

What kind of technologies will provide such MTU ?

We have minimum MTU which is given by the technology and the current MTU
which is select regarding the transmission condition and a frame can have
any size between the [mMTU, cMTU]. We do not have some steps in payload
size.

Laurent

On Thu, Oct 4, 2018 at 11:28 AM <[email-masked]> wrote:

> So in essence you're saying that in my stupid example of MTUs to be chosen
> among 55, 21, 17 and 13 bytes, I have the following possible strategies :
>
>    - Decide that mMTU is 1 bytes, and then I can send fragment payloads
>    of 55, 21, 17 and 13 bytes, but I consume 1 bit in the Bitmap per byte (so
>    55 bits maximum in the bitmap)
>    - Decide that mMTU is 4 bytes, and then I consume 1 bit in the Bitmap
>    for every 4 bytes (so 13 bits maximum in the bitmap) but I can only send
>    payloads of 52, 20, 16 and 12 bytes, not quite filling up my available MTUs
>    - Decide that mMTU is 8 bytes, and then I consume 1 bit in the Bitmap
>    for every 8 bytes (so 6 bits maximum in the bitmap) but I can only send
>    payloads of 48, 16, 16 and 8 bytes, not using a significant fraction of my
>    available MTUs
>    - Or even decide that mMTU is 5 bytes, and then I consume 1 bit in the
>    Bitmap for every 5 bytes (so 11 bits maximum in the bitmap) but I can only
>    send payloads of 55, 20, 15 and 10 bytes,
>    - Decide that mMTU is 3 bytes, <<<left as an exercise to the reader>>>
>    - Etc.
>    - Etc.
>
> De : Laurent Toutain <[email-masked]>
> À : Dominique Barthel <[email-masked]>
> Cc : lp-wan <[email-masked]>, Nicolas Sornin <[email-masked]>
>> Since they are co-prime numbers, does it mean that the mMTU is one byte
>> in this example ? So we have one Bitmap bit per byte?
>> Thanks
>>
>> Dominique
>>
>> I’m very impressed by what all the team did for Ack on Error, it was a
>> lot of discussions but we have a very efficient protocol. In optimal
>> situation we can have a single ack for a packet, and it just increases with
>> the number of transmission errors. Padding also is quite amazing since the
>> cost is only 3..5 bits. I think we have achieve a great fragmentation
>> protocol and we don’t yet see all its possibilities :-)
>>
>>
>>
>>
>> Laurent
>>

---

From: <[email-masked]>
Date: Thu, 4 Oct 2018 11:43:47 +0000

In LoRaWAN regional parameters,
EU863-870 band, maximum payload size, no repeater, I read
DR0 = SF12: 51 bytes
DR1 = SF11: 51 bytes
DR2 = SF10: 52 bytes
DR3 = SF9: 115 bytes
DR4 = SF8: 242 bytes
DR5 = SF7: 242 bytes

For US902-928, no repeater
DR0: 11
DR1: 53
DR2: 125
DR3: 242
DR4: 242

Did I miss something?

Dominique

Date : Thursday 4 October 2018 11:57

We have minimum MTU which is given by the technology and the current MTU which is select regarding the transmission condition and a frame can have any size between the [mMTU, cMTU]. We do not have some steps in payload size.

Laurent

On Thu, Oct 4, 2018 at 11:28 AM <[email-masked]<mailto:[email-masked]>> wrote:
So in essence you're saying that in my stupid example of MTUs to be chosen among 55, 21, 17 and 13 bytes, I have the following possible strategies :

Hi Dominique

Laurent

Dominique

Hi,

Laurent

---

From: Nicolas Sornin <[email-masked]>
Date: Thu, 4 Oct 2018 12:27:02 +0000

Dear all.
Thanks to Laurent for the proposal. I am still banging my head on it , but it looks like this could work.
I have no problem at all to enforce that fragment size must be a multiple of an mMTU.
We do not have to always use the full PHY payload max size, so very probably we will select a mMTU that is optimal for the lowest data rate.
Example :10 bytes for the US, 50 bytes in Europe

If that works, this is great news !

From: [email-masked] <[email-masked]>
Sent: Thursday, October 4, 2018 1:44 PM
To: Laurent Toutain <[email-masked]>
Cc: lp-wan <[email-masked]>; Nicolas Sornin <[email-masked]>
Subject: Re: [lp-wan] fragmentation for variable MTU

WARNING - External Email
________________________________
In LoRaWAN regional parameters,
EU863-870 band, maximum payload size, no repeater, I read
DR0 = SF12: 51 bytes
DR1 = SF11: 51 bytes
DR2 = SF10: 52 bytes
DR3 = SF9: 115 bytes
DR4 = SF8: 242 bytes
DR5 = SF7: 242 bytes

Did I miss something?

Dominique

Laurent

Hi Dominique

Laurent

Dominique

Hi,

Laurent

Thank you.

Thank you.

Thank you.

To view our privacy policy, including the types of personal information we collect, process and share, and the rights and options you have in this respect, see www.semtech.com/legal.

---

From: ivaylo petrov <[email-masked]>
Date: Thu, 4 Oct 2018 15:20:21 +0200

Dear all,

This truly seems quite interesting and I completely agree with Nicolas that
not sending a message with the maximum allowed size is not going to be a
big problem if we can have changing MTU during fragmented packet sending.

I am also not entirely sure I understand all the details of the proposal
that Laurent described, so let me try to explain an example with my
understanding. Please correct me if I misunderstood something.

If I have a packet that is 500 bytes and my mMTU is 51 byes, and my
fragmentation header size is 1 byte. If at first my conditions allow me to
send at least 102 bytes, I send 2 fragments (fragmentation header fragment
1 + payload fragment 1 + fragmentation header fragment 2 + payload fragment
2). The receiver will then know that it needs to mark both fragments as
received and the sender marks both fragments as sent.

Afterwards if the conditions change and I can send 204 bytes, I send 4
fragment at a time. And I progress this way (sending as much fragments per
L2 message as possible) until I finish sending. If the receiver tells me
that he is missing fragments 1 and 2 in the end, I can resend them
separately (if the conditions mandate this) or together if the conditions
are good.

Is might understanding correct?

Thank you!
--
Ivaylo Petrov
Chief Architect
ivaylo at ackl.io

www.ackl.io

1137A avenue des Champs Blancs,
35510 Cesson Sévigné
FRANCE

Phone: +33 (0)2 22 06 05 77

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you!

On Thu, Oct 4, 2018 at 2:27 PM Nicolas Sornin <[email-masked]> wrote:

> We do not have to always use the full PHY payload max size, so very
> probably we will select a mMTU that is optimal for the lowest data rate.
>
> *From:* [email-masked] <[email-masked]>
> *Sent:* Thursday, October 4, 2018 1:44 PM
> *To:* Laurent Toutain <[email-masked]>
> *Cc:* lp-wan <[email-masked]>; Nicolas Sornin <[email-masked]>
> *Subject:* Re: [lp-wan] fragmentation for variable MTU
>
>
>
> *WARNING - External Email*
> ------------------------------
>
> *De : *Laurent Toutain <[email-masked]>
> *Date : *Thursday 4 October 2018 11:57
> *À : *Dominique Barthel <[email-masked]>
> *Cc : *lp-wan <[email-masked]>, Nicolas Sornin <[email-masked]>
> *Objet : *Re: [lp-wan] fragmentation for variable MTU
>
>
>
> *De : *Laurent Toutain <[email-masked]>
> *Date : *Thursday 4 October 2018 10:51
> *À : *Dominique Barthel <[email-masked]>
> *Cc : *lp-wan <[email-masked]>, Nicolas Sornin <[email-masked]>
> *Objet : *Re: [lp-wan] fragmentation for variable MTU
>
>
>
> Hi Dominique
>
>
>
> *De : *lp-wan <[email-masked]> on behalf of Laurent Toutain <
> *Date : *Thursday 4 October 2018 08:42
> *À : *lp-wan <[email-masked]>
> *Cc : *Nicolas Sornin <[email-masked]>
> *Objet : *[lp-wan] fragmentation for variable MTU
>
>
>
> Hi,
>
>
>
> using the extended window/FCN  field. A byte can be enough to number a 1500
> To view our privacy policy, including the types of personal information we
> collect, process and share, and the rights and options you have in this
> respect, see www.semtech.com/legal.
> _______________________________________________
> lp-wan mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lp-wan
>

---

From: <[email-masked]>
Date: Thu, 4 Oct 2018 22:52:06 +0000

Hello Ivaylo,

I think the idea is to send multiple fragment payloads but only one fragment header. If we multiplied the headers, we would be wasting bandwidth. It would be simple frame aggregation. Besides, the SCHC Fragment Header don't have any information about the SCHC Fragment Payload size, the latter is all inferred from the L2 payload size. Therefore, there would be no way of telling where the second SHC Fragment Header starts.

I understood Laurent's proposal this way:
Let say you number fragments having in mind 50 bytes as the smallest fragment payload (that is, 51 bytes of mMTU and 1 byte of header).
It turns out your current MTU is 201 bytes or more.
You send a SCHC Fragment Header with MAX_WIND_FCN and stuff 200 bytes of SCHC Fragment Payload. This is cheating but the receiving process will compute an index into the reassembly buffer and stuffs the 200 bytes correctly in place. The bitmap is updated to reflect that 4 mMTUs have been received.
You then send a SCHC Fragment Header with MAX_WIND_FCN-4 and stuff the next 200 bytes of SCHC Fragment Payload. This fragment gets lost.
You then send a SCHC Fragment Header with MAX_WIND_FCN-8 and stuff the next 200 bytes of SCHC Fragment Payload.
Etc.
When the ACK comes, 4 mMTUs are reported missing. Bad luck, the channel has now changed to a 51 bytes MTU.
You now retransmit a SCHC Fragment Header with MAX_WIND_FCN-4 and the first 50 bytes of Payload. The receiving process compute the index into the reassembly buffer and stuffs the 50 bytes correctly in place. The bitmap is updated to reflect that 1 mMTUs has been received.
Then a SCHC Fragment Header with MAX_WIND_FCN-5 and the next 50 bytes of Payload. The receiving process compute the index into the reassembly buffer and stuffs the 50 bytes correctly in place. The bitmap is updated to reflect that 1 mMTUs has been received.
Etc.

Does this make sense?
Laurent, did I get you right?
However, this does change on behaviour in the receive process. So far, we had assumed that the SCHC Fragment Payloads were contiguous by numbers, whatever their sizes. Now we assume that each FCN unit corresponds to some fixed number of bytes. Is this correct?

Best regards

Dominique

De : lp-wan <[email-masked]<mailto:[email-masked]>> on behalf of ivaylo petrov <[email-masked]<mailto:[email-masked]>>
Date : Thursday 4 October 2018 15:20
À : Nicolas Sornin <[email-masked]<mailto:[email-masked]>>
Cc : "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Laurent Toutain <[email-masked]<mailto:[email-masked]>>, Dominique Barthel <[email-masked]<mailto:[email-masked]>>
Objet : Re: [lp-wan] fragmentation for variable MTU

Dear all,

This truly seems quite interesting and I completely agree with Nicolas that not sending a message with the maximum allowed size is not going to be a big problem if we can have changing MTU during fragmented packet sending.

I am also not entirely sure I understand all the details of the proposal that Laurent described, so let me try to explain an example with my understanding. Please correct me if I misunderstood something.

If I have a packet that is 500 bytes and my mMTU is 51 byes, and my fragmentation header size is 1 byte. If at first my conditions allow me to send at least 102 bytes, I send 2 fragments (fragmentation header fragment 1 + payload fragment 1 + fragmentation header fragment 2 + payload fragment 2). The receiver will then know that it needs to mark both fragments as received and the sender marks both fragments as sent.

Afterwards if the conditions change and I can send 204 bytes, I send 4 fragment at a time. And I progress this way (sending as much fragments per L2 message as possible) until I finish sending. If the receiver tells me that he is missing fragments 1 and 2 in the end, I can resend them separately (if the conditions mandate this) or together if the conditions are good.

Thank you!
--
Ivaylo Petrov
Chief Architect
ivaylo at ackl.io<http://ackl.io/>

[https://drive.google.com/uc?id=0B4itASCdgeQKTXBiSVREZVAxRDRyNlRIcnpIZXNlSWhMNWNB&export=download]

www.ackl.io<http://www.ackl.io/>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you!

On Thu, Oct 4, 2018 at 2:27 PM Nicolas Sornin <[email-masked]<mailto:[email-masked]>> wrote:
Dear all.

From:[email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, October 4, 2018 1:44 PM
To: Laurent Toutain <[email-masked]<mailto:[email-masked]>>
Cc: lp-wan <[email-masked]<mailto:[email-masked]>>; Nicolas Sornin <[email-masked]<mailto:[email-masked]>>
Subject: Re: [lp-wan] fragmentation for variable MTU

WARNING - External Email

Did I miss something?

Dominique

Laurent

Hi Dominique

Laurent

Dominique

Hi,

Laurent

Thank you.

Thank you.

Thank you.

To view our privacy policy, including the types of personal information we collect, process and share, and the rights and options you have in this respect, see www.semtech.com/legal<http://www.semtech.com/legal>.
_______________________________________________
lp-wan mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lp-wan

---

From: <[email-masked]>
Date: Thu, 4 Oct 2018 22:59:12 +0000

Hello Laurent,

In the general case, having LPWAN MTUs multiples of some mMTU does not equate with having the SCHC Fragment Payloads being multiples of the minimal SCHC Fragment Payload (mSFP). This is because the size of the header is added, and does not multiply.

For example, if the SCHC Fragment Header is 11 bits, then

MTU in bytes MTU in bits SCHC Fragment Payload in bits

3 24 13

4 32 21

5 40 29

6 48 37

7 56 45

The larger SCHC Fragments Payloads are not multiples of the smaller ones. The small fragments will not tile nicely with the large ones in the re-assembly buffer.

And we can't decide to use a smaller payload size, because the size is enforced by the L2 alignement (since we decided to be very smart :-) and not have padding in the fragments except the last one).

I guess the solution could be to enforce that the SCHC Fragment Header is also a multiple of the L2 Word.

For example, if the SCHC Fragment Header is 2 bytes, then

MTU in bytes MTU in bits SCHC Fragment Payload in bits

3 24 8

4 32 16

5 40 24

6 48 32

7 56 40

8 64 48

And now they tile nicely.

However, we lose some freedom on the SCHC Header format.

I guess this loss is acceptable in regard of the benefit of being able to work with variable MTU.

Comments?

Dominique

De : lp-wan <[email-masked]<mailto:[email-masked]>> on behalf of Dominique Barthel <[email-masked]<mailto:[email-masked]>>
Date : Friday 5 October 2018 00:52
À : ivaylo petrov <[email-masked]<mailto:[email-masked]>>, Nicolas Sornin <[email-masked]<mailto:[email-masked]>>
Cc : "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Laurent Toutain <[email-masked]<mailto:[email-masked]>>
Objet : Re: [lp-wan] fragmentation for variable MTU

Hello Ivaylo,

Does this make sense?

Best regards

Dominique

Dear all,

Thank you!

WARNING - External Email

Did I miss something?

Dominique

Laurent

Hi Dominique

Laurent

Dominique

Hi,

Laurent

Thank you.

Thank you.

Thank you.