Subject: [Anima] GRASP issue 58: Maximum message size?

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Oct 2016 09:05:40 +1300

58: Maximum message size?

MR: I'm also a bit concerned that we have no statement about maximum message size.

BC: Me too. I coded my prototype with recvfrom limited to 2048 bytes, but we probably
need to specify something. On the other hand, Michael B wants to send infinitely long
Intents, I think.

Regards
   Brian

---

From: "Michael Behringer (mbehring)" <[email-masked]>
Date: Wed, 19 Oct 2016 08:41:38 +0000

> -----Original Message-----
> From: Anima [mailto:[email-masked]] On Behalf Of Brian E
> Carpenter
> Sent: 18 October 2016 22:06
> To: Anima WG <[email-masked]>
> Subject: [Anima] GRASP issue 58: Maximum message size?
> 
> BC: Me too. I coded my prototype with recvfrom limited to 2048 bytes, but
> we probably need to specify something. On the other hand, Michael B wants
> to send infinitely long Intents, I think.

:-) 

My view: 
- We use GRASP to signal which Intent version we have, and where the latest version can be fetched from, and with which method / protocol. 
- We use that protocol to actually get the Intent file. 

This is the most flexible and extensible model, IMHO.

So, from my point, no problem with a maximum message size of 2048 bytes, as long as we send the Intent file separately :-)

Michael
 
> Regards
>    Brian
> 
> 
> _______________________________________________
> Anima mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/anima

---

From: Michael Richardson <[email-masked]>
Date: Wed, 19 Oct 2016 16:13:46 -0400

Brian E Carpenter <[email-masked]> wrote:
    > BC: Me too. I coded my prototype with recvfrom limited to 2048 bytes,
    > but we probably need to specify something. On the other hand, Michael B
    > wants to send infinitely long Intents, I think.

libcbor nicely responds to an incomplete message with CBOR_ERR_NOTENOUGHDATA.
This lets you grow the data structure for the situation where Intents are
large.
https://libcbor.readthedocs.io/en/latest/api/decoding.html#associated-data-structures

It would be better if the initial parts of the structure could be examined
even in the error case, and if it was the case that one didn't want to deal
with the rest, one could be told how much more to discard.  Or, in some
cases, one wouldn't know (streaming arrays, etc), and perhaps the answer is
drop the TCP connection.
I think that this could be added to libraries.

What I guess I'm concerned about is a CBOR definite array structure that says
that there are 10 Billion items to be expected, the library goes and
allocates that much ram (10G), and the system fails or goes into swap hell.
No need to actually transmit the 10G of data.

libcbor can be configured to use a custom allocator rather than malloc(3),
and this is likely the correct answer: to provide some reasonable limit on
amount of memory for decoding... which comes back to how big will the Intents
be.
kilobytes?      probably.
megabytes?      maybe one or two Mbytes.
gigabytes?      most certainly not.

*could* we make the maximum objective (assuming intents are directly in an
objective) size itself be a synchronizable parameter?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Wed, 19 Oct 2016 16:16:21 -0400

Michael Behringer (mbehring) <[email-masked]> wrote:
    > My view: - We use GRASP to signal which Intent version we have, and
    > where the latest version can be fetched from, and with which method /
    > protocol.  - We use that protocol to actually get the Intent file.

I agree that this is a better way to do things.
I think that an Intent to get the ACP working ought be less than a few KB.

    > So, from my point, no problem with a maximum message size of 2048
    > bytes, as long as we send the Intent file separately :-)

RFC6920 defines ni:, which might prove useful.
Naming Things with Hashes

Abstract

   This document defines a set of ways to identify a thing (a digital
   object in this case) using the output from a hash function.  It
   specifies a new URI scheme for this purpose, a way to map these to
   HTTP URLs, and binary and human-speakable formats for these names.
   The various formats are designed to support, but not require, a
   strong link to the referenced object, such that the referenced object
   may be authenticated to the same degree as the reference to it.  The
   reason for this work is to standardise current uses of hash outputs
   in URLs and to support new information-centric applications and other
   uses of hash outputs in protocols.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 20 Oct 2016 12:27:42 +1300

> This is the most flexible and extensible model, IMHO.

One possible tweak: if Intent is in fact short enough to fit in one message,
we send it via GRASP. If not, we send the file coordinates. Just needs
a little syntactic sugar in the Intent objective.

> So, from my point, no problem with a maximum message size of 2048 bytes, as long as we send the Intent file separately :-)

Yes. I would tend to do it by defining a new constant GRASP_MSG_LIMIT
so that the value (2048 or whatever) is only specified in one place,
along with the other protocol constants.

Any other opinions?

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 20 Oct 2016 15:34:21 +1300

> objective) size itself be a synchronizable parameter?

We could even make it negotiable, since GRASP supports negotiation :-).
But I think we'd better start off with a hard limit so that we don't
encounter 10GB requirements during initialization.

I could imagine adding a max_size parameter to the API, too, so that
a given ASA could accept larger messages for a given objective. So maybe
the best thing for now is to define a *default* maximum size in the spec.

Regards
   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 25 Oct 2016 12:34:41 +1300

I'm going to take silence as assent: i.e. we will specify
a default maximum message size of 2048, and leave the question
of how it can be exceeded to be a negotiable property of individual
objectives.

Regards
   Brian