Subject: Re: [Jmap] [Jmap for Sieve] Binary/blob vs. Inline

From: Hans-Joerg Happel <[email-masked]>
Date: Fri, 5 Nov 2021 17:37:25 +0100

On 26.10.21 06:43, Neil Jenkins wrote:
> On Fri, 22 Oct 2021, at 03:41, Hans-Joerg Happel wrote:
>> => We'd argue that this process is both complex to implement plus it's
>> not particularly elegant to create a need for storing a temporary file.
>
> Depends how you look at it; it /is/ elegant to have a single 
> consistent method for referencing binary data. I think the answer here 
> if you are trying to add support on a legacy system where this is 
> difficult is to make the Sieve its own account. Then when you upload a 
> blob to that account you know it can only be a sieve script, which 
> should allow you to do something reasonable.

Thanks for elaboration - I missed to recognize that it is possible use 
the "Account-workaround" also for writing.

However, enforcing this process makes it still hard to implement in 
various imaginable cases. Take my initial Database storage example: You 
might force an implementer to store a temp Sieve script (from blob call) 
until the second call comes in to continue glue together that data and 
do the database INSERT.

While this is no strict blocker, I think enforcing a blob storage 
concept so widely is in conflict with the goal of making it easy for as 
many people as possible to adopt JMAP.

Some additional minor points, for the record:

* The blobId approach opens up many potential error cases, which 
server+client implementers need to have in mind (e.g, think about the 
script validation workflow)
* It makes the JMAP Sieve workflow essentially different from the known 
and closely related ManageSieve workflows

>
>> It would therefore be good if it is possible to (alternatively?) submit
>> the script inline in the "SieveScript/set" command instead of a blobid.
>
> Using Bron's blob extension you can create a blob in-band and then 
> backreference it from the SieveScript/set command. But I wouldn't want 
> to say that was the only way you could do it; that breaks the 
> consistency of the blob handling.

If I understand it correctly, while it is atomic from a HTTP call 
perspective, the chained processing of commands on server-side is not 
necessarily atomic from a logical point of view? Which would mean that a 
client still needs to deal with different cases (partial fail / complete 
fail ... ) of the chained calls?

Best,
Hans-Joerg

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 8 Nov 2021 10:16:15 +0700

Hello,

Nitpicking on this topic.

On 05/11/2021 23:37, Hans-Joerg Happel wrote:
>> consistent method for referencing binary data. I think the answer
>> here if you are trying to add support on a legacy system where this
>> is difficult is to make the Sieve its own account. Then when you
>> upload a blob to that account you know it can only be a sieve script,
>> which should allow you to do something reasonable.
>
> various imaginable cases. Take my initial Database storage example:
> You might force an implementer to store a temp Sieve script (from blob
> call) until the second call comes in to continue glue together that
> data and do the database INSERT.
>
> concept so widely is in conflict with the goal of making it easy for
> as many people as possible to adopt JMAP.
>
> * It makes the JMAP Sieve workflow essentially different from the
> known and closely related ManageSieve workflows
>
>
"blob based" objects (by this I mean that can only be created with
"blob" calls) of the Email spec involves attachments, expected to be
large. A few kbs -> MBs. Having an off band mechanisms for those makes
perfect sense.

However (to me) Sieve scripts are better compared with an HTML body.
Sure they are no upper bound but except few outliers, most would be a
few KBs (As an administrator, I likely do not want people to upload
scripts of more than a few KBs in the first place). HTML bodies can be
created in and out of band.

The added semantic of having a blob allows things like attaching the
sieve script to an email without needing to download + re-upload it (for
instance) but it do not look like a common feature to me.

So while I am not shocked by the "a Sieve script is a blob" approach, I
would consider an inlined String feels more natural.

Allowing in and out of bound (so inlined string AND blobId based Sieve
script) sure is doable, would settle the debate but brings in a bit more
complexity to the backend implementers who now have to code 2 systems
instead of one... Would it be "overkill" here?

Finally, I think all JMAP capable system should be able to handle blob.
This is a requirement for the core spec. Sure for a JMAP proxy on top of
a "ManageSieve" server this means that the proxy would have to store
temporarily blobs but I would expect JMAP-IMAP proxy to already have
similar behavior. This sounds like a non argument to me.

>> backreference it from the SieveScript/set command. But I wouldn't
>> want to say that was the only way you could do it; that breaks the
> necessarily atomic from a logical point of view? Which would mean that
> a client still needs to deal with different cases (partial fail /
> complete fail ... ) of the chained calls?
>
Yes I do not think this would spare the "intermediate" storage as a blob
you were mentioning above.

Best regards,

Benoit
>
> Best,
> Hans-Joerg
>
>
> _______________________________________________
> Jmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jmap

---

From: "Neil Jenkins" <[email-masked]>
Date: Mon, 08 Nov 2021 15:47:42 +1100

On Sat, 6 Nov 2021, at 03:37, Hans-Joerg Happel wrote:
> While this is no strict blocker, I think enforcing a blob storage concept so widely is in conflict with the goal of making it easy for as many people as possible to adopt JMAP.
> 

Blob support is a part of JMAP core, so if you want to implement JMAP you're probably going to need it.

> * The blobId approach opens up many potential error cases, which server+client implementers need to have in mind (e.g, think about the script validation workflow)

Not sure I follow. You validate it just the same, it's just where you get the bytes from.

> * It makes the JMAP Sieve workflow essentially different from the known and closely related ManageSieve workflows 

Making it work the same as ManagedSieve is secondary to making it work consistently with other JMAP APIs in my view.

I personally don't particularly mind whether the API is blob based or just inline code, but I know this was discussed earlier and I believe there were good reasons to go with blob.

> If I understand it correctly, while it is atomic from a HTTP call perspective, the chained processing of commands on server-side is not necessarily atomic from a logical point of view?
> 

That's right.

> Which would mean that a client still needs to deal with different cases (partial fail / complete fail ... ) of the chained calls?

Well, it can probably just look at the response of the second call; if the blob creation succeeded but the sieve update failed, you can just ignore the blob bit; it should get cleaned up at some point automatically. And if the blob failed, then the sieve update will fail as the result ref won't resolve, so you can still just look at the second error.

—Neil

---

From: Hans-Joerg Happel <[email-masked]>
Date: Tue, 9 Nov 2021 12:09:25 +0100

Hi Neil,

I think we're mainly viewing JMAP from different perspectives:

* Yours seems like "a system will be built according to the (full) JMAP 
specs"
* Mine/ours is more like: JMAP can be quite useful to adapt also for 
existing systems, even if only partially implemented (Every piece of 
JMAP makes the world a little better ;-)

Both perspectives have their merits in my opinion. I am just trying to 
bring the standard(s) forward by providing some arguments to consider 
for that second perspective.

I do think this is a valid concern, given that there is a large set of 
existing systems in the area JMAP addresses. Also CardDAV/CalDAV as 
slighly related standards are mostly implemented on top of existing systems.

I agree with you that some of my points don't seem incredibly important, 
but I think we should also pay attention not to enforce JMAP-specific 
constraints if not strictly necessary.

> Not sure I follow. You validate it just the same, it's just where you 

I basically had in mind the additional errors cases (as described in 
your chained call response below), which a client would not need to deal 
with in an approach more similar to ManageSieve PUTSCRIPT.

> Making it work the same as ManagedSieve is secondary to making it work 
> consistently with other JMAP APIs in my view.

ok, fair point.

> I personally don't particularly mind whether the API is blob based or 
> just inline code, but I know this was discussed earlier and I believe 
> there were good reasons to go with blob.

I just found this thread 
(https://mailarchive.ietf.org/arch/msg/jmap/6u3p0fGs9cdDrBo8y6aU_8Hi2HU/) 
which does not discuss this too extensive. But I see the point Ken made, 
that both alternatives are viable, but that it would be better to go 
with one option only.

Thanks and best,
Hans-Joerg

>> necessarily atomic from a logical point of view?
>>
>
> That's right.
>
>> Which would mean that a client still needs to deal with different 
>> cases (partial fail / complete fail ... ) of the chained calls?
>
> Well, it can probably just look at the response of the second call; if 
> the blob creation succeeded but the sieve update failed, you can just 
> ignore the blob bit; it should get cleaned up at some point 
> automatically. And if the blob failed, then the sieve update will fail 
> as the result ref won't resolve, so you can still just look at the 
> second error.
>
> —Neil
>