Subject: [httpapi] Using Date in requests

From: "Martin Thomson" <[email-masked]>
Date: Wed, 09 Feb 2022 16:29:33 +1100

Hi Everyone,

I've just posted https://www.ietf.org/archive/id/draft-thomson-httpapi-date-requests-00.html which talks about how to use Date in requests.  This is not a typical thing you see, but some of the things we're doing elsewhere makes this relevant.  For example, signing requests [1] and oblivious HTTP [2] - depending on circumstances - might want to use Date for managing anti-replay.

I'm bring this here as I think that this group has more direct expertise and the work is related to some of the other stuff you are doing, like the idempotency-key, which is complementary.  This also uses the problem details work (RFC 7807bis) for signaling when Date is missing or incorrect.

I'm happy to go into the use case in more detail if the content of the draft isn't clear enough.  Mostly, I'm just doing this so  there is a complete and robust solution for how to manage replay for the aforementioned work.

Cheers,
Martin

[1] https://httpwg.org/http-extensions/draft-ietf-httpbis-message-signatures.html
[2] https://ietf-wg-ohai.github.io/oblivious-http/draft-ietf-ohai-ohttp.html

---

From: Ben Bucksch <[email-masked]>
Date: Wed, 9 Feb 2022 08:25:13 +0100

Am 09.02.22 um 06:29 schrieb Martin Thomson:
> I've just postedhttps://www.ietf.org/archive/id/draft-thomson-httpapi-date-requests-00.html  which talks about how to use Date in requests.  ... use Date for managing anti-replay.

Hello Martin,

thanks for posting this and asking for feedback.

I appreciate that you acknowledge the problem of clock screw in the 
spec. Some clocks are so bad that they can easily be minutes out of 
sync. Yet others are simply set to the completely wrong time, but the 
user doesn't use that clock, so he doesn't know. This leads to serious 
problems in such systems that compare clocks of different systems, 
particularly if the systems are managed by different entities, like in 
most client/server scenarios.

I ran into that very painfully with JWT [1]. It took multiple developers 
several days to debug a failure, which eventually turned out to be a 
simple clock screw.

I came to the conclusion that any system that compares clocks of two 
different systems is inherently flawed. That general idea should be 
avoided in any system - be it a custom site-specific security measure, 
or an Internet standard.

Rather, time should always be *relative*. Coincidentally, Einstein 
agreed ;-).

So, what I would propose is rather something that works like a special 
browser cookie. For the sake of the discussion, I call it "time cookie", 
but it doesn't need to be technically a cookie, but it should have the 
same storage properties and lifetime as a cookie, for privacy and 
tracking purposes. This time cookie would contain a simple Integer, and 
the only special thing about it is that the browser stores the local 
time when it was received, and when it's sent, the browser adds the 
*time difference between receiving and sending*, in the form of the 
difference of Unixtime.

For example, the server sends 1,000,000 . That's an arbitary Integer 
value, but let's say the server uses the Unixtime of its local server. 
When receiving, the browser stores this value, together with the current 
local time, on the client system, e.g. as Unixtime (which is let's say 
1,001,000, i.e. client and server disagree by 1 000 seconds = 17 
minutes, but that doesn't matter). 10 seconds later, the browser makes 
another request to the same server, and the browser sends the 
server-supplied value of 1,000,000 plus 10 (seconds) = 1,000,010. The 
server recieves this, the server compares it to its own local Unixtime 
(or any other base value), and it matches, and the request works.

If you were to send the client time, the client would have sent 
1,001,010, and the value would not have matched and the request would 
have failed.

I believe that such a system that sends /only/ the time difference is 
inherently safer and more reliable, less prone to security attacks, and 
also eventually simpler to implement, if you consider all the 
counter-measures that a system based on absolute time would have to 
implement against clock screw and security attacks.

I appreciate very much the thought that you put into all the problems of 
clock screw, security attacks manipulating local time, and even privacy 
threats that abuse the stored server-time for tracking. Very well done. 
But I believe that all of these could be avoided by a system and simply 
uses only relative time and never ever compares clocks between two systems.

Due to these known problems, I would recommend to explicitly /avoid/ to 
use a format that uses an absolute date/time format and instead to 
intentionally specify that the browser sends the time difference in the 
form of seconds or milliseconds. You all have experienced first hand how 
often even the most simple things (like HTTP error codes) are 
implemented without reading the specs carefully, so I am concerned that 
many implementations will not consider the extensive thoughts about 
clock screw that you put in, or probably won't even read any part of the 
spec, but if there's a "Date" header in datetime format, many devs will 
simply send the current local time, without any adjustment or 
consideration. So, a system that is explicitly specced to transmit 
*only* the time *difference* and does not use the format of an absolute 
time is far more likely to be implemented correctly.

Let's just forget the idea that there is an universally agreed absolute 
time.

Repectfully,

Ben Bucksch

[1] Very painful experience with JWT 
<https://github.com/mozilla/addons-server/issues/1072>

We wrote a script to upload [something to a third-party server] as part 
of our build process. The code was working for [the dev who] wrote the 
script, but I just got "Invalid authentication credentials".

We were debugging it for hours and hours, suspecting shell argument 
passing and parsing, OS differences and what not. We compared 
credentials and confirmed they are identical. Still, the result was 
different. During debugging, once, by coincidence, it worked. At a time 
when it should be expired.

What was the reason? My clock on my computer was fast.

I was screaming. Client server protocols must NEVER EVER compare client 
clocks with server clocks, because clocks are very often off. Even if it 
was originally set exactly right by the second, clocks always have a 
drift in one way or another. If you have 5 million users, you will 
surely have [many] users whose clocks are off... In my case, my clock 
was 95 seconds fast. That's not uncommon. I don't run ntpdate regularly, 
because ntp regularly has security problems, too.

This is a fatal misdesign of the protocol. You simply cannot compare 
client clock with server clocks. That something that engineers generally 
know from experience. Often hard lessons, like mine above. Sometimes 
worse lessons, when 5 million people were hit by this problem.

---

From: Roberto Polli <[email-masked]>
Date: Wed, 9 Feb 2022 13:03:49 +0100

Hi Martin,

Il giorno mer 9 feb 2022 alle ore 06:30 Martin Thomson
<[email-masked]> ha scritto:
> I've just posted https://www.ietf.org/archive/id/draft-thomson-httpapi-date-requests-00.html which talks about how to use Date in requests.
Thanks for your draft, I think that the considerations are interesting,
and are widely applicable when using Date. It seems to me to be a kind of BCP.
Probably some of them would fit the `Date` field definition.

> [...] signing requests [1] and oblivious HTTP [2] - depending on circumstances - might want to use Date for managing anti-replay.

While I can't tell on oblivious HTTP, all the considerations
in the document suggest to me that  using the signature validity and
timestamps expressed
via JWT (iat, nbf, exp) or via Signature's created, expires parameters
is more reliable than signing Date.

> [..] the work is related to some of the other stuff you are doing,
> like the idempotency-key, which is complementary.
> This also uses the problem details work (RFC 7807bis) for signaling when Date is missing or incorrect.
I am curious about whether this applies to the SF-Date proposal too
(eg. to signal the presence of both SF-Date and Date eg when both fields
are present in signatures, ...)

> I'm happy to go into the use case in more detail
Please, do!

Thanks for your time and have a nice day,
R.

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 10 Feb 2022 13:59:01 +1100

Hi Ben,

You will note that the system I describe is almost exactly like the one you have.  It's just that the "arbitrary number" is formatted as a Date field.  The resulting change (if any) is stored in the same way as a cookie, with all the same considerations.

I agree that this is suboptimal, but we're dealing with a system that exists.  More expansive solutions are less likely to get implemented, particularly for niche use cases.  Not many people want to include dates in requests.  Many of those that do can use something like JWT/JWS rather than a Date field (same problems of course).  

Spelling it differently won't change much, except make the change more disruptive.  I'm looking to make a surgical tweak that many people won't notice in order to deal with the problem.  I think that's an easier change to enact than a more comprehensive change, even if that change might be more principled.

Also, for all those cases where people do use dates and get the problems you cite, the draft proposes a nice problem message ... and automated remediation strategy.

Cheers,
Martin

p.s., Funny you should say that clocks might be wrong by minutes.  I've seen years and have credible reports of *decades*.

On Wed, Feb 9, 2022, at 18:25, Ben Bucksch wrote:
>> I've just posted https://www.ietf.org/archive/id/draft-thomson-httpapi-date-requests-00.html which talks about how to use Date in requests.  ... use Date for managing anti-replay. 
>
>
> Hello Martin,
>
> I ran into that very painfully with JWT [1]. It took multiple 
> developers several days to debug a failure, which eventually turned out 
> to be a simple clock screw.
>
> Rather, time should always be *relative*. Coincidentally, Einstein agreed ;-).
>
> browser cookie. For the sake of the discussion, I call it "time 
> cookie", but it doesn't need to be technically a cookie, but it should 
> have the same storage properties and lifetime as a cookie, for privacy 
> and tracking purposes. This time cookie would contain a simple Integer, 
> and the only special thing about it is that the browser stores the 
> local time when it was received, and when it's sent, the browser adds 
> the *time difference between receiving and sending*, in the form of the 
> difference of Unixtime.
>
> When receiving, the browser stores this value, together with the 
> current local time, on the client system, e.g. as Unixtime (which is 
> let's say 1,001,000, i.e. client and server disagree by 1 000 seconds = 
> 17 minutes, but that doesn't matter). 10 seconds later, the browser 
> makes another request to the same server, and the browser sends the 
> I believe that such a system that sends *only* the time difference is 
> I appreciate very much the thought that you put into all the problems 
> of clock screw, security attacks manipulating local time, and even 
> privacy threats that abuse the stored server-time for tracking. Very 
> well done. But I believe that all of these could be avoided by a system 
> and simply uses only relative time and never ever compares clocks 
> between two systems.
>
> Due to these known problems, I would recommend to explicitly *avoid* to 
> form of seconds or milliseconds. You all have experienced first hand 
> how often even the most simple things (like HTTP error codes) are 
> clock screw that you put in, or probably won't even read any part of 
> the spec, but if there's a "Date" header in datetime format, many devs 
> will simply send the current local time, without any adjustment or 
> Let's just forget the idea that there is an universally agreed absolute time. 
>
> Repectfully,
>
> Ben Bucksch
>
>
>
>
>
>
>
> clocks with server clocks, because clocks are very often off. Even if 
> it was originally set exactly right by the second, clocks always have a 
> was 95 seconds fast. That's not uncommon. I don't run ntpdate 
> regularly, because ntp regularly has security problems, too.
>
> client clock with server clocks. That something that engineers 
> generally know from experience. Often hard lessons, like mine above. 
> Sometimes worse lessons, when 5 million people were hit by this problem.
>
>
>
>
> -- 
> httpapi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/httpapi

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 10 Feb 2022 14:06:07 +1100

> is more reliable than signing Date.

Probably true.  Though I think the same considerations apply there as they do to Date.

Oblivious HTTP doesn't really have the luxury of defining specific fields for this information.  It is passing messages with generic HTTP semantics.  For those cases where a server wants to do anti-replay, we want to have a place we can point to to say "here you need to have a monotonic sequence number that is roughly globally consistent (*cough* a Date) for the purposes of limiting the amount of state the server needs to maintain for anti-replay purposes".

In discussing this problem for OHTTP we concluded that Date is an ideal fit for that requirement, modulo the clock skew issue.  We considered minting our own field, but it just ended up being the same as Date anyway.  It has simple HTTP semantics, it is well-defined, and most servers broadcast it already.  It's nearly perfect, hence this draft.  Date is also a pain to parse (an ongoing project to fix) and has some awkwardness involved, but I think all we need to do is improve comprehension of the problem.

---

From: James <[email-masked]>
Date: Fri, 11 Feb 2022 16:17:29 +0100

Hi Martin,
Thanks for writing this up. I'm unsure if this was discussed elsewhere, but another consideration about the Date header is its off-label use for rough time synchronisation - in particular, phk wrote an implementation[1], and later a rough specification[2] with an entry in the well-known registry[3] using a different header aimed to either synchronise, or validate a local clock. I both hope and doubt nobody is using this approach, but its existence leads me to think that some overall guidance on how people transmit time information more broadly in HTTP messages and rely on it might be helpful for your audience. Happy to help write that.

- J

1: http://phk.freebsd.dk/time/20151212/
2: http://phk.freebsd.dk/time/20151129/
3: https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml

> On 9 Feb 2022, at 06:29, Martin Thomson <[email-masked]> wrote:
> 
> Hi Everyone,
>

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 14 Feb 2022 10:19:27 +1100

Yeah, it seems like this is one of the major revelations from this discussion.  That is, that times (or dates) might appear in multiple places and the considerations here aren't limited to the Date field.

Any input regarding text would be greatly appreciated.  I'll get around to doing something eventually, but my todo list keeps growing.

On Sat, Feb 12, 2022, at 02:17, James wrote:
> Hi Martin,
> Thanks for writing this up. I'm unsure if this was discussed elsewhere, 
> but another consideration about the Date header is its off-label use 
> for rough time synchronisation - in particular, phk wrote an 
> implementation[1], and later a rough specification[2] with an entry in 
> the well-known registry[3] using a different header aimed to either 
> synchronise, or validate a local clock. I both hope and doubt nobody is 
> using this approach, but its existence leads me to think that some 
> overall guidance on how people transmit time information more broadly 
> in HTTP messages and rely on it might be helpful for your audience. 
> Happy to help write that.
>
> - J
>

---

From: Herbert Van de Sompel <[email-masked]>
Date: Mon, 14 Feb 2022 09:45:02 +0100

Dear Martin,

Probably not really relevant for you but the Memento "Time Travel for the
Web" protocol [1] uses dates in requests using the Accept-Datetime header
as a means to do content negotiation in the datetime dimension. The major
motivation for devising the protocol was achieving a level of
interoperability across web archives. But the protocol can be supported by
all systems that support datetime-based resource versioning such as wikis
etc.

Greetings

Herbert Van de Sompel

[1] HTTP framework for time-based access to resource states - Memento.
Herbert Van de Sompel, Michael L. Nelson, Robert Sanderson. RFC 7089.
http://www.rfc-editor.org/rfc/rfc7089.txt

On Wed, Feb 9, 2022 at 6:30 AM Martin Thomson <[email-masked]> wrote:

> Hi Everyone,
>
> I've just posted
> https://www.ietf.org/archive/id/draft-thomson-httpapi-date-requests-00.html
> which talks about how to use Date in requests.  This is not a typical thing
> you see, but some of the things we're doing elsewhere makes this relevant.
> For example, signing requests [1] and oblivious HTTP [2] - depending on
> circumstances - might want to use Date for managing anti-replay.
>
> I'm bring this here as I think that this group has more direct expertise
> and the work is related to some of the other stuff you are doing, like the
> idempotency-key, which is complementary.  This also uses the problem
> details work (RFC 7807bis) for signaling when Date is missing or incorrect.
>
> I'm happy to go into the use case in more detail if the content of the
> draft isn't clear enough.  Mostly, I'm just doing this so  there is a
> complete and robust solution for how to manage replay for the
> aforementioned work.
>
> Cheers,
> Martin
>
>
> [1]
> https://httpwg.org/http-extensions/draft-ietf-httpbis-message-signatures.html
> [2]
> https://ietf-wg-ohai.github.io/oblivious-http/draft-ietf-ohai-ohttp.html
>
> --
> httpapi mailing list
> [email-masked]

-- 
==================
Herbert Van de Sompel
https://hvdsomp.info
https://orcid.org/0000-0002-0715-6126

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 14 Feb 2022 20:22:13 +1100

On Mon, Feb 14, 2022, at 19:45, Herbert Van de Sompel wrote:
> Probably not really relevant for you but the Memento "Time Travel for 
> the Web" protocol [1] 

Yeah, this is a different thing entirely.  I might tweak the text to make that clearer.  Memento asks about arbitrary times (with Accept-Datetime), whereas this is intended to capture the value of *now* and to address the problems that can arise when attempting to do that.