Subject: [MMUSIC] File-transfer issue 31: Errors when multiple hashes

From: Miguel Garcia <[email-masked]>
Date: Mon, 21 May 2007 16:37:05 +0300

Paul Kyzivat wrote:
 >
 > Section 8.2.2:
 >
 >    If the SDP offer included several
 >    'hash' parameters, the file sender SHOULD include at least one in the
 >    SDP answer, selected among those present in the offer and, at the
 >    same time, supported by the file sender.  If the SDP offer did not
 >    include a 'hash' parameter, the file sender SHOULD add one or more
 >    'hash' parameters, according to the supported hashing algorithms.
 >
 > What should the file sender do if it supports multiple hash algorithms
 > but the hashes supplied are not all consistent? Namely a file that
 > matches one of them doesn't match the others. E.g. suppose the offer
 > contained:
 >
 >     a=file-selector:hash:sha-1:ABC hash:md5:DEF
 >
 > and the matching file matches the sha-1 hash but not the md5 hash.
 >
 > It could treat this as an error and reject the request,

How can you reject the request due to a malformed hash, if you still
don't have the file? To calculate that the hash is correct, you need
to have the file, compute the hash over it (with an appropriate
algorithm) and then compare the obtained hash with the one that the
sender sent you in the SDP.

 > or it could
 > treat this as ok as long as one of the hashes matches. The latter seems
 > better. In that case, I think it should only return the hashes that
 > actually match the file being returned. This does imply that if it
 > supports multiple hashes then it needs to test the file against all of
 > those supplied. Alternatively, perhaps it should only return those
 > hashes that it has actually tested against the file being returned.

This is obviously an error case: if you implement several hashing
encoding algorithms, then each one should produce a correct output and
all should be consistent.

Thinking with an implementation mindset, I think this is the sequence
flow at the file receiver side. Once I have received the offer, I
would select ONE hash algorithm that I implement. I would use this one
to detect if I have already got the same file or not. If not, I will
create the SDP answer, get the file, and hash it with the same
algorithm that I selected to detect whether the file was received or
not. If the result matches the hash declared in the SDP, then fine. If
not, then I would deem it as a transmission error and inform the user
that the file is corrupted. I would never check another hashing
algorithm, even if supported.

Conclusion: once the file receiver selects a hashing algorithm, it is
like if only that algorithm has been sent in SDP. Do we agree with the
conclusion? Is there anything that needs to be added to the draft?

/Miguel
-- 
Please note my new e-mail address: miguel.garcia at nsn.com
Miguel A. Garcia           tel:+358-50-4804586
Nokia Siemens Networks     Helsinki, Finland

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 21 May 2007 15:42:36 -0400

Inline.

	Paul

> sender sent you in the SDP.

Well, you may have found the file based on the sha-1 hash. So you have 
the file. But perhaps the md5 hash of that file is not "DEF"?

(I think we may have different assumptions about the cases where this 
occurs. See below for more.)

> all should be consistent.

Right. For this to occur somebody must implement a hash incorrectly. (Of 
course the one that detects a problem will assume that it is the other 
party that made the mistake.) Its just a matter of how to respond.

> algorithm, even if supported.

I'm a little lost above.

The case I had in mind was a pull scenario. The file receiver sends an 
offer with multiple hash algorithms as I stated before. The file sender 
receives the offer and uses one of the hash algorithms to select the 
file. (Or it might use the name to select the file.) It then gets ready 
to send the file, so it prepares the answer. In the answer it needs to 
include hash information. As the draft was written, it wasn't clear 
whether it was to echo the hash parameters from the offer into the 
answer, or to include the hash algorithms from the offer in the answer, 
with the hash values it computes for the file. If the hash values 
requested are the same as those it computes it doesn't matter, but if 
they are different it does matter.

I think it would be bad in the case I mentioned for it to include md5 
with the incorrect value of DEF. It might be confusing for it to include 
a correct md5 hash that differed from what was requested, or maybe that 
would be an ok thing to do. It might be a burden on some file senders to 
recalculate all the hashes for the file that were mentioned in the request.

I think a reasonable answer is for the file *sender* in this case to 
include those hashes it knows for the file, including the one (if any) 
that was used to select the file. This could be limited to those 
mentioned in the request, but I see no reason to make such a limitation.

> conclusion? Is there anything that needs to be added to the draft?

Yes, I think something is needed. For one thing, I note that there are 
no actions specified for the offerer when the answer is received, or 
when the file itself is received.

I think the following could be added to the end of 8.1.2:

"When the File Receiver receives the answer, if the port number of the 
answer for a file request is non-zero then it SHOULD receive the file 
using the MSRP protocol. If the answer contains a supported hashing 
algorithm, then the receiver SHOULD compute the hash of the file after 
receipt and check against the hash received in the answer."

In 8.2.2 I would suggest a change in the following:

                                    ... If the SDP offer included several

changed as follows:

                                    ... The file sender SHOULD add
    one or more 'hash' parameters in the answer, with algorithms
    supported by the file sender and values computed by the file sender,
    and SHOULD include at least one with a hash algorithm selected among
    those present in the offer. The file sender MUST NOT include a 'hash'
    parameter with an algorithm that was present in the offer and a value
    that differs from what was in the offer. (If a hash value computed
    by the file sender differs from that specified by the file receiver,
    the file sender can either send the file without the hash value or
    reject the request with a 400 (Bad Request) response.

I also now see a typo in 8.2.1:

    In a push operation the answerer is the file receiver.  When the file
    receiver gets the SDP answer, it extracts the attributes and

in above s/SDP answer/SDP offer/

---

From: Miguel Garcia <[email-masked]>
Date: Tue, 22 May 2007 10:35:39 +0300

Some text was cut. Let's go to the point of discussion.

>> algorithm, even if supported.

Clarification: The above paragraph tried to describe the push scenario. 
You thought of the pull scneario, so that explains the confusion.

> file. (Or it might use the name to select the file.) 

So far so good. Clarification: the draft says that the file receiver 
must select one hash (if there are multiple) and all the other 
selectors. So, it can't use only the file name, it has to use the file 
name and one hash.

> It then gets ready 

I think it is written:

- Push scenario, Section 8.2.1:

    If
    the offer contains one ore more 'hash' parameters in the 'file-
    selector' attribute, the answerer discards those with non-supported
    hashing algorithms and MUST copy the remaining (if any) to the 'file-
    selector' attribute of the answer.

- Pull scenario, Section 8.2.2 (see also discussion at the end of this 
mail):

> receipt and check against the hash received in the answer."

I am fine with the text.. I will loose a bit the words, because MSRP is 
used only because it is indicated in the m= line, so it is possible that 
in the future we extend the mechanism to use other transfer protocols. I 
have a problem with the first "SHOULD", because it depends on an action 
done by the remote endpoint (because it has to send), so it is obviously 
a "should". I have also indicated what the user agent should do if the 
check fails (inform the user and consider that the file transfer 
failed). The current text reads:

    When the file receiver receives the SDP answer, if the port number of
    the answer for a file request is non-zero then it should receive the
    file using the protocol indicated in the m= line.  If the SDP answer
    contains a supported hashing algorithm in the 'hash' parameter of the
    'file-selector' attribute, then the file receiver SHOULD compute the
    hash of the file after receipt and check against the hash received in
    the answer.  In case the computed hash does not match the one
    contained in the SDP answer, the file receiver SHOULD consider that
    the file transfer failed and SHOULD inform the user.

>    reject the request with a 400 (Bad Request) response.

I think we need to discuss the principles a bit more. To put things in 
context first: An file receiver (offerer) learned by some magic (e.g., 
resource event package draft) one or more hashes of a file. It selects 
one or more that it supports and included into the offer (the offerer is 
not going to add a hash algorithm it does not support). It sends the 
offer to the file receiver, who extracts one and only one hash algorithm 
to select the file (perhaps together with other selectors, that is 
irrelevant). Then, why in the heck does the file sender need to send 
MORE hash algorithms, (and presumably compute them). I think it is a 
waist of time and CPU cycles.

Also, your text "The file sender MUST NOT include a 'hash' parameter 
with an algorithm that was present in the offer and a value that differs 
from what was in the offer". This will never happen with the scenario I 
mentioned, because:
a) there is only one chosen hash algorithm
b) the hash is used to select the file, if the hash was wrong, then it 
would not select a file.

> in above s/SDP answer/SDP offer/

good catch. Fixed.

BR,

     Miguel

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 22 May 2007 12:58:13 -0400

I think we are close to converging.  More inline. (I'm trimming 
extraneous text.)

	Paul

Miguel Garcia wrote:

>> It then gets ready to send the file, so it prepares the answer. In the 
>> answer it needs to include hash information. As the draft was written, 
>> it wasn't clear whether it was to echo the hash parameters from the 
>> offer into the answer, or to include the hash algorithms from the 
>> offer in the answer, with the hash values it computes for the file. If 
>> the hash values requested are the same as those it computes it doesn't 
>> matter, but if they are different it does matter.
> 
> I think it is written:
> 

None of the above says that the sender computes the hash that it 
includes. Copying the hash values from the offer seems to be perfectly 
consistent with the above text. In fact, the above seems to demand 
copying them rather than recomputing or verifying. And it could be an 
easy and natural thing to do. But its the *wrong* thing to do if the 
value hasn't been confirmed to be correct.

>    the file transfer failed and SHOULD inform the user.

This seems fine.

> waist of time and CPU cycles.

It doesn't NEED to. And the text I suggested doesn't require that. I 
said "one or more" that are "supported by the file sender", at least one 
of which has an algorithm included in the offer. One is enough to 
satisfy all of those requirements. The sender can include others if it 
wishes, mentioned in the offer or not.

Why would the sender include extras? Perhaps it keeps all the hashes 
around for its files, and its simply easier to include them all than to 
check which of them is an algorithm mentioned in the request.

If there are several in the answer, then the recipient can choose which 
one to use for verifying the receipt of the file. It might choose an 
easier one to compute than the one it requested.

Also, this unifies with the case where the offer included *no* hashes, 
because none were known. In that case the sender doesn't know what 
hashes the receiver supports, so sending multiples increases the odds 
that the receiver will be able to check the hash.

Doesn't really matter. This just gives a little more latitude to the 
sender without requiring any more work.

> b) the hash is used to select the file, if the hash was wrong, then it 

I think it could still be possible, if the sender can include multiple 
hashes. Suppose the offer included sha-1:ABC and md5:DEF. And suppose 
the answerer/sender chose the sha-1 hash to select the file, so it does 
actually have the sha-1:ABC hash. But suppose the sender's md5 hash 
value for the file is CDE.

It could return 'sha-1:ABC md5:CDE' but that might expose the conflict 
with what the offerer requested. This statement just prevents that conflict.

But I don't care much about this. It would simplify things to leave it out.

	Paul

---

From: Miguel Garcia <[email-masked]>
Date: Wed, 23 May 2007 13:48:21 +0300

Hi Paul:

See some more inlines discussion

Paul Kyzivat wrote:
>>> It then gets ready to send the file, so it prepares the answer. In 
>>> the answer it needs to include hash information. As the draft was 
>>> written, it wasn't clear whether it was to echo the hash parameters 
>>> from the offer into the answer, or to include the hash algorithms 
>>> from the offer in the answer, with the hash values it computes for 
>>> the file. If the hash values requested are the same as those it 
>>> computes it doesn't matter, but if they are different it does matter.
>>
>> I think it is written:
>>
> includes. 

But it has to compute the has. Let me re-explain:
- Push scenario:
   If  a file sender is going to send a file along with its hash, then 
at some point in time the hash was computed for that file.

- Pull scenario:
   The offerer includes a file selector that includes one or more 
hashes. The draft says that the answerer chooses one hash and together 
with the other selectors, selects a file. This means that the file 
sender goes comparing the computed hashes of existing files with that in 
the offer.

In both cases the hash has been computed, perhaps earlier, not in 
realtime, but I think the time at which a hash is computed is an 
implementation issue.

What am I missing here?

> Copying the hash values from the offer seems to be perfectly 

>>> "When the File Receiver receives the answer, if the port number of 
>>> the answer for a file request is non-zero then it SHOULD receive the 
>>> file using the MSRP protocol. If the answer contains a supported 
>>> hashing algorithm, then the receiver SHOULD compute the hash of the 
>>> file after receipt and check against the hash received in the answer."
>>
>> I am fine with the text.. I will loose a bit the words, because MSRP 
>> is used only because it is indicated in the m= line, so it is possible 
>> that in the future we extend the mechanism to use other transfer 
>> protocols. I have a problem with the first "SHOULD", because it 
>> depends on an action done by the remote endpoint (because it has to 
>> send), so it is obviously a "should". I have also indicated what the 
>> user agent should do if the check fails (inform the user and consider 
>> that the file transfer failed). The current text reads:
>>
>> one or more that it supports and included into the offer (the offerer 
>> is not going to add a hash algorithm it does not support). It sends 
>> the offer to the file receiver, who extracts one and only one hash 
>> algorithm to select the file (perhaps together with other selectors, 
>> that is irrelevant). Then, why in the heck does the file sender need 
>> to send MORE hash algorithms, (and presumably compute them). I think 
>> it is a waist of time and CPU cycles.
> 

Ok, I can agree with this issue. So the answerer will include all the 
supported hashes he wants to include in the answer. Those are locally 
computed, not copied from the offer.

>> with an algorithm that was present in the offer and a value that 
>> differs from what was in the offer". This will never happen with the 
>> scenario I mentioned, because:

If we go for the suggestion I had below, the error case is possible, but 
certainly unlikely. In any case, there should be at least one common 
hash in the offer and the answer, so the error does not have a side effect.

> But I don't care much about this. It would simplify things to leave it out.

I'll write some text around these lines and post it for further comments.

/Miguel