Subject: [P2Prg] BitTorrent Paper Summary

From: Sam Joseph <[email-masked]>
Date: Fri, 13 Feb 2004 06:57:04 +0900

Hi All,

So I'm posting this to p2prg as well as p2p-metadata as well, since I 
haven't seen much discussion on either.  To get us going with some 
meta-data and trust topics, I'm posting my summary notes on the 
BitTorrent paper:

http://www.sims.berkeley.edu/research/conferences/p2pecon/papers/s4-cohen.pdf

I would be very interested to hear any comments or feedback.  I think 
BitTorrent is very interesting because it uses special meta-data 
encoding and tit-for-tat choking in order to make sure of fair downloads.

CHEERS> SAM

Incentives Build Robustness in BitTorrent

1. Problems addressed

Downloading files from multiple sources.
Difficult due to:
a) complexity of assignment
b) churn
c) fairness

2. Interface

Absolute simplicity of interface perhaps key to take up of BitTorrent

3. MetaData

Torrent files specify information about a file length, name, hashing 
information, and url of a tracker - all bencoded to reduce size

4. Trackers and peers

Peers get random lists of other peers that are sharing parts of a file 
from trackers.  Updated each 3-5 minutes, and at least 1800 trackers 
currently in existence.  Tracker is simple, runs over http and has 
minimal comms.  Files are broken into chunks with hashes to ensure 
correctness.  All peers continuously download chunks from all peers they 
can.

5.  Various Heuristics used to improve performance

a) random connections
b) pipelining - always have more requests for file pieces lined up
c) try to download the rarest piece of a file first
d) when downloading starts go for random first peice
e) special endgame mode to cancel slow downloads
f) tit for tat choking + optimistic unchoking
g) Snubbing

_______________________________________________
P2prg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2prg

---

From: "Serguei Osokine" <[email-masked]>
Date: Thu, 12 Feb 2004 21:40:08 -0800

On Thursday, February 12, 2004 Sam Joseph wrote:
> 3. MetaData
>
> Torrent files specify information about a file
> length, name, hashing information, and url of a
> tracker - all bencoded to reduce size

	Sam, why do you call it metadata? I always assumed that
metadata would be various stuff like performer, album, bitrate,
quality, etc. Okay, so Bram Cohen himself calls .torrent files
'metainfo' - but is it a good term for the contents of that
file? What .torrent has (and btw, I think that it hashes not
only the file itself, but also every separate piece of the
file) - anyway, it seems to be a fairly straightforward
integrity checking mechanism, not unlike, say, TigerTree
hash or Kazaa hash.

	Do you really insist on calling it metadata? Am I missing
something?

	Best wishes -
	S.Osokine.
	12 Feb 2004.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Sam
Joseph
Sent: Thursday, February 12, 2004 1:57 PM
To: p2p-irtf; [email-masked]
Subject: [P2Prg] BitTorrent Paper Summary

Hi All,

http://www.sims.berkeley.edu/research/conferences/p2pecon/papers/s4-cohen.pd
f

CHEERS> SAM

1. Problems addressed

2. Interface

3. MetaData

4. Trackers and peers

a) random connections

---

From: Sam Joseph <[email-masked]>
Date: Fri, 13 Feb 2004 15:40:18 +0900

Hi Serguei,

Serguei Osokine wrote:

>	Sam, why do you call it metadata?
>
I guess because it is data about data --> meta-data

> I always assumed that
>quality, etc. 
I think that is also meta-data.

>Okay, so Bram Cohen himself calls .torrent files
>file)
I think you're right - I was just summarising from Bram's p2pecon paper.

> - anyway, it seems to be a fairly straightforward
I agree.

>	Do you really insist on calling it metadata? Am I missing
I don't insist on calling it meta-data.  I guess I was just tying it in 
with the ideas in my decentralized meta-data strategies paper:

http://www.neurogrid.net/php/publications.php

in which I say the following:

-------------------------------------------------------------------
Taking an inclusive view of meta-data it seems that we can consider a 
number of possible types:

(1) Document Hash - An id generated from the document contents via some 
hashing algorithm that ideally will be unique to each document
(2) Document Id  - An id assigned arbitrarily to a document according to 
some scheme - different from a hash in that it must be generated by some 
authority
(3) Statistical representation - A representation generated by 
performing a statistical operation on a document, that may involve 
statistics relating to a larger document collection, e.g. TFIDF (see 
section 2.1.1)
(4) Human assigned - Keywords or more complex statements such as RDF 
(see section 2.1.4)

The important differences between the types of meta-data are whether the 
representation is unique among documents, whether it can be 
automatically generated, and the logical complexity of the associations.
-------------------------------------------------------------------

Basically I was just following that thinking.  Perhaps I am muddying the 
water?  But I guess I would want to make a distinction between metadata 
that is semantic annotations, and "structural"? "factual" metadata, that 
can be generated by some process of inspection.

However, even if you wanted to avoid calling data that is derived by 
inspection "meta-data", we still have things like the tracker url.  I 
would have said that that is metadata.  How to organise and co-ordinate 
that data about files is arguably a meta-data issue - but then I would 
be just as happy to use another word - well, I guess I have written a 
lot about meta-data, so to use a different word now might be confusing 
... still I see it all as part of coordination of data about data in p2p 
systems ....

CHEERS> SAM

---

From: "Serguei Osokine" <[email-masked]>
Date: Sat, 14 Feb 2004 15:36:11 -0800

On Thursday, February 12, 2004 Sam Joseph wrote:
> I guess I was just tying it in with the ideas in my
> decentralized meta-data strategies paper:
>
> Taking an inclusive view of meta-data it seems that we can 
> consider a number of possible types:...

	Fair enough. I was thinking mainly about the 

> (4) Human assigned...

- since... well... that's the hard part, I guess. As opposed
to computing hash, which is easy, fast, and straightforward,
so I tend not to think about it a lot.

	But in yur classification it really is metadata - in fact,
everything that is not in a file itself seems to be metadata.
Okay, that makes sense.

	Best wishes - 
	S.Osokine.
	14 Feb 2004.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Sam
Joseph
Sent: Thursday, February 12, 2004 10:40 PM
To: [email-masked]
Cc: p2p-irtf; [email-masked]
Subject: Re: [P2Prg] BitTorrent Paper Summary

Hi Serguei,

I agree.

CHEERS> SAM

---

From: Justin Chapweske <[email-masked]>
Date: Mon, 16 Feb 2004 10:56:34 -0600

I see no reason why it isn't valid to refer to the file size,
last-modified, hash, etc., as file metadata.  

As far as encoding goes, we've found that HTTP header fields are a
perfectly capable mechanism for transferring file metadata, and if you
need something with even more structure, then just use RDF.

---

From: Scott Michel <[email-masked]>
Date: Mon, 16 Feb 2004 11:38:57 -0800

>integrity checking mechanism, not unlike, say, TigerTree
Filesystem attributes are and will continue to be called metadata. The 
difference here is that the metadata is stored in two different places, 
hence the name change to disambiguate which metadata is which. Even 
worse is when the file content metadata to which you're referring is 
stored in the file itself as opposed to a FS structure.

>	Do you really insist on calling it metadata? Am I missing
Yes, I'm relatively sure that information about a file, irrespective of 
where it is stored (in the FS or inside the file) will always be called 
metadata.

-scooter