Subject: Re: todo: add language encoding information

From: Henry Story <[email-masked]>
Date: Thu, 22 Dec 2005 02:04:20 +0100

Sorry again to take so long to respond. I have been a little too busy  
to respond recently.

On 4 Dec 2005, at 16:42, Simon Phipps wrote:
> On Dec 4, 2005, at 14:33, Henry Story wrote:
>
>> I have written my first blog entry in French [1] which has made me  
>> aware that
>> it would be very useful to add language information to the atom  
>> file generated by
>> BlogEd. A menu beneath the Entry icon would probably do the trick.
>>
>> Henry
>>
>> [1] http://bblfish.net/blog/page9.html#t2005_12_04.14_17_02_642
>
> Does Atom allow there to be multiple parallel renditions of a blog  
> entry in different languages? I've been discussing with Dave  
> Johnson how we might provide a facility in Roller where a blog  
> entry might be provided in multiple languages and then select the  
> appropriate entry based on the contents of the HTTP GET. If this  
> were to be implemented, we'd need a way to express the same reality  
> in the Atom feed.

This is a complicated and delicate issue. An atom entry has been  
syntactically limited by the atom spec by the following clause

[[
    If multiple atom:entry elements with the same atom:id value  
appear in
    an Atom Feed Document, they represent the same entry.  Their
    atom:updated timestamps SHOULD be different.  If an Atom Feed
    Document contains multiple entries with the same atom:id, Atom
    Processors MAY choose to display all of them or some subset of them.
    One typical behavior would be to display only the entry with the
    latest atom:updated timestamp.
]] last para of section 4.1.1 of http://www.ietf.org/rfc/rfc4287

So it is not really possible to put atom entries with the same id and  
updated time stamp in a  feed (without a SHOULD level violation) even  
if they are translation of each other. That means that the Swiss  
would not be able to publish a law with the same atom id in french  
and in german, as they are obliged to publish these at the exact same  
time by law. (No linguistic preference)

If we want to respect this we have to give different translations of  
a document different ids. We should be able to specify that they are  
translations of each other some other way. Using the Atom OWL  
ontology [1] we could do this by adding some translation relation  
between the entries. Once this the best way to do this is settled and  
that it satisfies our needs it should be possible to put a proposal  
to the atom syntax group for an extension to their xml format.

So perhaps something like this is needed

[ a :Feed, :Version;

     :title [ :value "Example Feed";
              :type "text/plain" ];
     :link  [ :href <http://example.org/>;
              :rel iana:alternate ];
     :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
     :author [ :name "John Doe" ];
     :id <urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6>;

     :entry [ a :Entry;
              :title [ :value "Atom-Powered Robots Run Amok";
                       :type "text/plain" ];
              :link [  :href <http://example.org/2003/12/13/en/atom03>;
                       :rel iana:alternate ];
              :id <tag:example.com,2005/en/atom03>;
              :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
              :summary [  :value "After a course of integration in  
French philosophy,...";
                          :type "text/plain" ];

              ext:translation <tag:example.com,2005/fr/atom03>

            ];

     :entry [ a :Entry;
              :title [ :value "Des Robot Atomiques se rebellent";
                       :type "text/plain" ];
              :link [  :href <http://example.org/2003/12/13/fr/atom03>;
                       :rel iana:alternate ];
              :id <tag:example.com,2005/fr/atom03>;
              :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
              :summary [  :value "Apres un cours d'integration en  
philosophie francaise...";
                          :type "text/plain" ]

              ext:translation <tag:example.com,2005/en/atom03>

            ];
] .

The ext:translation relation above relates a :Entry to an :EntryId  
which is to say that it cannot distinguish between one version of an  
entry and another. So we would either be saying that each version of  
one entry is always going to be a translation of the other, or that  
there is some version of the other entry that is a translation of  
this one (which is a little too vague for comfort).

We could be more precise by giving each entry version its unique id,  
in which case we could say that this particular entry is a  
translation of that particular entry, which would be more precise and  
satisfactory. In N3:

[ a :Feed, :Version;

     :title [ :value "Example Feed";
              :type "text/plain" ];
     :link  [ :href <http://example.org/>;
              :rel iana:alternate ];
     :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
     :author [ :name "John Doe" ];
     :id <urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6>;

     :entry [ a :Entry;

              = _:someid1;

              :title [ :value "Atom-Powered Robots Run Amok";
                       :type "text/plain" ];
              :link [  :href <http://example.org/2003/12/13/en/atom03>;
                       :rel iana:alternate ];
              :id <tag:example.com,2005/en/atom03>;
              :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
              :summary [  :value "After a course of integration in  
French philosophy,...";
                          :type "text/plain" ];

              ext:translation _:someid2;

            ];

     :entry [ a :Entry;

               = _:someid2;

              :title [ :value "Des Robot Atomiques se rebellent";
                       :type "text/plain" ];
              :link [  :href <http://example.org/2003/12/13/fr/atom03>;
                       :rel iana:alternate ];
              :id <tag:example.com,2005/fr/atom03>;
              :updated "2003-12-13T18:30:02Z"^^xsd:dateTime;
              :summary [  :value "Apres un cours d'integration en  
philosophie francaise...";
                          :type "text/plain" ]

              ext:translation _:someid1;

            ];
] .

So in RDF we have no trouble doing the right thing and we have choice.

It should therefore be possible to express something like the above  
in atom syntax. How to do it is of course less clear given the lack  
of clear semantics for the atom xml.

Henry Story

[1] http://groups.google.com/group/atom-owl/browse_frm/thread/ 
a5ff6dcca7b512b8

> S.

---

From: "James Holderness" <[email-masked]>
Date: Thu, 22 Dec 2005 15:34:32 -0000

Henry Story wrote:
>> Does Atom allow there to be multiple parallel renditions of a blog  entry 
>> in different languages?
>
> updated time stamp in a  feed (without a SHOULD level violation) even  if 
> they are translation of each other. That means that the Swiss  would not 
> be able to publish a law with the same atom id in french  and in german, 
> as they are obliged to publish these at the exact same  time by law. (No 
> linguistic preference)

The Swiss may need to publish laws in multiple languages simultaneously, but 
most users surely don't need to read those laws in multiple languages 
simultaneously. Why waste their bandwidth including several translations in 
one feed when it would be far more convenient if you just had a separate 
feed for each language?

If you want to link the various translations together you can add one or 
more link elements at the top of the feed with rel="alternate" and hreflang 
set to the language of the alternate feed. If you're feeling really 
enthusiastic you can include alternate links pointing to the translated html 
pages for each entry too.

No extensions need to be made to the Atom spec. No wasted bandwidth by 
having multiple translations in a feed that may not be used. But you can 
still publish muliple language simultaneously by making sure you update all 
feeds simultaneously. You could even publish all the feeds at the same URL, 
serving the correct translation based on the HTTP Accept-Language header.

I may be missing something here, but it seems to me like a reasonable 
solution to the problem.

Regards
James

---

From: Henry Story <[email-masked]>
Date: Thu, 22 Dec 2005 20:16:21 +0100

Simon makes some good practical points in the message I forwarded  
just previous to this one. But I would like to make some more  
abstract points too for those of you who are more of the Jungian  
introspective/rational character types (most of us here I guess or  
else we would be out surfing on the beach ;-)

On 22 Dec 2005, at 16:34, James Holderness wrote:
> Henry Story wrote:
>>> Does Atom allow there to be multiple parallel renditions of a  
>>> blog  entry in different languages?
>>
>> So it is not really possible to put atom entries with the same id  
>> and updated time stamp in a  feed (without a SHOULD level  
>> violation) even  if they are translation of each other. That means  
>> that the Swiss  would not be able to publish a law with the same  
>> atom id in french  and in german, as they are obliged to publish  
>> these at the exact same  time by law. (No linguistic preference)
>
> The Swiss may need to publish laws in multiple languages  
> simultaneously, but most users surely don't need to read those laws  
> in multiple languages simultaneously. Why waste their bandwidth  
> including several translations in one feed when it would be far  
> more convenient if you just had a separate feed for each language?

The atom syntax has a very clear restriction which I believe is of  
semantic importance [1]. And that is as I mentioned in my original  
post that:

[[

What would be the point of having this restriction if it was not of  
semantic significance? It seems a little odd to tell people that they  
cannot put two entries into a feed with the same updated time stamp,  
but that they can put them into two different feeds and all is ok.  
Why have that restricition in that case? My guess is that the  
restriction was accepted because it worked on the following  
intuition. The atom id is a string that identifies something we can  
think of as a document. It therefore does not make sense to have two  
different incompatible versions of the same document with the same  
time stamp. The atom id is therefore much more restrictive than a URL  
[2]. A URL in web architecture can have any number of different  
representations at the same time. Atom clearly rules this out. So my  
guess is that we should take this seriously. The id is identifying a  
document. And a document cannot be wholly in two languages at once.

Once we accept this, then there is no big problem. We just need the  
translations to have 2 different ids. They are after all two  
different documents. We can then put the two translations into the  
same feed, or different feeds without problem.

All we need is to find a way to state that one document is a  
translation of the other. And James Snell's proposal is a good start  
at getting us there (it is a possible translation of the N3 proposal  
in my original e-mail).

> If you want to link the various translations together you can add  
> one or more link elements at the top of the feed with  
> rel="alternate" and hreflang set to the language of the alternate  
> feed. If you're feeling really enthusiastic you can include  
> alternate links pointing to the translated html pages for each  
> entry too.

If my argument above is sound then we should be able to put the  
translations in the same feed. What we want to do then, is find some  
way to state that one entry in the feed is a translation of the other  
entry in the same feed. So though the solution saying that one feed  
is a translation of the other is ok (apart from it breaking what I  
think are the inherent semantics in atom) it is also too general. We  
need more precise tools.

> No extensions need to be made to the Atom spec. No wasted bandwidth  
> by having multiple translations in a feed that may not be used. But  
> you can still publish muliple language simultaneously by making  
> sure you update all feeds simultaneously. You could even publish  
> all the feeds at the same URL, serving the correct translation  
> based on the HTTP Accept-Language header.

Those are cool ideas. But again, it would be good to have a way to  
specify that an entry in one feed is a translation of a particular  
different entry in the other feed.

>
> I may be missing something here, but it seems to me like a  
> reasonable solution to the problem.
>

Those were my first thoughts too btw. And also Retos. :-)

> James

I hope this type of argument also helps show how the semantics we are  
working on at the atom-owl group can help reveal hidden meanings  
lurking in the atom syntax. We do welcome people to help us improove  
it so that it can become an official semantics for atom (as was  
initially specified in the atom charter)

Thanks for your attention,

Henry Story

[1] for some quick thoughts on syntax and semantics see my recent post
http://blogs.sun.com/roller/page/bblfish? 
entry=the_relation_between_xml_and
(not quite right, but nearly there)
[2] something we need to fix in the current atom-owl ontology

---

From: Henry Story <[email-masked]>
Date: Thu, 22 Dec 2005 20:51:21 +0100

I still think it would be good to be able to have to entries in one  
feed and be able to state that they are translations of one another.  
I don't think that putting them in different feeds is going to cover  
all the cases. See below.

On 22 Dec 2005, at 17:45, James M Snell wrote:
> One possibility for this in Atom is to provide multiple Atom  
> documents, each covering their own language.  Given that the date  
> restriction only covers Atom entries within the *same* feed  
> document, the following would be perfectly acceptable:
>
> <feed xml:lang="de">
>   ...
>   <entry>
>     <id>tag:example.org,2005:some_entry</id>
>     ...
>   </entry>
> </feed>
>
> <feed xml:lang="fr">
>   ...
>   <entry>
>     <id>tag:example.org,2005:some_entry</id>
>     ...
>   </entry>
> </feed>
>

[note I change my mind on this half way through the thought]

Ah. I did not read your post carefully enough when posting my  
previous reply to James Holderness.
I don't in fact think this is ok. If the date restriction covers  
entries inside the same feed (which can be millions of entries long  
btw) it has to do so for some good reason. Or else why put the  
restriction in there at all? Now that atom is final, and we cannot  
change the restriction I think we had better assume it is there for a  
very good reason. That will make us all look more intelligent that  
otherwise for one.

So my best explanation is that the id at a time identifies one and  
only one entry - universally, semantically if you will. If books had  
isbn numbers that persisted over editorial changes then this would be  
equivalent to our id. By using the isbn number of a book we could  
then refer to all the transformations of that book. But it would not  
make sense perhaps to have two different books versions published at  
the same time, with the same isbn number... (no that makes  
sense?!)... by the same publisher... Oops. That looks like the feed  
has taken the role of the publisher in the analogy...

(ponders)

Ok. So that would make me change my mind. It should be possible to  
have two entries with the same id and the same updated time stamp but  
different content in two different feeds. This would suggest that the  
feed is really part of the identity of the entry (and that the motto  
"it's the entry stupid" is wrong).

> To indicate that the feeds were translations of one another, a new  
> "translation" link rel could be established on the feed level
>
> <feed xml:lang="de">
>   ...
>   <link rel="translation"
>         hreflang="fr"
>         href="http://.../thefeed?lang=fr"
>   <entry>
>     <id>tag:example.org,2005:some_entry</id>
>     ...
>   </entry>
> </feed>
>
> <feed xml:lang="fr">
>   ...
>   <link rel="translation"
>         hreflang="de"
>         href="http://.../thefeed?lang=de"
>   <entry>
>     <id>tag:example.org,2005:some_entry</id>
>     ...
>   </entry>
> </feed>
>

I still would suggest it be better to have different ids for the  
translations so that they can be put in the same feed. After all, why  
not allow that? If I am a publisher and I am publishing books, why  
could I not simultaneously publish a book and its translation? It  
should be possible then somehow to state that one entry is a  
translation of the other. Is there a way to do what I was going  
trying to state here in N3.  This is really the problem that Simon  
Phipps was looking at.

---

From: "James Holderness" <[email-masked]>
Date: Thu, 22 Dec 2005 20:12:59 -0000

Henry Story wrote:
> I still think it would be good to be able to have to entries in one  feed 
> and be able to state that they are translations of one another.  I don't 
> think that putting them in different feeds is going to cover  all the 
> cases. See below.

Fair enough. Simon certainly seemed to have a valid use case for that.

> translations so that they can be put in the same feed.

I agree. I never said so in my original message but even if the entries are 
in different feeds I think their ids should be different. An identical id to 
me implies an identical entry (which an Atom processor can assume is a 
duplicate or an update).

> It should be possible then somehow to state that one entry is a 
> translation of the other. Is there a way to do what I was going  trying to 
> state here in N3.  This is really the problem that Simon  Phipps was 
> looking at.

Why not just have @rel='alternate' links on each atom:entry which point to 
their translations. Now I realise it's not possible to point to a specific 
entry in a feed, but you can point to its html permalink with 
@type='text/html'. Alternatively you could create a separate Atom document 
for each entry and then each entry in the feed would include a @rel='self' 
link pointing to their corresponding document with 
@type='application/atom+xml' as well as a @rel='alternate' links pointing to 
the Atom documents of any translations (also with 
@type='application/atom+xtml').

Regards
James