Subject: Fwd: I-D ACTION:draft-chakrabarti-ipv6-addrselect-api-05.txt

From: Julien Laganier <[email-masked]>
Date: Mon, 12 Mar 2007 10:31:26 +0100

Folks,

An updated version of the IPv6 address selection API 
draft has been published (see below). Over the years, 
this draft has been reviewed by many participants to 
the IPv6 WG, and has been supported by many of them. 
Their feedback has been incorporated in successive 
revision of the draft. There are also serious 
implementers waiting for this draft to be published.

Therefore, we'd like this draft to be published as an 
Informational RFC.

What do you think?

Best,

--julien

-------------------------------------------------------------
A New Internet-Draft is available from the on-line 
Internet-Drafts 
directories.

	Title		: IPv6 Socket API for Address Selection
	Author(s)	: E. Nordmark, et al.
	Filename	: 
draft-chakrabarti-ipv6-addrselect-api-05.txt
	Pages		: 34
	Date		: 2007-3-7
	
The IPv6 default address selection document [RFC3484] 
describes the
   rules for selecting source and destination IPv6 
addresses, and
   indicates that applications should be able to 
reverse the sense of
   some of the address selection rules through some 
unspecified API.
   However, no such socket API exists in the basic 
[RFC3493] or advanced
   [RFC3542] IPv6 socket API documents.  This document 
fills that gap by
   specifying new socket level options and flags for 
the getaddrinfo()
   API to specify preferences for address selection 
that modify the
   default address selection algorithm.  The socket API 
described in
   this document will be particularly useful for IPv6 
applications that
   want to choose between temporary and public 
addresses, and for Mobile
   IPv6 aware applications that want to use the care-of 
address for
   communication.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-chakrabarti-ipv6-addrselect-api-05.txt

To remove yourself from the I-D Announcement list, send 
a message to 
[email-masked] with the word unsubscribe 
in the body of 
the message. 
You can also visit 
https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. 
Login with the 
username "anonymous" and a password of your e-mail 
address. After 
logging in, type "cd internet-drafts" and then 
"get draft-chakrabarti-ipv6-addrselect-api-05.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-chakrabarti-ipv6-addrselect-api-05.txt".
	
NOTE:	The mail server at ietf.org can return the 
document in
	MIME-encoded form by using the "mpack" utility.  To 
use this
	feature, insert the command "ENCODING mime" before the 
"FILE"
	command.  To decode the response(s), you will need 
"munpack" or
	a MIME-compliant mail reader.  Different 
MIME-compliant mail readers
	exhibit different behavior, especially when dealing 
with
	"multipart" MIME messages (i.e. documents which have 
been split
	up into multiple messages), so check your local 
documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant 
mail reader
implementation to automatically retrieve the ASCII 
version of the
Internet-Draft.

Subject: I-D ACTION:draft-chakrabarti-ipv6-addrselect-api-05.txt
From: [email-masked]
To: [email-masked]
Date: Wed, 07 Mar 2007 15:50:02 -0500
List-Id: i-d-announce.ietf.org

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title		: IPv6 Socket API for Address Selection
	Author(s)	: E. Nordmark, et al.
	Filename	: draft-chakrabarti-ipv6-addrselect-api-05.txt
	Pages		: 34
	Date		: 2007-3-7
	
The IPv6 default address selection document [RFC3484] describes the
   rules for selecting source and destination IPv6 addresses, and
   indicates that applications should be able to reverse the sense of
   some of the address selection rules through some unspecified API.
   However, no such socket API exists in the basic [RFC3493] or advanced
   [RFC3542] IPv6 socket API documents.  This document fills that gap by
   specifying new socket level options and flags for the getaddrinfo()
   API to specify preferences for address selection that modify the
   default address selection algorithm.  The socket API described in
   this document will be particularly useful for IPv6 applications that
   want to choose between temporary and public addresses, and for Mobile
   IPv6 aware applications that want to use the care-of address for
   communication.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-chakrabarti-ipv6-addrselect-api-05.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-chakrabarti-ipv6-addrselect-api-05.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-chakrabarti-ipv6-addrselect-api-05.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www1.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Mon, 12 Mar 2007 18:08:18 +0200

> implementers waiting for this draft to be published.

This new version still breaks ABI compatibility, meaning any 
implementors would have to rebuild any software using getaddrinfo 
against a new library.

This is an example code that shows one breakage case: ai_eflags gets an 
undefined value. You simply cannot change the size of a known structure 
in C APIs (unless you pass a size value, such as bind() and connect() 
do).

http://www.remlab.net/files/divers/eaddr.c

There are use-case for this; I DO know applications using this kind of 
construct.

To avoid this, either a new API is defined (e.g. geteaddrinfo with a 
struct eaddrinfo), or preference flags are stored in ai_flags which 
still has more than enough bits available for this purpose.

Also, what is the rationale for not following the current practice of 
using -1 as the system default for setsockopt, instead of requiring the 
application to store the value? That only "kills" one bit, but will 
allow for the corner-case race condition where the local default has 
changed in-between, and simplify implementors life a lot (there are so 
many case where you just pass socket handles through different layers, 
and simply cannot pass anything else without large changes).

-- 
Rémi Denis-Courmont
http://www.remlab.net/

---

From: Julien Laganier <[email-masked]>
Date: Mon, 12 Mar 2007 17:50:26 +0100

Rémi, 

First, please note that ABI does change with time, 
that's why we have APIs :)

Further, the new API does change the ABI, but it does 
not break backward ABI compatibility: 

- compile an old version of the library on an old 
system without the new API so that is conforms to the 
old ABI. 

- compile a piece of code on a old system without the 
new API so that it conforms to the old ABI. 

- link the piece of code conforming to the old ABI with 
a library conforming to the old ABI. 

- run the resulting old ABI-conforming binary on a new 
system with the new ABI: it works.

Now what your example essentially does is: 

- compile an old library on an old system without the 
new API so that is conforms to the old ABI. 

- compile a piece of code on a new system with the new 
API so that it conforms to the new ABI. 

- link the piece of code conforming with the new ABI 
with a library conforming to an old ABI. 

Surprise: it breaks! 

Regarding the need for an application to save defaults 
if it want to restore them, I don't see that saving 
requirement as an issue. As we saw before, an 
application has already many things to save (e.g. save 
a value of the pointer to an addrinfo structure if it 
wants to free it :)

--julien

On Monday 12 March 2007 17:08, Rémi Denis-Courmont 
wrote:
> > An updated version of the IPv6 address selection
> > API draft has been published (see below). Over the
> > years, this draft has been reviewed by many
> > participants to the IPv6 WG, and has been
> > supported by many of them. Their feedback has been
> > incorporated in successive revision of the draft.
> > There are also serious implementers waiting for
> > this draft to be published.
>
> This new version still breaks ABI compatibility,
> meaning any implementors would have to rebuild any
> software using getaddrinfo against a new library.
>
> This is an example code that shows one breakage
> case: ai_eflags gets an undefined value. You simply
> cannot change the size of a known structure in C
> APIs (unless you pass a size value, such as bind()
> and connect() do).
>
> There are use-case for this; I DO know applications
> using this kind of construct.
>
> To avoid this, either a new API is defined (e.g.
> geteaddrinfo with a struct eaddrinfo), or preference
> flags are stored in ai_flags which still has more
> than enough bits available for this purpose.
>
>
> Also, what is the rationale for not following the
> current practice of using -1 as the system default
> for setsockopt, instead of requiring the application
> to store the value? That only "kills" one bit, but
> will allow for the corner-case race condition where
> the local default has changed in-between, and
> simplify implementors life a lot (there are so many
> case where you just pass socket handles through
> different layers, and simply cannot pass anything
> else without large changes).

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Mon, 12 Mar 2007 19:07:54 +0200

> with a library conforming to an old ABI.

Of course it breaks. I am well aware that there some systems have 
versioning capabilities and all to handle this... But, err, we are 
dealing with the system C library here! Breaking that means you have to 
rebuild and update the whole system (well, the C library, and every 
thing that depends on it, so almost the whole system). Very hard in 
general, and outright IMPOSSIBLE if you happen to support a platform 
with third party closed-source software.

Continuing this way, you can be sure that some OS vendors will either 
not implement this, or do it in a non-standard way. With these 
considerations, it is much more practical to define a new API name.

Besides, since in anycase, addrselect-aware applications and libraries 
will need source code modification, what is the problem with the latter 
approach?

> a value of the pointer to an addrinfo structure if it

1/ That's completely different. The addrinfo structure lifetime is much 
shorter than that of a network socket, both in term of source code 
path, and real time clock.
2/ That's not at all an excuse for *both* deviating from existing 
established practices AND making the API harder to use. That makes TWO 
(however minor that may be) inconveniences, with ZERO benefits.

These issues were already raised more than 6 months ago. Are you trying 
to optimize for one specific vendor that would have already shipped a 
draft implementation ?? I have been asking for a rationale for as long, 
and still haven't received any sane answer. I really don't understand 
why you stick that hard to a broken design. In terms of spec work, it's 
very easy to fix; in fact, it would most likely make the spec shorter.

---

From: Julien Laganier <[email-masked]>
Date: Mon, 12 Mar 2007 18:14:53 +0100

Overlooked that part of On Monday 12 March 2007 17:08, 
> software using getaddrinfo against a new library.

[...]

> than enough bits available for this purpose.

The reason for introduction of the extended flag is 
that indeed someone complained that the ai_flags field 
will get crowded, and worse, to IPv6 benefit only.

--julien

---

From: Julien Laganier <[email-masked]>
Date: Mon, 12 Mar 2007 18:54:28 +0100

Rémi,

On Monday 12 March 2007 18:07, Rémi Denis-Courmont 
wrote:
> > - compile an old library on an old system without
> > the new API so that is conforms to the old ABI.
> >
> > - compile a piece of code on a new system with the
> > new API so that it conforms to the new ABI.
> >
> > - link the piece of code conforming with the new
> > ABI with a library conforming to an old ABI.
> >
> > Surprise: it breaks!
>
> Of course it breaks. 

That was irony, FWIW.

> I am well aware that there some 
> systems have versioning capabilities and all to
> handle this... But, err, we are dealing with the
> system C library here! Breaking that means you have
> to rebuild and update the whole system (well, the C
> library, and every thing that depends on it, so
> almost the whole system). Very hard in general, and
> outright IMPOSSIBLE if you happen to support a
> platform with third party closed-source software.

My point has been that we do not break backward ABI 
compatibility

Rebuilding the whole OS is what you do when you update 
the C library.

Now if you're using closed source libraries for the old 
version of the OS, you can still use software compiled 
on the old OS and linked against that old library.

It's not the problem of the OS or its ABI compatibility 
if you insist on linking software compiled on the new 
OS with outdated libraries supporting the old OS.

It's the probelm of your third party software vendor: 
You should just ask straight to him for new versions 
of these libraries compiled on the new system. 

> Continuing this way, you can be sure that some OS
> vendors will either not implement this, or do it in
> a non-standard way. With these considerations, it is
> much more practical to define a new API name.
>
> Besides, since in anycase, addrselect-aware
> applications and libraries will need source code
> modification, what is the problem with the latter
> approach?

We cannot invent new API names and functions each time 
someone insists on using an old library on a new 
system and link it with new software.

> > Regarding the need for an application to save
> > defaults if it want to restore them, I don't see
> > that saving requirement as an issue. As we saw
> > before, an application has already many things to
> > save (e.g. save a value of the pointer to an
> > addrinfo structure if it wants to free it :)
>
> 1/ That's completely different. The addrinfo
> structure lifetime is much shorter than that of a
> network socket, both in term of source code path,
> and real time clock.
> 2/ That's not at all an excuse for *both* deviating
> from existing established practices AND making the
> API harder to use. That makes TWO (however minor
> that may be) inconveniences, with ZERO benefits.

First of all, note that I don't care much about this. 
It has been in the draft since the beginning, and 
besides you nobody ever complained about it. 

Regarding the point you made in your previous note:

> [...] simplify implementors life a lot (there are so 
> many case where you just pass socket handles through

So here you mean that you cannot pass that saved system 
default without rewriting the software, right?

But presumably, if the code want to reset system 
default on a given socket, it has to know they've been 
altered, right? So still you need to pass that 
something else that says "address selection 
preferences altered"

Or do you assume such code will blindly reset socket 
preference each time a socket handles passes through 
layer boundaries? 

So IMHO your point doesn't raise an issue with the 
current design.

> These issues were already raised more than 6 months
> ago. Are you trying to optimize for one specific
> vendor that would have already shipped a draft
> implementation ?? 

Rémi, please abstain to suggest such things, this is 
not helping the discussion we're having here.

> I have been asking for a rationale for as long, and
> still haven't received any sane answer. 

I hope you do not find my answers insane...

> I really don't understand why you stick that 
> hard to a broken design. 

Please see above.

What is IMHO broken is 1) linking code compiled on a 
new system with libraries compiled an old system, 
dynamic linking isn't like composing a bouquet, and 2) 
blindingly resetting socket address selection 
preference because someone doesn't want to rewrite 
code.

--julien

---

From: James Carlson <[email-masked]>
Date: Mon, 12 Mar 2007 14:08:20 -0400

> OS with outdated libraries supporting the old OS.

It doesn't matter much whose fault it is; it fails, it fails
"silently" (underlying attempts at interface versioning won't help
this specific problem), and it blows away the application.

> of these libraries compiled on the new system. 

Why should we require that third party software vendors recompile in
order to have their software continue working on new systems?  Isn't
avoiding that problem exactly the point of having ABI compatibility?

We end up in the strange position where old third-party applications
continue to work, but old third-party libraries may fail depending on
the vintage of the application that uses them.

I'm not sure I understand why ABI compatibility should extend to
"applications" alone and not specifically to libraries.  Are libraries
less interesting or easier to recompile?

(It seems to me that a new "AI_EXTPRESENT" entry in ai_flags would do
the trick, though, in all cases other than attempted structure copies
of 'addrinfo'.)

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 1 Network Drive         71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

---

From: James Carlson <[email-masked]>
Date: Mon, 12 Mar 2007 14:31:06 -0400

Julien Laganier writes:
> What breaks is linking software compiled on the new 
> system to library compiled on the old system.

That should still work.

What breaks is a library that does something "strange" with the
structure.  The only issue, I think, is whether that strangeness is
something warranted.

> > (It seems to me that a new "AI_EXTPRESENT" entry in
> > ai_flags would do the trick, though, in all cases
> > other than attempted structure copies of
> > 'addrinfo'.)
> 
> This is exactly what the draft says: 

Right; that was a typo.  I did mean AI_EXTFLAGS.  I was saying that
the mechanism appears to be sufficient provided that structure copy
isn't an important usage.

---

From: Julien Laganier <[email-masked]>
Date: Mon, 12 Mar 2007 19:23:01 +0100

James,

On Monday 12 March 2007 19:08, James Carlson wrote:
> Julien Laganier writes:
> > It's not the problem of the OS or its ABI
> > compatibility if you insist on linking software
> > compiled on the new OS with outdated libraries
> > supporting the old OS.
>
> It doesn't matter much whose fault it is; it fails,
> it fails "silently" (underlying attempts at
> interface versioning won't help this specific
> problem), and it blows away the application.
>
> > It's the probelm of your third party software
> > vendor: You should just ask straight to him for
> > new versions of these libraries compiled on the
> > new system.
>
> Why should we require that third party software
> vendors recompile in order to have their software
> continue working on new systems?  Isn't avoiding
> that problem exactly the point of having ABI
> compatibility?

Third party software compiled on the old system 
continues to work on the new system. That's ABI 
compatibility.

> We end up in the strange position where old
> third-party applications continue to work, but old
> third-party libraries may fail depending on the
> vintage of the application that uses them.

Old third-party applications using old third-party 
libraries continue to work. 

Linking software compiled on new systems with old 
libraries of the old system breaks. AFAIK this is 
nothing new and happened before the proposed API.

> I'm not sure I understand why ABI compatibility
> should extend to "applications" alone and not
> specifically to libraries.  Are libraries less
> interesting or easier to recompile?
>

A new flag (AI_EXTFLAGS) is defined for the "ai_flags" 
flag-set field of the addrinfo data structure to tell
the system to look for the "ai_eflags" extended
flag-set field in the addrinfo structure.  It is
defined in the <netdb.h> header:

      AI_EXTFLAGS /* extended flag-set present */

--julien

---

From: James Carlson <[email-masked]>
Date: Mon, 12 Mar 2007 14:28:16 -0400

> OS with outdated libraries supporting the old OS.

Actually, in thinking about it a bit more, I think the real problem is
with a strange usage case -- copying the addrinfo structure inside a
library will lead to a garbage result.

It isn't just the old library that's the problem; it's the old library
attempting to manipulate a structure that 'belongs' to someone else
and thus getting it wrong.

I agree that AI_EXTFLAGS ought to be more than sufficient for
reasonable applications.

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Mon, 12 Mar 2007 20:55:06 +0200

Le lundi 12 mars 2007 20:28, James Carlson a écrit :
> Actually, in thinking about it a bit more, I think the real problem
> is with a strange usage case -- copying the addrinfo structure inside
> a library will lead to a garbage result.

Or the other way, a rebuilt library against a non rebuilt software would 
yield to an out-of-bound memory read, as the library source code won't 
know to check for AI_EXTFLAGS bit and withdraw some bytes from sizeoz() 
if it's not set -- Currently, POSIX says this about the *hints* 
parameter:

"If the argument hints is not null, it refers to a structure containing 
input values that may direct the operation by providing options and by 
limiting the returned information to a specific socket type, address 
family and/or protocol. In this hints structure every member other than 
ai_flags, ai_family, ai_socktype and ai_protocol must be zero or a null 
pointer."

All of this four fields are integer and hence valid candidates for 
memory coypying, same (in general) for zeros/NULL. Maybe, instead of 
plain memory copy, one should memset, and copy any of the four allowed 
parameters. That makes no difference to my test case though, the 
AI_EXTFLAGS will be incorrectly passed through all the same, and the 
same error will occur afterward.

So there is a known, if minor, problem, and a very low cost solution: 
Two new APIs and one struct (there are >1000 anyway). Also, 
generic/system-wide source address selection should and will be relied 
upon by most software. IMHO, better put the burden on those that use 
it, than on everything else.

Regards,

-- 
Rémi Denis-Courmont
http://www.remlab.net/

---

From: James Carlson <[email-masked]>
Date: Mon, 12 Mar 2007 16:07:29 -0400

Rémi Denis-Courmont writes:
> memory coypying,

Isn't setting ai_flags to a value provided to you by someone else, but
that contains bits you don't understand, essentially equivalent to
setting arbitrary undefined bits?

I think the point is that copying isn't very safe.

> it, than on everything else.

I don't see how a new API will help.  Won't the old code still use the
old interface but with the new bits provided by the new application?

There's no way that the application can know that the library calls
getaddrinfo instead of geteaddrinfo (or some such).

If you create a different struct and make sure that such an
application would fail to compile properly (can't compile against the
old headers, and can't run-time link because you need the new
library), that might work.

Conservative implementations might rename the interface anyway (using
#define or redefine_extname or similar) and force the unknown bits
off.  That'd produce a different set of breakage, but at least
produces obvious results and doesn't require a different API.

---

From: JINMEI Tatuya / 神明達哉<[email-masked]>
Date: Tue, 13 Mar 2007 16:23:23 +0900

At Mon, 12 Mar 2007 18:14:53 +0100,

> will get crowded, and worse, to IPv6 benefit only.

I guess that "someone" includes (or is) me.  I was and am against
about abusing the neutral ai_xxx flag space for purposes specific to
one particular address family (I know we already have such flags; I
argued we shouldn't do it this time).  I thought I also pointed out
the available space might not be that large since it's an 'int' field
(which may be 16-bit long).

(The "abuse" aspect of the issue is not resolved even with ai_eflags,
though, so I'm personally not 100% happy with the current
specification).

Regarding the example that would be broken by the extension to the
addrinfo structure, it seems to me artificial and very rare (if it
ever happens).  It's not surprising that the extension is unfriendly
with some tricky usage, but IMO the point is whether the benefit of
the extension outweighs the possible breakages.  I'm actually not sure
if I have a clear opinion on it; on one had, the breakage example
given so far looks artificial as I said above (so a concrete example
of an existing application would be helpful if it exists).  On the
other hand, I don't see much benefit in this extension either,
because, as far as I can see, only very few and special type of
applications want to use it.

In conclusion, I'd personally not support this document to be
published; however, I don't want to be a show-stopper just due to some
philosophical arguments either.  If I were to vote, I'd thus probably
abstain.

					JINMEI, Tatuya
					Communication Platform Lab.
					Corporate R&D Center, Toshiba Corp.
					[email-masked]