Subject: IETF and APIs

From: Sam Hartman <[email-masked]>
Date: Tue, 29 Mar 2011 04:13:48 -0400

At the mic at the technical plenary last night, I made a comment that I
strongly supported the IETF doing API work.

I've realized that people may have assumed I meant that it would be
appropriate for the IETF to specify an API and not a protocol for some
given task.

That's not what I meant, so I am writing to clarify.

I think that multiple levels of interoperability will be required for
building blocks used in platforms including the web platform.

we're the IETF. We should definitely specify protocols for the building
blocks we create.

However, one problem that hurts interoperability is when platforms
provide different APIs or abstract interfaces to applications.  As an
example, when we worked on TLS channel bindings to other protocols, we
realized that not all TLS implementations provided access to the
information we need to construct these channel bindings.

Especially as people are trying to build IETF technologies into
platforms, it would be strongly desirable to specify what interfaces
applications can depend on.

I think that we should move more into that business.
I see no problem with actually specifying a language-specific API when
the WG involved has the skills to do a good job.
When we do not, specifying abstract interfaces we expect platforms to
provide still has significant value.

That, rather than APIs instead of protocols, is what I advocate.

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 29 Mar 2011 10:53:32 +0200

> building blocks used in platforms including the web platform.

"Multiple levels of interoperability" sounds interesting, and very possibly 
quite important.

Can you elaborate on its meaning?

> provide different APIs or abstract interfaces to applications.
...

> the WG involved has the skills to do a good job.

Wow.  What is the list of languages we should work on?  C, C++, Javascript, 
Java, Python, ...?

Which is not to deny the problem you cite:  APIs differ and this hurts 
interoperability.

One approach to solving it is, indeed, to specify /all/ of the APIs that map to 
the protocol.

Another is to do more and better interoperability testing and let the API 
developers see their deficiencies and fix them.  The benefit of this is that it 
moves the problem to the folks with the knowledge and incentives to work on it 
and it takes this very expensive specification task out of the IETF's critical path.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Dave Cridland <[email-masked]>
Date: Tue, 29 Mar 2011 10:03:45 +0100

On Tue Mar 29 09:53:32 2011, Dave CROCKER wrote:
> Wow.  What is the list of languages we should work on?  C, C++,  
> Javascript, Java, Python, ...?
> 
> 
COBOL, obviously.

More seriously, C has the benefits that an actual C API can often be  
rapidly pulled into another language, and if reasonably well designed  
can be remodelled for other languages easily.

> Another is to do more and better interoperability testing and let  
> the API developers see their deficiencies and fix them.  The  
> benefit of this is that it moves the problem to the folks with the  
> knowledge and incentives to work on it and it takes this very  
> expensive specification task out of the IETF's critical path.

Right, but that's in line (more or less) with what Sam went on to say  
in the paragraph you snipped:

"When we do not, specifying abstract interfaces we expect platforms  
to provide still has significant value".

It'll often be sufficient to point out the shortcomings and specify  
what data needs to be accessed and (roughly) what form.

I'm all in favour of this.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Sam Hartman <[email-masked]>
Date: Tue, 29 Mar 2011 05:17:42 -0400

>>>>> "Dave" == Dave CROCKER <[email-masked]> writes:

    Dave> On 3/29/2011 10:13 AM, Sam Hartman wrote:
    >> 
>
> At the mic at the technical plenary last night, I made a comment that
    >> I strongly supported the IETF doing API work.
    >> 
    >> I've realized that people may have assumed I meant that it would
    >> be appropriate for the IETF to specify an API and not a protocol
    >> for some given task.
    >> 
    >> I think that multiple levels of interoperability will be required
    >> for building blocks used in platforms including the web platform.

    Dave> "Multiple levels of interoperability" sounds interesting, and
    Dave> very possibly quite important.

One level is the traditional protocol interoperability we normally
discuss.

Another level shows up when you want to write a cross-platform
application.  It's not good enough if Windows has some API. I want to
have confidence that functionality is available on Windows, OSX, Linux
and some of the mobile platforms before I depend on that functionality
in a cross-platform API.

Within the web platform, I want to make sure functionality is available
on multiple browsers before I depend on it in my cross-browser
application.

    >> we're the IETF. We should definitely specify protocols for the
    >> building blocks we create.
    >> 
    >> However, one problem that hurts interoperability is when
    >> platforms provide different APIs or abstract interfaces to
    >> applications.
    Dave> ...

    >> I think that we should move more into that business.  I see no
    >> problem with actually specifying a language-specific API when the
    >> WG involved has the skills to do a good job.

    Dave> Wow.  What is the list of languages we should work on?  C,
    Dave> C++, Javascript, Java, Python, ...?
Any of the above when it makes sense for a WG to do the work.
I'd expect that typically you'd only specify one or two language
bindings for a given interface.
You should have a need to do the work, have the necessary skills to have
probable success, have the necessary skills to get review and have
people volunteer to do the work.

    Dave> Which is not to deny the problem you cite: APIs differ and
    Dave> this hurts interoperability.

    Dave> One approach to solving it is, indeed, to specify /all/ of the
    Dave> APIs that map to the protocol.

    Dave> Another is to do more and better interoperability testing and
    Dave> let the API developers see their deficiencies and fix them.
    Dave> The benefit of this is that it moves the problem to the folks
    Dave> with the knowledge and incentives to work on it and it takes
    Dave> this very expensive specification task out of the IETF's
    Dave> critical path.

My experience is that protocol interoperability testing does not
actually lead to cross-platform interoperability.  This is especially
true for building blocks intended to be used by components that are
developed later.

The issue is that the people doing interoperability testing at the
protocol layer don't tend to be testing for whether things work the same
way between platforms.

It is when you try and build something on top of a building block that
you notice the problem.
You tend to notice at one of two layers.
The first is if you standardize the higher level item and have
participants familiar with multiple platforms involved in the
standardization.
You can then discover that you don't have sufficient overlapping
functionality on platforms to do what you want.

Another case where you discover the problem is when you implement and
people discover that they cannot  do so.

Factors that contribute to cross-platform interoperability issues
include the following. Often, the people designing and implementing the
building block are not the same as the people using the building
block. Often, there is separation in time between building block design
and building block usage. Often, various factors complicate changing the
platform to expose new functionality.

In conclusion, in cases where these types of issues are likely to be
important, I believe we should do work. Work can include work on
abstract interfaces, which will be easier to justify. Work can include
concrete APIs which will sometimes be appropriate. I would prefer that
this work be standards track not information. Discussions of how to
advance MIBs, GSS-API and other standards-track elements already give us
approaches to judge  interoperability for such elements.

--Sam

---

From: "R. B." <[email-masked]>
Date: Tue, 29 Mar 2011 11:25:14 +0200

2011/3/29 Dave Cridland <[email-masked]>

>> Javascript, Java, Python, ...?

Nah...  INTERCAL or APL

> rapidly pulled into another language, and if reasonably well designed can be
> remodelled for other languages easily.
>
>
>
Although I am not a C fan, I support the idea of (at list) a C-like syntax
to specify the API because of its diffusion and also because, as you say, it
is quite easy to translate a C-specified API to other languages.

>> developers see their deficiencies and fix them.  The benefit of this is that
>> it moves the problem to the folks with the knowledge and incentives to work
>> on it and it takes this very expensive specification task out of the IETF's
>> critical path.
>>
>
> Right, but that's in line (more or less) with what Sam went on to say in
> the paragraph you snipped:
>
>
> "When we do not, specifying abstract interfaces we expect platforms to
> provide still has significant value".
>
> It'll often be sufficient to point out the shortcomings and specify what
> data needs to be accessed and (roughly) what form.
>

Let me add my +1e10 to the idea that we should (SHOULD?) specify at least a
minimum API, at least at a very abstract level.  I run in this type of
problem recently.  We were discussing about using DCCP for video streaming
and it would have been useful to be able to access from the application the
allowed rate estimated by DCCP.  Missing a standard API, it is not clear if
this is possible.  Maybe the possibility depends on your software vendor,
making portability a nightmare....

I would also like to add my thoughts (0.02) about how the API specification.
 You can have a very abstract description like

   ... the API for the Fast Overlay Obfuscation (FOO) protocol SHOULD allow
the programmer to change the Maximum_Obfuscation_Rate, to select an
Obfuscation_Procedure, ...

This is of course language independent and it describes the constraint that
the API should verify.  On the cons side, it could be that several vendors
implement the abstract description in different way, giving rise to a new
portability nightmare. The alternative to such an abstract description could
be something like

   ... the API for the Fast Overlay Obfuscation (FOO) protocol SHOULD
include at least the following functions

      set_Maximum_Obfuscation_Rate (int rate);
      set_Obfuscation_Procedure (int procedure_index);

This clearly depends on the chosen language. Moreover, if some of the data
used in the API are quite complex (I am thinking, for example, to a X.500
certificate) a syntactically valid description of the API could be very
large.  On the other hand, one would have more uniformity between different
implementations. I do not know which solution is the best one.  I guess it
will depend on the context.

> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 29 Mar 2011 12:08:55 +0200

> on multiple browsers before I depend on it in my cross-browser

With respect to application-level system design, I think your distinction is 
equivalent to component-testing vs. integrated system testing.  Within the IETF, 
it should be a distinction between lower-level end-to-end and upper-level 
end-to-end.

I certainly agree that upper-level end2end testing is essential.

The Other Dave C's highlighting the possibility of an "abstract" API is also 
worth considering.

d/
-- 

   Dave Crocker

---

From: Simon Josefsson <[email-masked]>
Date: Tue, 29 Mar 2011 12:28:51 +0200

Dave CROCKER <[email-masked]> writes:

> The Other Dave C's highlighting the possibility of an "abstract" API
> is also worth considering.

Yes.  Today I would prefer to define abstract APIs in the IETF and let
implementers or other organizations map them to their language or
environment of choice.

The IETF is not the best place to go for implementation wisdom these
days, unfortunately.  Specifying APIs could make more API designers come
to the IETF, which would be good, but I fear the IETF will just produce
poor APIs until a sufficient mass of clueful people have joined.  I
don't want to see another GSS-API.

A compromise could be to specify one abstract APIs on the standards
track and allow multiple rendition of the abstract APIs in various
programming languges.  Then if someone dislikes a particular concrete
API enough, they could propose a better approach.

/Simon

---

From: Eric Burger <[email-masked]>
Date: Tue, 29 Mar 2011 13:28:55 +0200

Would we not be better off just asking (mandating?) at least one open source implementation?  That effort would produce a de facto API.

---

From: Chris Elliott <[email-masked]>
Date: Tue, 29 Mar 2011 13:47:41 +0200

> Would we not be better off just asking (mandating?) at least one open source implementation?  That effort would produce a de facto API.

To co-opt wording from another thread, I think you are conflating an (even de facto) API with an application. I've seen many applications that implement protocols without clearly delineating an interface to the protocol as anything approaching an API. Even if there is a de facto API, it is often unusable unless it was designed as an API fr the ground up and didn't simply grow out of the need to abstract the protocol from the application during development.

Chris.

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Tue, 29 Mar 2011 13:53:23 +0200

> Would we not be better off just asking (mandating?) at least one open source implementation?  That effort would produce a de facto API.

What you need is a reference implementation (i.e. an inefficient but complete 
implementation, with a license that permits at least to see the source and 
generate a binary) AND a test suite.

>>>>>>> "Dave" == Dave CROCKER<[email-masked]>  writes:
>>

-- 
Marc Petit-Huguenin
Personal email: [email-masked]
Professional email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org

---

From: Dave Cridland <[email-masked]>
Date: Tue, 29 Mar 2011 13:14:06 +0100

On Tue Mar 29 12:28:55 2011, Eric Burger wrote:
> Would we not be better off just asking (mandating?) at least one  
> open source implementation?  That effort would produce a de facto  
> API.

Unfortunately this doesn't work in practise.

One of the examples that Sam raised originally concerned TLS, and  
that has (at least) three different open-source implementations in  
library form, all with fairly different APIs.

Some effort has been made to provide basic functionality in some  
libraries using an OpenSSL compatible layer, but given OpenSSL's API  
is only sparingly documented, I'm not sure of the utility.

Dave.

---

From: Sam Hartman <[email-masked]>
Date: Tue, 29 Mar 2011 09:43:34 -0400

>> open source implementation?  That effort would produce a de facto

    Marc> What you need is a reference implementation (i.e. an
    Marc> inefficient but complete implementation, with a license that
    Marc> permits at least to see the source and generate a binary) AND
    Marc> a test suite.

Others have pointed out some of the issues with this approach.  However,
having been involved in a fairly widely used reference implementation
(MIT Kerberos) for over 10 years and having watched it try and evolve
from a reference implementation to a platform component, I do not
believe that  the sorts of things you want out of a reference
implementation are what you want out of a platform component. 

A reference implementation focuses on demonstrating a protocol and being
able to develop and debug that protocol and understand it.

A platform component focuses on extensibly making the aspects of the
protocol that can be building blocks available to the platform.

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 29 Mar 2011 22:44:41 +0800

> is also worth considering.

You need to look at WebIDL <http://www.w3.org/TR/WebIDL>.  This is what the W3C use.

One size does not fit all language paradigms.  You have to pick something.

--Martin

---

From: Scott Brim <[email-masked]>
Date: Tue, 29 Mar 2011 17:53:03 +0200

On Tue, Mar 29, 2011 at 14:14, Dave Cridland <[email-masked]> wrote:
>> Would we not be better off just asking (mandating?) at least one open
>> source implementation?  That effort would produce a de facto API.
>
> One of the examples that Sam raised originally concerned TLS, and that has
> (at least) three different open-source implementations in library form, all
> with fairly different APIs.

"API" has too many meanings.  In this case there is the "API" between
app and library, and "API" in the sense of a client/server protocol on
the net.  I've learned a few new meanings of API e.g. from cloud
people.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 29 Mar 2011 17:03:10 +0100

On Tue Mar 29 16:53:03 2011, Scott Brim wrote:
> open
> that has

Well, for the avoidance of doubt, I took Sam's comment as being  
exclusively about the traditional meaning of API, that is, the  
interface between app and library. In the (inaccurate and annoying)  
protocol sense, the IETF is obviously well-placed to define standard  
protocols.

Dave.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 29 Mar 2011 18:25:53 +0200

What we are really talking about here is

1) Defining the boundary between one layer and another
2) Specifying a means of reifying that boundary as something that is
concrete of which the most familiar form is an API

We discuss APIs all the time in IETF. Some examples:

"We can't use X because Javascript does not expose that to the application"
"We can't do Y because CryptoLibs don't show that to applications"
"We can't move from IPv4 to IPv6 as applications use gethostbyname (x) to do
everything"

The limitations of the BSD sockets library have affected (nay controlled!)
application protocol design for decades.

Having the API do too little can make things worse, not better. The current
networking API for networks requires applications to be really very aware of
whether their transport layer is

* IPv4 or IPv6
* Layered over raw TCP, SSL or IPSEC
* Uses A record, SRV or URI discovery

All of which are things that in my view the application should not care
about at all. A pipe is a pipe and the application should be able to
abstract above the specific type of pipe.

The application MAY care about the trust properties of a pipe but should not
be required to care about the precise details of how those trust properties
are achieved.

Web IDL might be a way to go here. It does at least permit the use of tools
and it avoids the risk of a discussion of what the best language to use for
specifying an API is.

The downside is that people would need the tools for their specific
languages. But that is probably not very difficult to write. I have for
years written that type of stuff for mapping abstract protocols to XML
schema rather than have to re-write a spec every time people come up with a
new way to use XML to address the problem.

Whatever we go with should be object oriented for specifying interfaces to
the application layer and should not be object oriented for lower layers.
Modern programming approaches are now object oriented, or at least use
inheritance and encapsulation and a number of properties that we have agreed
to call object oriented. Even scripting languages tend to have objectish
features.

At the lower layers, objectish features are probably a bad idea, just like
regular expressions and recursive structures and many other powerful tools
that are a little too powerful.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 29 Mar 2011 22:59:59 +0200

Sam,

Thanks for your comments about APIs.  I have already had a very vibrant
debate with some folks about this in another context.  I would simply
suggest here that the IETF has mixed success with APIs, especially
abstract APIs, and that while I wouldn't object to us "going there",
given that we've "been there", I would ask that we wade carefully
further into this space deliberately, recognizing what has worked, and
what has not.  The hardest part of this exercise, I'm afraid, will be on
agreeing which fit which category.  Also, it may not be appropriate to
develop APIs in all cases.  Documenting dome understanding of when that
effort likely to succeed would be VERY useful.

Regards,

Eliot

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Mar 2011 01:11:44 -0700

Hi, all,

Perhaps we're not talking about an API, or even an abstract API, but 
just the "application interface" (or just "upper layer service" 
specification).

RFC793 is a great example that a protocol provides a service, and that 
service needs to be explained - and that explaining it does NOT need to 
be done in a specific language.

Joe

---

From: "t.petch" <[email-masked]>
Date: Wed, 30 Mar 2011 10:21:16 +0200

----- Original Message ----- 
From: "Joe Touch" <[email-masked]>
To: "Sam Hartman" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Wednesday, March 30, 2011 10:11 AM
> be done in a specific language.

Not a specific language, no, but in a highly recognisable could-be-a-language,
at least for those of us old enough to recall the GoTo-less programming that 
preceded Object-Oriented programming.  I could easily translate all those
If-Then-Else and Return into (almost) any program language of the day
(but not, perhaps, APL).

And yes, it is still one of the great RFC; is that because it describes 
something that became rather popular or is it because it describes
something so well that popularity was inevitable?

Tom Petch

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Mar 2011 04:02:38 -0700

...
...
> something so well that popularity was inevitable?

I think the latter; it understood that a protocol is:

A- format on the wire of messages that arrive and depart
B- state within participating parties
C- events from/to the upper layer
D- rules that relate the three above

The "API" as we commonly call it is "C" above. I've seen quite a few 
IETF "protocols" that focus only on "A".

IMO it takes all four to be complete.

Joe

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 30 Mar 2011 07:14:29 -0400

Specifying the service interface a protocol provides (and what it 
requires from other protocols) is often very useful.  And is something 
we have done in many cases.
Writing a set of subroutine definitions in some specific language is, in 
my experience, usually far less useful.

As a related note, a program is not a protocol spec, and a protocol spec 
is not an implementation design document.  They serve different purposes 
and should not be conflated.

Yours,
Joel

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 30 Mar 2011 13:20:27 +0200

On 3/30/2011 1:14 PM, Joel M. Halpern wrote:
> Specifying the service interface a protocol provides (and what it requires from
> other protocols) is often very useful. And is something we have done in many cases.

In some cases, there is a surprising benefit:

      It makes a clear distinction between what is "internal" to the protocol, 
versus what is payload that is delivered to the consumer (next layer up or 
receiving application.)

Sometimes, the way a protocol is specified does not make this distinction 
completely clear.

d/
-- 

   Dave Crocker

---

From: Martin Sustrik <[email-masked]>
Date: Wed, 30 Mar 2011 13:33:07 +0200

On 03/30/2011 01:20 PM, Dave CROCKER wrote:

> versus what is payload that is delivered to the consumer (next layer up
> or receiving application.)
>
> Sometimes, the way a protocol is specified does not make this
> distinction completely clear.

Good point. For example, BSD socket API makes it clear that TCP packet 
boundaries are not visible to the application. If it was not so, some 
TCP implementations could expose those boundaries to the applications. 
Those, in turn, could use it to drive the business logic. Such 
applications would then break if a middlebox re-segmented the data.

Martin

---

From: Bob Braden <[email-masked]>
Date: Wed, 30 Mar 2011 11:09:42 -0700

+1

Bpb Braden

On 3/30/2011 1:11 AM, Joe Touch wrote:
> service needs to be explained - and that explaining it does NOT need 
> to be done in a specific language.
>
> Joe

---

From: Tom Yu <[email-masked]>
Date: Wed, 30 Mar 2011 17:40:39 -0400

Personal observation: "API" is a subclass of "interface".  "Network
protocol" is a subclass of "interface".  Interfaces (in the general
case) are valuable things to standardize.  An abstract interface
("abstract API"?) that gives guidance to implementors above and beyond
the bare specification of the network protocol is useful for achieving
conceptual alignment.

Do people agree?