Subject: [tap] RFC Status?

From: Andrew de Andrade <[email-masked]>
Date: Mon, 11 Aug 2014 11:42:19 -0700

I'm working on some testing libraries for JavaScript based on TAP.
Specifically, I'm trying to explore making it possible for library authors
to not only check the results of their TAP tests, but also map-reduce the
TAP results of the dependents of their library to see how changes to their
library impact the correctness of their dependents.

e.g. I am the author of library A. I publish version 0.1.0. Another author
includes my library as a dependency in library B and in library C and uses
TAP to test those libraries. Given this scenario, it would be nice if I
could, when modifying my library, run not only my tests, but also run the
tests of all my dependents built with my previous version and my current
version and compare how my changes impacted the "correctness" of their
programs.

Beyond the benefits to a community, helping authors know when they can
likely safely upgrade their deps, this type of runner would also help in an
corporate environment with lots of code re-use between teams.

While thinking about this, I decided to go out and figure out if there is
such a thing as nested TAP in the protocol specification. While searching
for this, I came across this group, which is a great, but it appears there
has been no activity here in a while and it looks like all the wiki pages
with prior information about the state of the TAP protocol becoming an IETF
RFC have disappeared.

What's the current state of TAP? Is there still interest in taking this to
RFC status? I found some previous discussions on the list about nested TAP.
Did those make it into the specification in any way? If so, where can I
find examples of correct nested tap results?

Furthermore, is there a standardized set of test fixtures for the current
version that any implementation can be tested against for correctness and
performance? Having worked a bit with JSON schema, I found that one of the
most useful tools was a standardized set of tests to check if a particular
implementation conforms to the protocol standard. (see:
https://github.com/json-schema/JSON-Schema-Test-Suite )

---

From: Gaurav Vaidya <[email-masked]>
Date: Mon, 11 Aug 2014 13:07:50 -0600

Hi Andrew!

On 11 Aug, 2014, at 12:42 pm, Andrew de Andrade <[email-masked]> wrote:
> While thinking about this, I decided to go out and figure out if there is such a thing as nested TAP in the protocol specification. While searching for this, I came across this group, which is a great, but it appears there has been no activity here in a while and it looks like all the wiki pages with prior information about the state of the TAP protocol becoming an IETF RFC have disappeared. 
> 
> What's the current state of TAP? Is there still interest in taking this to RFC status? I found some previous discussions on the list about nested TAP. Did those make it into the specification in any way? If so, where can I find examples of correct nested tap results?
The most recent version of the spec I could find was at https://github.com/gaurav/test-anything-protocol/blob/master/output/draft-vaidya-test-anything-protocol-00.txt, but this doesn’t talk about nesting at all. I’ve been out of the loop for a long, long time for graduate school reasons, but I think I remember someone (Ovid?) writing up a spec for nested TAP elsewhere?

cheers,
Gaurav

---

From: "Bruno P. Kinoshita" <[email-masked]>
Date: Mon, 11 Aug 2014 13:22:24 -0700

Hi all, 

Some time ago I started learning about TAP for a Jenkins plugin (TAP Plug-in), for use in another plug-in (TestLink Plug-in) and consequently had to write a Java API for both plug-ins (tap4j.org). There are some users of the TAP plug-in that utilize it for node.js testing, others use the yslow plug-in, and others use for Perl or other kind of automation in their builds.

> What's the current state of TAP? Is there still interest in taking this to RFC status? I found some previous discussions on the list about nested TAP. Did those make it into the specification in an…

It'd been a while since the last time I visited testanything.org, but the site looks great now! I liked the new layout, kudos to whoever made it.

I couldn't find some old content in the new site though. In tap4j we implement subtests (as in Test::More), like:

  1..3
  ok 1 - First test
      # Subtest: An example subtest
      1..2
      ok 1 - This is a subtest
      ok 2 - So is this
  ok 2 - An example subtest
  ok 3 - Third test

The example below is from this page: http://search.cpan.org/~exodist/Test-Simple-1.001003/lib/Test/More.pm

But IIUC, it is not part of TAP 13 specification. There were some proposals for it in the old site, looking at the internet archive the best I could find was https://web.archive.org/web/20120730055245/http://testanything.org/wiki/index.php/TAP_Proposals

Take a look at "Test Groups" and "Test Blocks". 

I'm participating in a test management OSS application development (PHP), and we are going to use TAP for automation too, so I'm looking forward to new releases/discussions on TAP :)

Hope that helps
Bruno

>________________________________
> From: Gaurav Vaidya <[email-masked]>
>To: Andrew de Andrade <[email-masked]> 
>Cc: [email-masked] 
>Sent: Monday, August 11, 2014 4:07 PM
>Subject: Re: [tap] RFC Status?
> 
>
>Hi Andrew!
>
>_______________________________________________
>tap mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/tap
>
>

---

From: Andrew de Andrade <[email-masked]>
Date: Mon, 11 Aug 2014 14:30:56 -0700

Yes, this all helps a bunch. I've been using substack's tape (
https://github.com/substack/tape) and isaacs' tap (
https://github.com/isaacs/node-tap) modules thus far.

tape currently supports v13, which as you pointed out Bruno, does not
formally specify how nesting should work, and it does support defining
tests in a nested way, but does not report them as nested. Instead, it
simply updates the plan count and continues with everything flat/unindented
and reports the plan count at the end. For example this code...

```
var test = require('tape');

test('foobar', function(t) {
  t.plan(3);
  //var test = tape.createHarness();

  t.ok(true, 'foo');
  t.ok(true, 'bar');

  t.test('fibble', function(tt) {
    tt.plan(2);
    tt.ok(true, 'wobble');
    tt.ok(true, 'wizzle');
  });
});
```
...produces...

```
TAP version 13
# foobar
ok 1 foo
ok 2 bar
# fibble
ok 3 wobble
ok 4 wizzle

1..4
# tests 4
# pass  4

# ok
```

What I did just find is an implementation from Ovid from earlier this year
in March:

http://blogs.perl.org/users/ovid/2014/03/perlqa-hackathon-in-lyon---day-1.html
https://github.com/Ovid/tap-stream

This stream approach is really nice, but the only thing that leaves me
scratching my head is that is implies that asynchrony is permitted
(parallelized tests, aggregation, out-of-order reporting), but I'm not
exactly sure what the implications for this are for the aggregate tap
producer.

Should the aggregate reporter report subtests as they are received or
should it keep the subtest results that finish out of order in a buffer and
always report the tests in a deterministic way? I did see this in the
readme, https://github.com/Ovid/tap-stream#caveats , implying AFAICT that
it is left up to the TAP aggregator consuming the stream to either report
things in order with or without numbers (if desired) or out of order, but
un-numbered.

Given my intended use case, buffering results so that results can always be
reported numbered in order would be desirable, since I want to be able to
diff TAP results across test runs under different conditions (e.g.
different dependency versions). Has anyone done any work with diffing TAP
results?

Bruno, is tap4j purely synchronous or at least in order or can subtests be
asynchronous and reported out of order?

Seeing that Ovid is still actively working on this stuff (comments on tap
on perlqa3 as recent as April this year), is it save to assume that there
are ideas here that will make it into version 14? The test blocks you
pointed to does reference version 14 in the example -->
https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks

---

From: "Bruno P. Kinoshita" <[email-masked]>
Date: Mon, 11 Aug 2014 15:41:16 -0700

> Yes, this all helps a bunch. I've been using substack's tape (https://github.com/substack/tape) and isaacs' tap (https://github.com/isaacs/node-tap) modules thus far. 

I've seen some users reporting issues in the tap-plugin for Jenkins where they were using node-tap. Hadn't heard about tape before. 

> What I did just find is an implementation from Ovid from earlier this year in March:
>

I liked Ovid's implementation. One thing that i have added to my TODO list is to look at his examples and use some of them as regression tests in tap4j. I think it goes along with your previous suggestion of a set of tests for TAP (as in JSON?).

> Bruno, is tap4j purely synchronous or at least in order or can subtests be asynchronous and reported out of order?

Purely synchronous at the moment. There is a factory, that creates a TapConsumer, which receives a TAP stream or java.util.File, and then it calls the parser.

The parser itself uses a state machine (based on SnakeYAML parser). So my guess is that it is not too hard to add a new asynchronous consumer, that passes each line received to the parser, in order. This assync consumer would deal with java threading, and raise any exception received from the parser (such as receiving tokens in the order Test-Result -> Tap-Plan -> Test-Result).

> Seeing that Ovid is still actively working on this stuff (comments on tap on perlqa3 as recent as April this year), is it save to assume that there are ideas here that will make it into version 14? The test blocks you pointed to does reference version 14 in the example -->  https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks

I'm not sure. Do you know if Ovid is in this list? Maybe we should check with him if he has interest in joining the thread and help crafting TAP 14 too? 

Bruno

> From: Andrew de Andrade <[email-masked]>
>To: Bruno P. Kinoshita <[email-masked]> 
>Cc: Gaurav Vaidya <[email-masked]>; "[email-masked]" <[email-masked]> 
>Sent: Monday, August 11, 2014 6:30 PM
>Yes, this all helps a bunch. I've been using substack's tape (https://github.com/substack/tape) and isaacs' tap (https://github.com/isaacs/node-tap) modules thus far. 
>
>
>tape currently supports v13, which as you pointed out Bruno, does not formally specify how nesting should work, and it does support defining tests in a nested way, but does not report them as nested. Instead, it simply updates the plan count and continues with everything flat/unindented and reports the plan count at the end. For example this code...
>
>
>```
>What I did just find is an implementation from Ovid from earlier this year in March:
>
>
>This stream approach is really nice, but the only thing that leaves me scratching my head is that is implies that asynchrony is permitted (parallelized tests, aggregation, out-of-order reporting), but I'm not exactly sure what the implications for this are for the aggregate tap producer. 
>
>
>Should the aggregate reporter report subtests as they are received or should it keep the subtest results that finish out of order in a buffer and always report the tests in a deterministic way? I did see this in the readme, https://github.com/Ovid/tap-stream#caveats , implying AFAICT that it is left up to the TAP aggregator consuming the stream to either report things in order with or without numbers (if desired) or out of order, but un-numbered. 
>
>
>Given my intended use case, buffering results so that results can always be reported numbered in order would be desirable, since I want to be able to diff TAP results across test runs under different conditions (e.g. different dependency versions). Has anyone done any work with diffing TAP results?
>
>Bruno, is tap4j purely synchronous or at least in order or can subtests be asynchronous and reported out of order?
>
>
>Seeing that Ovid is still actively working on this stuff (comments on tap on perlqa3 as recent as April this year), is it save to assume that there are ideas here that will make it into version 14? The test blocks you pointed to does reference version 14 in the example -->  https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks
>
>
>
>

---

From: Andrew de Andrade <[email-masked]>
Date: Mon, 11 Aug 2014 16:59:35 -0700

On Mon, Aug 11, 2014 at 3:41 PM, Bruno P. Kinoshita <
[email-masked]> wrote:

> I've seen some users reporting issues in the tap-plugin for Jenkins where
> they were using node-tap. Hadn't heard about tape before.
>

If you have links to those issues, that would be most appreciated, since
I'll report them to isaacs. These days I'm using only substack/tape since
I'm doing mostly browser testing. Right now, I'm focused on aggregating the
test results. Once I try out a CI solution like Strider, Jenkins or Travis,
I planned on using tap. If you send me those issues, I might be able to
patch them once I get around to using tap with a CI server.

> I liked Ovid's implementation. One thing that i have added to my TODO list
> is to look at his examples and use some of them as regression tests in
> tap4j. I think it goes along with your previous suggestion of a set of
> tests for TAP (as in JSON?).
>

Yeah, standardized test suites are immensely useful. I'm not entirely
certain how it would work in this case of testing TAP compliancy though. I
guess JSON is one option (but wouldn't we then be defining yet another test
protocol?). JSON is the logical approach since it would mean that the TAP
examples server as tests for the JSON and vice versa, and all languages
have mature libraries for consuming JSON and working with JSON.

Even if there isn't a clear way to automate checking if an implementation
is correct (since the tests for each would have to be written in the
language of each implementation), just having a corpus of known valid
examples, would be immensely useful.

> > Seeing that Ovid is still actively working on this stuff (comments on
> tap on perlqa3 as recent as April this year), is it save to assume that
> there are ideas here that will make it into version 14? The test blocks you
> I'm not sure. Do you know if Ovid is in this list? Maybe we should check
> with him if he has interest in joining the thread and help crafting TAP 14
> too?
>
>

I emailed him at the email listed on his github profile. I reckon if he is
interested (or still subscribed via a different email address he's used in
the past), he'll participate directly.

---

From: Ovid <[email-masked]>
Date: Tue, 12 Aug 2014 00:26:18 -0700

Hi all,

Yes, I'm still on this list. I was pleasantly surprised at the burst of activity I saw today.

I'd love to see a TAP v14. More importantly, I'd love to see a standard created out of this so that we could actually say "look at the RFC".
 
Best,
Ovid
--
IT consulting, training, international recruiting
       http://www.allaroundtheworld.fr/.
Buy my book! - http://bit.ly/beginning_perl
Live and work overseas - http://www.overseas-exile.com/

---

From: Ovid <[email-masked]>
Date: Tue, 12 Aug 2014 00:49:09 -0700

Should the aggregate reporter report subtests as they are received or should it keep the subtest results that finish out of order in a buffer and always report the tests in a deterministic way? I did see this in the readme, https://github.com/Ovid/tap-stream#caveats , implying AFAICT that it is left up to the TAP aggregator consuming the stream to either report things in order with or without numbers (if desired) or out of order, but un-numbered. 
The following assumes that I correctly understood your question :)

Presumably an aggregate reporter either runs tests in order (in which case tests come back in order), or or runs them in parallel (in which case order is not guaranteed). Let's say that your test harness runs test programs A and B. It cannot control their output, so A and B must guarantee that they return valid TAP. What that means in the case of subtests is the following.

Let's say that A returns:
1..2
>ok 1 - We have lift off!
>not ok 2 - Houston, we have a problem
And B returns this:
ok 1 - Trap is set
>ok 2 - Trap is sprung
>ok 3 - Compromising photos of boss emailed
>1..3
With that, if we run in parallel, either of those TAP chunks can be returned first, but until the consumer receives the entire output from A or B (in whatever order), it can produce no output. It could, in theory, block on one, but this behavior isn't defined because the specification is about the TAP producers, not the consumers. Maybe that that should be rectified? (Or maybe it is also about the consumers and it's been too long since I worked on it and just plum forgot).

Assuming all goes well, the consumer can produce:
    1..2
>    ok 1 - We have lift off!
>    not ok 2 - Houston, we have a problem
>not ok 1 - A
>    ok 1 - Trap is set
>    ok 2 - Trap is sprung
>    ok 3 - Compromising photos of boss emailed
>    1..3
>ok 2 - B
>1..2
Or this:
    ok 1 - Trap is set
>    ok 2 - Trap is sprung
>    ok 3 - Compromising photos of boss emailed
>    1..3
>ok 1 - B
>    1..2
>    ok 1 - We have lift off!
>    not ok 2 - Houston, we have a problem
>not ok 2 - A
>1..2
Or (but not recommended):
not ok 1 - A
>ok 2 - B
The latter is not recommended because you're throwing away information, but it should parse the same way as the previous ones with an older TAP parser because ok/not ok binds to the beginning of the line (and thus indented TAP is ignored).

In fact, you could put the 1..2 plans before the tests if you wanted to (also called a "leading plan").

The reason the nested TAP behavior isn't well-specified is because there were many arguments over it at conferences and hackathons, and then one day (I was in Norway, I think), I got tired of the arguments and just sat down and implemented a mostly backwards-compatible solution. Implementation trumps opinion, so I won.

Another thing to note about TAP is that we still don't have structured diagnostics. Again, this was largely because no one could agree on what they should look like. Some wanted producers to be able to output them in whatever format they wanted (JSON, YAML, XML, and so on). I argued that was madness and having a single standard would make parsing much easier. Putting that aside, there were also arguments about how and what information should be captured in said diagnostics. Filename and line number are clear, but what about tests for which that information doesn't make sense? Or what about a timestamp? Local time? GMT (useless)? UTC?

So there's lots of work to be done and lots of behavior to be nailed down. Fortunately, TAP's been going strong since the 1980s, but it's only been the past decade or so since it's been named and formally developed.

Ask any questions you like.

Best,
Ovid

---

From: "Bruno P. Kinoshita" <[email-masked]>
Date: Tue, 12 Aug 2014 07:21:56 -0700

Hi Ovid!

> Another thing to note about TAP is that we still don't have structured diagnostics.

Indeed. Most projects in Jenkins, even today, rely on JUnit for reporting test results. They have a somewhat modified XML schema for handling attachments in JUnit (like in the junit-attachments-plugin). 

In tap4j, tap-plugin and testlink-plugin, we used the YAML blocks in our TAP stream.

ok 1
  ---
  name: test IO
  description: yadda yadda
  ...
not ok 2
1..2

> I argued that was madness and having a single standard would make parsing much easier.

+1 for some bugs in tap4j, I was very glad I could point users to the TAP 13 details page, and tell them that we would have to stick with the docs in testanything.org. That still helps a lot.

> Putting that aside, there were also arguments about how and what information should be captured in said diagnostics. Filename and line  number are clear, but what about tests for which that information doesn't make sense? Or what about a timestamp? Local time? GMT (useless)? UTC?

Agreed too, we adopted the main entries from [1] in tap4j, plus the extensions entry (which is free to have whatever users want).

While this works well for tap4j and the jenkins plug-in, a standardization would definitely help, specially to integrate with other tools.

> Ask any questions you like.

What do you think that would be the next steps to push forward a TAP 14 draft, and make some progress on the RFC? Maybe something like monthly meetings in IRC or some other IM, and some discussion panels in hackathons/meetups? 

Thanks!
Bruno

[1] https://web.archive.org/web/20111017073902/http://testanything.org/wiki/index.php/TAP_diagnostic_syntax

> From: Ovid <[email-masked]>
>To: Andrew de Andrade <[email-masked]>; Bruno P. Kinoshita <[email-masked]> 
>Cc: "[email-masked]" <[email-masked]> 
>Sent: Tuesday, August 12, 2014 4:49 AM

---

From: Jonathan Kingston <[email-masked]>
Date: Tue, 12 Aug 2014 20:36:15 +0100

@Bruno I worked on the layout and the rest is on Github where conversations
can be continued for proposals as I didn't really like the site being a
wiki style. To gain any form of authority TAP needs to portray itself much
like any other body where conversations can be held elsewhere.

My plan was to make a repo just for test proposals if you like in the style
of w3c proposals under the TestAnything org: https://github.com/TestAnything

There is some bits that have been raised on Github which needs some
discussions as realistically people are developing alternatives to TAP
because of its shortcomings. However the JSON formats that have been
developed so far seem to have issues themselves despite being an attractive
concept.

I am very much interested in it being an RFC however like the site
maintenance I am lacking a little in time.

On 11 August 2014 21:22, Bruno P. Kinoshita <[email-masked]>
wrote:

> Hi all,
>
> Some time ago I started learning about TAP for a Jenkins plugin (TAP
> Plug-in), for use in another plug-in (TestLink Plug-in) and consequently
> had to write a Java API for both plug-ins (tap4j.org). There are some
> users of the TAP plug-in that utilize it for node.js testing, others use
> the yslow plug-in, and others use for Perl or other kind of automation in
> their builds.
>
> > What's the current state of TAP? Is there still interest in taking this
> to RFC status? I found some previous discussions on the list about nested
> TAP. Did those make it into the specification in any way? If so, where can
> I find examples of correct nested tap results?
>
> It'd been a while since the last time I visited testanything.org, but the
> site looks great now! I liked the new layout, kudos to whoever made it.
>
> I couldn't find some old content in the new site though. In tap4j we
> implement subtests (as in Test::More), like:
>
>   1..3
>   ok 1 - First test
> The example below is from this page:
> http://search.cpan.org/~exodist/Test-Simple-1.001003/lib/Test/More.pm
>
> But IIUC, it is not part of TAP 13 specification. There were some
> proposals for it in the old site, looking at the internet archive the best
> I could find was
> https://web.archive.org/web/20120730055245/http://testanything.org/wiki/index.php/TAP_Proposals
>
> I'm participating in a test management OSS application development (PHP),
> and we are going to use TAP for automation too, so I'm looking forward to
> new releases/discussions on TAP :)
>
> Hope that helps
> Bruno
>
>   ------------------------------
>  *From:* Gaurav Vaidya <[email-masked]>
> *To:* Andrew de Andrade <[email-masked]>
> *Cc:* [email-masked]
> *Sent:* Monday, August 11, 2014 4:07 PM
> *Subject:* Re: [tap] RFC Status?
>
> Hi Andrew!
>
> > While thinking about this, I decided to go out and figure out if there
> is such a thing as nested TAP in the protocol specification. While
> searching for this, I came across this group, which is a great, but it
> appears there has been no activity here in a while and it looks like all
> the wiki pages with prior information about the state of the TAP protocol
> becoming an IETF RFC have disappeared.
> >
> > What's the current state of TAP? Is there still interest in taking this
> to RFC status? I found some previous discussions on the list about nested
> TAP. Did those make it into the specification in any way? If so, where can
> I find examples of correct nested tap results?
> The most recent version of the spec I could find was at https://github.com/gaurav/test-anything-protocol/blob/master/output/draft-vaidya-test-anything-protocol-00.txt,
> but this doesn’t talk about nesting at all. I’ve been out of the loop for
> a long, long time for graduate school reasons, but I think I remember
> someone (Ovid?) writing up a spec for nested TAP elsewhere?
>
> cheers,
> Gaurav
>

---

From: Andrew de Andrade <[email-masked]>
Date: Tue, 12 Aug 2014 13:13:47 -0700

On Tue, Aug 12, 2014 at 12:36 PM, Jonathan Kingston <[email-masked]>
wrote:

> @Bruno I worked on the layout and the rest is on Github where
> conversations can be continued for proposals as I didn't really like the
> site being a wiki style. To gain any form of authority TAP needs to portray
> itself much like any other body where conversations can be held elsewhere.
>
> My plan was to make a repo just for test proposals if you like in the
> style of w3c proposals under the TestAnything org:
> https://github.com/TestAnything
>

Agree. Moving everything under the TestAnything github org sounds like a
solid approach.

> developed so far seem to have issues themselves despite being an attractive

Do you have any links to these alternatives and to discussions regarding
what shortcomings in TAP that people are becoming frustrated with?

> maintenance I am lacking a little in time.

Since the site is a github.io page, maintenance can be handled by any of us
via a pull request. Are there any outstanding tasks that need to be
addressed on the site?

---

From: Jonathan Kingston <[email-masked]>
Date: Tue, 12 Aug 2014 21:47:47 +0100

Hi Andrew,

The most conversation was there about the JSON / TAP/Y work:
https://github.com/TestAnything/testanything.github.io/pull/6

I have seen a lot of conversation on other lists but I will have to dig out
where they were, I think there was talk on http://karma-runner.github.io/
The issue as I see it is the ability for robust meta data with and agreed
set of parameters and meanings. TAP/Y(
https://github.com/rubyworks/tapout/wiki/TAP-Y-J-Specification) has gone
some way towards addressing these however not far enough in clearing up the
format.

There is smaller amounts of things that need merging in but most are
conversations, the issue realistically is that I'm not the authority so it
need people to vote and comment on them. So on the issue above I can add my
comments and merge it in but realistically it is so much of a change that I
don't think it is fair for me to just merge it in.

I am happy to be a gate keeper of the changes of the site as realistically
its not going to change at a rapid pace, however I am looking to the
community to contribute to the issues and out of that perhaps I can get
more moderators on side.

I have created a new repo(https://github.com/TestAnything/Specification)
for TAP which can be used to track changes better, conversations and also
building TAP 14 separate to the site itself which I see as a published
authority.

Thanks

On 12 August 2014 21:13, Andrew de Andrade <[email-masked]> wrote:

> On Tue, Aug 12, 2014 at 12:36 PM, Jonathan Kingston <[email-masked]
> > wrote:
>
> Since the site is a github.io page, maintenance can be handled by any of
> us via a pull request. Are there any outstanding tasks that need to be
> addressed on the site?
>
>

---

From: Salve J Nilsen <[email-masked]>
Date: Tue, 12 Aug 2014 23:07:58 +0200 (CEST)

> Agree. Moving everything under the TestAnything github org sounds like a

FWIW, I agree! :D

- Salve

-- 
#!/usr/bin/env perl
sub AUTOLOAD{$AUTOLOAD=~/.*::(\d+)/;seek(DATA,$1,0);print# Salve Joshua Nilsen
getc DATA}$"="'};&{'";@_=unpack("C*",unpack("u*",':50,$'.#    <[email-masked]>
'3!=0"59,6!`%%P\0!1)46%!F.Q`%01,`'."\n"));eval "&{'@_'}";  __END__ is near! :)

---

From: "Bruno P. Kinoshita" <[email-masked]>
Date: Tue, 12 Aug 2014 21:31:55 -0700

Hello,

> The immediate need is for someone to take ownership of the IETF process and publicly say "I'll see this through." There was a lot of great work done in the past, but we're obviously not there yet.

Great idea, let's see if someone will step forward to volunteer (maybe you? :-). I'm relocating to another continent in less than 1 year, but if no one else volunteers I can work on this till June next year. 

> With that, we can start scratching items off out TODO list one-by-one. I would humbly suggest that we not go with monthly meetings as they're so hard to organize. Instead, keep the list open, have the TODO list on a github wiki and let's see what we can make happen.

Sounds good to me. Let's use https://github.com/TestAnything/Specification ? 

Even though I do like keeping decisions and discussions in the mailing list (like the Apache way), I feel sometimes it helps to have something like an IRC channel, for real time chat, validate some ideas before sending to the mailing list or brainstorm with somebody else. What do you think? I started collecting TAP-related links in a subreddit too, but is has very low activity and got only 2 readers http://www.reddit.com/r/testanythingprotocol

Thanks Ovid!
Bruno

>To: Bruno P. Kinoshita <[email-masked]>; Andrew de Andrade <[email-masked]> 
>Sent: Wednesday, August 13, 2014 1:10 AM
>The immediate need is for someone to take ownership of the IETF process and publicly say "I'll see this through." There was a lot of great work done in the past, but we're obviously not there yet.
>
>
>Next, the goal needs to be clearly defined. I would also suggest we define why we want the goal accomplished. Once done:
>	1. What we've successfully done.
>
>	2. What we still need to do.
>
>	3. What might need to change (danger, Will Robinson!)
>
>Those three items can give us a TODO list. Item #1 might seem superfluous, but when tackling a large project, I like to remind people of all of the groundwork already laid to keep things in perspective. Plus, a sense of historical context might help those who are unfamiliar with the history.
>
>
>With that, we can start scratching items off out TODO list one-by-one. I would humbly suggest that we not go with monthly meetings as they're so hard to organize. Instead, keep the list open, have the TODO list on a github wiki and let's see what we can make happen.
>
>
>Best,
>Ovid
>--

---

From: Ovid <[email-masked]>
Date: Tue, 12 Aug 2014 22:17:28 -0700

I certainly can't volunteer to run it. My time is crazy, I'm working a couple of different contracts right now, often have to be at conferences and sometimes get called away on short notice to travel to other countries. On top of that, I'm a father and husband. My time is somewhat limited to brief bursts of energy in my "spare" time :)
 
And you now have 3 readers for your subreddit, but I'm subscribed to enough subreddit's that I may very well miss changes there!

Best,
Ovid

---

From: Andrew de Andrade <[email-masked]>
Date: Tue, 12 Aug 2014 22:50:46 -0700

For those that have been around the longest, can someone share some links
that describe what exactly is going to be required in terms of know-how and
time to take ownership of a process like this. Of course, I've found some
general guides on the IETF process (links below), but if there is anything
specific to what's been done with TAP in general, that would be useful. In
the past, who had taken ownership of the process, so we can at least go
back in the archives and see how the previous person handled ownership of
this process with respect to TAP.

I'm about the change jobs in ~1 month to work entirely on engineering tool
for front-end teams at Uber, so I can't promise anything yet, but if the
commitment involved is reasonable and it's ends up aligned with my
responsibilities at my new job, I just might be able to take ownership
here. No guarantees just yet.

This is what I found and will go through to better understand the IETF
process.

https://www.ietf.org/about/standards-process.html
http://www.ietf.org/about/process-docs.html
https://www.ietf.org/rfc/rfc2026.txt
http://en.wikipedia.org/wiki/Request_for_Comments
http://www.rfc-editor.org/rfc/rfc6410.txt
http://tools.ietf.org/html/rfc2028

On Tue, Aug 12, 2014 at 10:17 PM, Ovid <[email-masked]> wrote:

> I certainly can't volunteer to run it. My time is crazy, I'm working a
> couple of different contracts right now, often have to be at conferences
> and sometimes get called away on short notice to travel to other countries.
> On top of that, I'm a father and husband. My time is somewhat limited to
> brief bursts of energy in my "spare" time :)
>
> And you now have 3 readers for your subreddit, but I'm subscribed to
> enough subreddit's that I may very well miss changes there!
>
> Best,
> Ovid
> --
> > The immediate need is for someone to take ownership of the IETF process
> and publicly say "I'll see this through." There was a lot of great work
> done in the past, but we're obviously not there yet.
>
> Great idea, let's see if someone will step forward to volunteer (maybe
> you? :-). I'm relocating to another continent in less than 1 year, but if
> no one else volunteers I can work on this till June next year.
>
> > With that, we can start scratching items off out TODO list one-by-one. I
> would humbly suggest that we not go with monthly meetings as they're so
> hard to organize. Instead, keep the list open, have the TODO list on a
> github wiki and let's see what we can make happen.
>
> Even though I do like keeping decisions and discussions in the mailing
> list (like the Apache way), I feel sometimes it helps to have something
> like an IRC channel, for real time chat, validate some ideas before sending
> to the mailing list or brainstorm with somebody else. What do you think? I
> started collecting TAP-related links in a subreddit too, but is has very
> low activity and got only 2 readers
> http://www.reddit.com/r/testanythingprotocol
>
> Thanks Ovid!
> Bruno
>
>  *From:* Ovid <[email-masked]>
> *To:* Bruno P. Kinoshita <[email-masked]>; Andrew de Andrade <
> [email-masked]>
> *Cc:* "[email-masked]" <[email-masked]>
> *Sent:* Wednesday, August 13, 2014 1:10 AM
> What do you think that would be the next steps to push forward a TAP 14
> draft, and make some progress on the RFC? Maybe something like monthly
> meetings in IRC or some other IM, and some discussion panels in
> hackathons/meetups?
>
> The immediate need is for someone to take ownership of the IETF process
> and publicly say "I'll see this through." There was a lot of great work
> done in the past, but we're obviously not there yet.
>
> Next, the goal needs to be clearly defined. I would also suggest we define
> why we want the goal accomplished. Once done:
>
> Those three items can give us a TODO list. Item #1 might seem superfluous,
> but when tackling a large project, I like to remind people of all of the
> groundwork already laid to keep things in perspective. Plus, a sense of
> historical context might help those who are unfamiliar with the history.
>
> With that, we can start scratching items off out TODO list one-by-one. I
> would humbly suggest that we not go with monthly meetings as they're so
> hard to organize. Instead, keep the list open, have the TODO list on a
> github wiki and let's see what we can make happen.
>
> Best,
> Ovid
> --

-- 
Q: Why is this email four sentences or less?
A: http://four.sentenc.es

---

From: Ovid <[email-masked]>
Date: Tue, 12 Aug 2014 21:10:00 -0700

Best,
Ovid

---

From: Andrew de Andrade <[email-masked]>
Date: Wed, 13 Aug 2014 11:04:39 -0700

> https://github.com/TestAnything/testanything.github.io/pull/6
I will take a look at the others. I'm also going to go through the web
archived wiki pages and move them to the TestAnything github org.

I was about to say that "right now there is one repo. I'm wondering if it
is best to have two repos, once with the spec and specification discussions
and one strictly for the website." and then I went to check out
TestAnything and saw that you already took care of that. I'll try to move
the original wiki pages over to TestAnything/Specification and start one
issue for each of the proposed ideas so that the discussion can take place
there.

> where they were, I think there was talk on http://karma-runner.github.io/

I'll try to some searching through the issue archives and see what I can
find. Let me know if there are any other places worth looking at.

> some way towards addressing these however not far enough in clearing up the

Will investigate.

> don't think it is fair for me to just merge it in.

Usually it's not even about being an authority, but recognizing where there
are strong differences of opinion and knowing how to polling a few of those
people who are authorities because they actually coded implementations
implementations and therefore know the pitfalls and the complaints and
desires of those using their code.

> community to contribute to the issues and out of that perhaps I can get

Sounds good. Any pressing changes that people really want to see can just
be submitted as pull requests.

> building TAP 14 separate to the site itself which I see as a published

Awesome. As I said, I'll move the old wiki content to the the wiki for this
repo.

---

From: Andrew de Andrade <[email-masked]>
Date: Wed, 13 Aug 2014 11:09:43 -0700

> building TAP 14 separate to the site itself which I see as a published
Instead of creating a brand new repo with no history, just clone --mirror
guarav's repo and --mirror push it to TestAnything/Specification. His
version already has a started RFC and has commits he's already made that
should be preserved. Alternatively, maybe Guarav can just transfer
ownership to the TestAnything organization.
https://github.com/gaurav/test-anything-protocol/

Guarav, are you ok with transferring ownership to the TestAnything github
org? This would have the spec in one central location where everyone can
fork from.

---

From: Gaurav Vaidya <[email-masked]>
Date: Thu, 14 Aug 2014 19:07:04 -0600

Hullo!

On 13 Aug, 2014, at 12:09 pm, Andrew de Andrade <[email-masked]> wrote:

> On Tue, Aug 12, 2014 at 1:47 PM, Jonathan Kingston <[email-masked]> wrote:
>> I have created a new repo(https://github.com/TestAnything/Specification) for TAP which can be used to track changes better, conversations and also building TAP 14 separate to the site itself which I see as a published authority.
> 
> Instead of creating a brand new repo with no history, just clone --mirror guarav's repo and --mirror push it to TestAnything/Specification. His version already has a started RFC and has commits he's already made that should be preserved. Alternatively, maybe Guarav can just transfer ownership to the TestAnything organization.
> Guarav, are you ok with transferring ownership to the TestAnything github org? This would have the spec in one central location where everyone can fork from.

Absolutely! Apparently, I need to be an admin on TestAnything to do the transfer, so could someone add me there (username: gaurav)? I’ll give up the admin bit as soon as I get https://github.com/gaurav/test-anything-protocol/ transferred.

cheers,
Gaurav