Subject: Hotel situation

From: Melinda Shore <[email-masked]>
Date: Wed, 16 Dec 2015 07:36:03 -0900

Why are we continuing to have hotel issues meeting after meeting
after meeting after meeting?

Melinda

---

From: Ted Lemon <[email-masked]>
Date: Wed, 16 Dec 2015 16:52:24 +0000

> after meeting after meeting?

Because we can't force hotels to give us large allocations.   This was discussed at length at the last plenary.   Getting venues that do everything we want is hard.

That said, I think you'd make a great candidate for the IAOC next time if you feel like trying to disprove this point!   :)

---

From: Melinda Shore <[email-masked]>
Date: Wed, 16 Dec 2015 07:59:16 -0900

On 12/16/15 7:52 AM, Ted Lemon wrote:
> Because we can't force hotels to give us large allocations.  

We already have a list of hotels which will.

Either meetings are important or they're not.  If we're
going to continue to treat meeting participation as necessary,
we need to make it easier logistically.  Instead we keep throwing
up barriers to participation.  The closest overflow hotel
is nearly a mile from the Hilton - how's that going to work
out for people with limited mobility?  I'm very happy having a
walk each morning and evening but I'll tell you that it can be
a huge PITA for breakfast meetings for people not staying
at the headquarter hotel.  Getting work done should be one
of the primary considerations, here.

Melinda

---

From: Tim Wicinski <[email-masked]>
Date: Wed, 16 Dec 2015 12:03:03 -0500

I have to agree with Melinda here.  It's nice being in the hotel to have 
those many discussions, but as WG Chair I always feel i do my groups a 
disservice not being easily available.

tim

---

From: Ted Lemon <[email-masked]>
Date: Wed, 16 Dec 2015 17:05:08 +0000

> We already have a list of hotels which will.
We do?  This is news to me.   If you mean "hotels that have in the past," I don't think that's any guarantee.

> The closest overflow hotel
> out for people with limited mobility?

This is an excellent point, and reinforces my belief that you'd be a good IAOC candidate.   I think there's a case to be made that if someone needs special accommodations, there should be a way for them to demonstrate that this is the case and get early access to the conference hotel room block.

> I'm very happy having a
> at the headquarter hotel.

I agree in principle, but it's not clear to me that there's anything that can be done about it.   We do need to trust the folks on the IAOC to take this stuff into account.   When I was an AD I spent a certain amount of energy second-guessing their decisions, but ultimately each decision made sense when I understood the factors that were considered.  Well, except maybe Hawaii, but you can't win them all--I think the pull of history was too much to resist.

> Getting work done should be one

Good point.   Let's get back to work!

---

From: Jared Mauch <[email-masked]>
Date: Wed, 16 Dec 2015 12:09:21 -0500

> On Dec 16, 2015, at 12:05 PM, Ted Lemon <[email-masked]> wrote:
> 

I can say that the perceived PITA and “cheapness” of other groups I’ve been part of made at least one hotel make it more difficult to return.  They wanted to add ancillary charges, including a “charge to use the fiber you built in our property last time”.

I’ve posted on this before, there is a list of the largest hotels out there.

Review this list and if you exclude anything that says “Las Vegas”, I suspect you may have heard of an IETF or similar conference at them.

This list is almost exclusively places that have more than 1k rooms.

https://en.wikipedia.org/wiki/List_of_largest_hotels_in_the_world

- Jared

---

From: Melinda Shore <[email-masked]>
Date: Wed, 16 Dec 2015 08:11:30 -0900

On 12/16/15 8:05 AM, Ted Lemon wrote:
> This is an excellent point, and reinforces my belief that you'd be a
> good IAOC candidate.   

I have no interest in being a professional IETFer but I
have considerable interest in having meetings be effective
(I'd like to see them deprioritized, but as long as there
are working group chairs who treat them as the locus of
working group activity that's not going to happen).  There
are productivity costs associated with being at an
overflow hotel - working breakfast meetings become more
difficult and impromptu conversations become constrained
to the hours someone is in the building.

I don't know what Usenix is doing these days but back
when I was heavily involved we always seemed to find
hotels that could accommodate most attendees.  But, they
had a very, very skilled professional meeting organizer
on staff rather than relying on a committee of volunteers.

Melinda

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Dec 2015 09:16:04 -0800

> This is an excellent point, and reinforces my belief that you'd be a good IAOC candidate.   I think there's a case to be made that if someone needs special accommodations, there should be a way for…

This incurs significant, additional effort for IETF staff and for
attendees.   Th effort to identify and process such folk also produces
all sorts of fairness concerns.  A plausible result is effectively just
moving the announcement date to be earlier for everyone...

A different approach is simply to require that venues /always/ be chosen
with excellent, close proximity of alternative lodging that covers a
wide range of prices.

In fact, that's generally what has been done in recent years.

Except when it hasn't, for various reasons that are invoked sporadically...

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 16 Dec 2015 18:22:52 +0100 (CET)

> with excellent, close proximity of alternative lodging that covers a 

When I found out about the rooms being released, the $270 option was the 
only one still there. I went to look at the alternative hotels, and since 
most of them didn't have online booking (one was call only, one was call 
or email), I just went on booking.com, found a hotel 1.5km walking 
distance from the Hilton that had free wifi etc, and booked it. It was 
around 100USD per night.

I did the same in Yokohama because we had the same situation there, main 
conference hotel sold out way before I could book, and with 300 rooms, no 
wonder.

So while I prefer to stay at the conference hotel, there are usually 
alternatives, and this should definitely be taken into account when 
choosing venue... unless of course the venue is huge and good 
price/performance so everybody can stay there.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 16 Dec 2015 17:20:19 +0000

On 2015-12-16, at 18:11, Melinda Shore <[email-masked]> wrote:
> hotels that could accommodate most attendees.

All of the USENIX meetings I'm familiar with are exclusively in the US.

>   But, they

We have very professional meeting organizers on staff. Our requirements are the issue.

Lars

---

From: Ole Jacobsen <[email-masked]>
Date: Wed, 16 Dec 2015 09:27:16 -0800 (PST)

> on staff rather than relying on a committee of volunteers.

Just on your final point: The IETF does not solely "rely on
a committee of volunteers," we use a very skilled professional
meeting organizer called AMS, aka the IETF Secretariat.

Ole

---

From: Nadeau Thomas <[email-masked]>
Date: Wed, 16 Dec 2015 12:32:14 -0500

I am totally in agreement with Melinda.  Either meetings are important, or they are not. 
At this point, its clear that having meetings somewhere as part of the outreach program is, but that
clearly is not optimizing for the existing attendees. 

	To that end, I’d rather have meetings in a few locations regularly that can fit the entire attendee list, rather than working this plan to have the IETF in strange and odd locations as part of some “outreach” program to increase participation; its clearly not.  The odd/off-the-beaten path location situation is becoming a real problem given the added costs (time/money/etc…) for people, as well as clearly causing issues for people who regularly attend the meetings to progress ongoing work and are trying to stay in the meeting venue hotel.  This is now the second time in a row where this is happening and its a real problem. If you can’t stay in the main venue, you have added travel costs (as Melida pointed out) not to mention the likelihood of added hotel costs due to increased, non-negotiated rates once the relatively few rooms in the main hotel are exhausted.

	—Tom

> On Dec 16, 2015:11:59 AM, at 11:59 AM, Melinda Shore <[email-masked]> wrote:
>

---

From: Paul Wouters <[email-masked]>
Date: Wed, 16 Dec 2015 12:42:12 -0500 (EST)

On Wed, 16 Dec 2015, Mikael Abrahamsson wrote:

> When I found out about the rooms being released, the $270 option was the only 
> one still there. I went to look at the alternative hotels, and since most of 
> them didn't have online booking (one was call only, one was call or email), I 
> just went on booking.com, found a hotel 1.5km walking distance from the 
> Hilton that had free wifi etc, and booked it. It was around 100USD per night.

That's right. I was there for ICANN and stayed in the Hilton hotel which was
double the price and by far not as good as the independent boutique hotels
I stayed at after the ICANN conference. Also San Telmo is a much nicer
area to stay in and just a 2km walk (or cheap taxi).

Recommended coffee places: Coffee Town, Negro (seemed renamed to El Gato Negro)

Paul

---

From: "John Levine" <[email-masked]>
Date: 16 Dec 2015 17:41:55 -0000

>We already have a list of hotels which will.

Unfortunately, none of them are in Argentina.  In B.A., the Sheraton
where ICANN met has 740 rooms, the Hilton has 417.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 16 Dec 2015 17:49:06 -0000

> If you can’t stay in the main venue, you have added travel costs

Actually, taxis in Buenos Aires are really cheap.  A week's worth of
taxis won't cost as much as the $160 reciprocity fee that US passport
holders have to pay.

R's,
John

---

From: Paul Wouters <[email-masked]>
Date: Wed, 16 Dec 2015 12:53:09 -0500 (EST)

> taxis won't cost as much as the $160 reciprocity fee that US passport

Canadians have to pay USD 92 (which is about CAD 160, so you are still
right, but :)

http://www.canadainternational.gc.ca/argentina-argentine/consular_services_consulaires/visa.aspx?lang=eng

Paul

---

From: "Deen, Glenn (NBCUniversal)" <[email-masked]>
Date: Wed, 16 Dec 2015 17:54:27 +0000

I don¹t think the fault lies in either out volunteers or AMS, I suspect we
are quite simply not being treated well by hotels.

I just now booked the Hilton, however not via the IETF link, because the
IETF link had no rooms available for the days of the week I needed one.

When I used the IETF rate link, it said there was NO availability for the
week at the IETF rate.    I then logged onto to Hilton.com directly,
checked the hotel and was offered the $270 rate (plus 21% tax -wow!), but
it gave me the Internet Society rate - which was weird.   So to get a
room, I booked through directly through hilton.com at the higher rate.

I had to do the same in Yokohama.  I tried to book the Yokohama hotel via
the IETF within about 30 minutes of the announcement of booking being
open.   ZERO rooms where available, but when I booked directly (at a much
higher rate), I was able to get a room.

>From a user¹s perspective this is great for the hotel.  They can offer a
limited ³IETF² rate, that is very limited and hard to get, but they can up
sell attendees who are willing to spend more to get into the hotel at much
higher rate.  This creates a false impression that the IETF isn¹t filling
the hotel because many people are booked in directly and not in the IETF
block.  So the hotels can justify small blocks in the future.

The hotel¹s are the ones that win here.  They get the IETF meeting rooms
and food costs.  They sell a limited set of IETF rate teaser rooms. Then
they fill up the hotel with high rate rooms which are still taken by IETF
attendees.    That seems like a very bad faith behavior on the part of the
hotel.

I¹d like to understand what¹s going on with the room blocks the hotels are
giving us.  I know that hotels used to give us very good sized blocks and
they would take a while to fill up.  Now they are filling up immediately -
or are they?  
  
    Q- What¹s the room block size we are getting at the recent venues
compared to what we got at previous ones like Vancouver, or Berlin?
    
    Q - Are hotels artificially limiting availability of the IETF block by
only releasing parts of it to the web booking?
        I¹ve seen hotels do this for other events.  While the whole block
maybe 500 rooms, they release them in 50 room blocks as the
        reservation block fills.  This creates the lucky 10th caller
scenario, where if you hit it at just the right time you win.

-glenn

---

From: Lou Berger <[email-masked]>
Date: Wed, 16 Dec 2015 12:54:39 -0500

> That said, I think you'd make a great candidate for the IAOC next time if you feel like trying to disprove this point!   :)

Meeting issues, including the original announcement of this one, are
what made me finally put my name in for IAOC.  Unfortunately, we have  a
long (3 year) planning window so change takes more time than we'd all like.

Which of course is a partial answer to:

> after meeting after meeting?

I also think another answer is documenting, and getting community
consensus on, what are the important and prioritized criteria for
selecting a meeting location and venue.  This is something I personally
hope to have time to work on in the next few months...

Lou

---

From: Melinda Shore <[email-masked]>
Date: Wed, 16 Dec 2015 08:56:39 -0900

> taxis won't cost as much as the $160 reciprocity fee that US passport

For individual, self-funded participants marginal increases
in costs (beyond the necessary costs to attend) can turn
painful costs into prohibitive costs.  The decisions that
we're making about these things are implicitly decisions
that we're making about IETF participation.

I continue to believe that deprioritizing meetings would go
a long way towards addressing some number of problems,
but having beat this particular drum for some number of years
it's clear that ain't gonna happen.  But I'm pretty confident
that as long as meeting participation is basically required in
order to move work forward and as long as the meetings are
not optimized around getting work done, there will continue
to be issues.

Melinda

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 16 Dec 2015 13:12:15 -0500

> on staff rather than relying on a committee of volunteers.

Looking at their list of past meetings, the vast majority are in the
US. The ones outside the US are almost all regional conferences or the
annual storage conference in Haifa.

---

From: Melinda Shore <[email-masked]>
Date: Wed, 16 Dec 2015 09:15:29 -0900

> annual storage conference in Haifa.

Yes, but that's how they've been able to have conference
facilities large enough for most attendees.  It's clear
that there are tradeoffs in the choice of meeting venues.
We've made the choice to prioritize location over facilities.
These are the consequences.

Melinda

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 16 Dec 2015 13:26:26 -0500

> These are the consequences.

USENIX does not play anything like the role that the Internet
plays in the global infrastructure. There are very good reasons
why folk outside the US were concerned about the US-centric
nature of the IETF and very good reasons why the IETF needs
to make being visibly open a priority.

USENIX is not the subject of any international treaties that I am
aware of. There are quite a few now that are very much about
the Internet.

On the reciprocity fee thing, Chile dropped their $160 fee
when the US dropped their $160 fee last year. It is possible
that a similar agreement will be negotiated with Argentina
before next April.

---

From: "John R Levine" <[email-masked]>
Date: 16 Dec 2015 13:40:24 -0500

> Canadians have to pay USD 92 (which is about CAD 160, so you are still

The reciprocity fee is set to match the cost of the corresponding visa 
application.  It's USD 160 for US passports, or USD 100 for Canadian or 
Australian passports.  It's good for one year for Australians, ten years 
for Americans and Canadians so don't lose it.  You can reprint it from 
their web site if you can remember when you applied for it, but it's 
easier just to save the PDF and/or printout.

http://cnyor.mrecic.gov.ar/en/content/reciprocity-fee-us-canadian-and-australian-citizens-online-payment

I got mine for a meeting in 2013 so this is the third time I'll use mine. 
What a bargain.

R's,
John

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 16 Dec 2015 11:35:06 -0800

+1 on the analysis. Which is yet another reason why i avoid
booking conf hotel at higher rates. It would just support their rigged scheme.

---

From: Sarah Banks <[email-masked]>
Date: Wed, 16 Dec 2015 13:05:24 -0800

Since we're piling on, I'll +1 this. I don't recall getting a hotel room the day of registration opening ever being such a huge issue as it has been. As a female, I prefer to be in the venue; it allows to meet as early or late as I want to, without having to worry about the walk "home", and the logistics that come with that. 

/S

> On Dec 16, 2015, at 8:59 AM, Melinda Shore <[email-masked]> wrote:
>

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 16 Dec 2015 16:41:55 -0500

> Since we're piling on, I'll +1 this. I don't recall getting a hotel room the day of registration opening ever being such a huge issue as it has been. As a female, I prefer to be in the venue; it al…

You obviously have not been involved with the IETF very long. There
were many years when it was the normal case that the meeting hotel
sold out immediately. And that was before the IAOC even existed and
before the regrettable decision to plan a meeting location on
marketing even though this inconveniences most attendees.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

---

From: "Livingood, Jason" <[email-masked]>
Date: Wed, 16 Dec 2015 22:17:42 +0000

On 12/16/15, 12:49 PM, "ietf on behalf of John Levine"
<[email-masked] on behalf of [email-masked]> wrote:
>>If you can¹t stay in the main venue, you have added travel costs
>

AND it is about to get cheaper following the dropping of currency controls
(and expected subsequent devaluation) that is about to happen under the
new President of Argentina.

Jason

---

From: "John Levine" <[email-masked]>
Date: 16 Dec 2015 22:30:08 -0000

>new President of Argentina.

Only for people who pay with credit cards or get cash from their
foreign accounts at ATMs.  At this point, the official rate that banks
use is about 9.5 pesos/USD while the "blue" rate that you get for your
$100 bill on Florida St is 14.5.

When the government drops currency controls, the official rate will
move toward the blue rate.  The blue rate builds in a bit of a risk
premium since it's officially illegal (although you can't walk two
metres on Florida without someone saying "cambio, cambio") so the
future real rate is likely to be not quite as good as the blue rate.

R's,
John

---

From: "Livingood, Jason" <[email-masked]>
Date: Wed, 16 Dec 2015 22:43:19 +0000

On 12/16/15, 5:30 PM, "John Levine" <[email-masked]<mailto:[email-masked]>> wrote:

Yes, that is happening now. http://www.bloomberg.com/news/articles/2015-12-16/argentina-to-announce-lifting-of-currency-controls-today So things will be very different by the time of the IETF meeting (it is very fluid and hard for any of us to speculate how it will work, as much in Argentina seems to be changing).

Jason

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 16 Dec 2015 18:05:42 -0500

Hi,

> clearly is not optimizing for the existing attendees. 

I don't understand this claim.  What is the history where we are
having meetings as part of outreach, _except_ for the current example
(which is admittedly outside our usual rotation).  We have a 1/1/1
rotation because we decided to share the pain among regions where most
of our participants come from, not from "outreach".  It's not
"outreach" to have a meeting in Japan, where a significant number of
participants live.

I certainly understand people's concern about availability at the
hotel, and I'm sure the IAOC and the meetings committee will respond
with the data people need.  But we must not mischaracterise the
reasons why we move around the world.

Best regards,

A (for myself and nobody else)

-- 
Andrew Sullivan
[email-masked]

---

From: Sarah Banks <[email-masked]>
Date: Wed, 16 Dec 2015 15:06:11 -0800

So, since you've been with the IETF you have a solution to this? That it happened a lot before my time doesn't make it any less of a pain in the rear.

Regards,
Sarah

> On Dec 16, 2015, at 1:41 PM, Donald Eastlake <[email-masked]> wrote:
>

---

From: Tim Chown <[email-masked]>
Date: Wed, 16 Dec 2015 23:13:38 +0000

> On 16 Dec 2015, at 17:09, Jared Mauch <[email-masked]> wrote:
> 

Interesting.

But have we had over 1,000 rooms in one venue in recent memory? 

I looked back at old meeting invitation emails, and it seems of late the number of rooms in the reserved block are generally not explicitly stated. It was more common for older meetings. From those I can see:

IETF 77, Anaheim, 1,000 rooms
IETF 73, Minneapolis, 750 rooms
IETF 80, Prague, 600 rooms

But these were 5+ years ago. It may simply have become harder over time to get such large blocks? 

Looking at IETF 96 in Berlin, the InterContinental Berlin only has 558 rooms and suites, and the Conrad Seoul for IETF 97 has 434 rooms.

Tim

---

From: Ray Pelletier <[email-masked]>
Date: Wed, 16 Dec 2015 18:24:59 -0500

Glenn,

> On Dec 16, 2015, at 12:54 PM, Deen, Glenn (NBCUniversal) <[email-masked]> wrote:
> 
>    Q- Whatąs the room block size we are getting at the recent venues

There are many hotels in Buenos Aires but they do not have many rooms.

When we announced on 10 December that registration would open on 16 December 
we said:

Registration and hotel reservations for IETF 95 in Buenos Aires will
open at 1500 UTC Wednesday, December 16. Hotel reservations will include
the headquarters hotel and the contracted overflow hotels available
at that time.

Because hotels in Buenos Aires have a limited number of guest rooms,
the IETF has been negotiating contracts with ten hotels.  Hotels not
available on December 16 will be announced when available.

On 16 December we said:

1. Hilton Buenos Aires (Headquarters Hotel, block of 300 rooms)
2. Holiday Inn Express Puerto Madero (30 rooms)
3. Sheraton Buenos Aires (140 rooms available)
4. InterContinental Buenos Aires (150 rooms)
5. Sheraton Libertador Hotel (70 rooms)

Buenos Aires is not Vancouver, Berlin, London, Paris or San Francisco.  
It’s more like Dublin, if you recently attended the ICANN conference there.  

>        Iąve seen hotels do this for other events.  While the whole block

That’s not the case here, or anywhere we have negotiated agreements.

We strive to contract for 600 on a peak night at the so-called headquarter’s hotel, 
but it depends on where we are. If we are in an area surrounded by hotels, and
at lower price points, we might contract for 400 on a peak night, and overflow
hotels for another 200 - 300 rooms on peak to get 780 rooms on peak.

Typically if we don’t get our target room block it’s because there’s another group
at the hotel, or sometimes it’s because the hotel is concerned about the risk 
of setting aside 70 - 90% for a group they’ve never done business with.  This concern
is also typically reflected in the cancellation provisions for guest rooms they will 
sign up to, and/or when they start cutting back the number of rooms in the block.  
All of which is evident in our first meeting in Latin America and Buenos Aires.  

For those who managed to make a reservation outside the IETF block, but at a 
higher price, we are trying to get you in the block at the IETF rate.  

Another 4 or so IETF contracted hotels are in the works and will be announced 
as soon as they are ready.

Ray

---

From: Brian Rosen <[email-masked]>
Date: Wed, 16 Dec 2015 19:10:43 -0500

Thanks Ray.

Can you tell us if the block was available to any people before the opening this morning?  We’re all 300 rooms available at 1500 UTC?
Does the contract have a split between the 209 rate and the 270 rate?  If so, what was the split?

I was prepared for this.  You told us what would happen, and everything you said would happen, happened.  I moved quickly, and got a room at the Hilton at the 209+tax rate.

Brian

> On Dec 16, 2015, at 6:24 PM, Ray Pelletier <[email-masked]> wrote:
> 
> Glenn,
>

---

From: "Deen, Glenn (NBCUniversal)" <[email-masked]>
Date: Thu, 17 Dec 2015 00:14:07 +0000

Thanks Ray for clarifying things.  I know that setting this meeting up has
been a lot of effort due to currency problems, locations not wanting to
make long term commitments to the iETF, etc.

And to be fair, YOU and IAOC DID warn us that rooms would be limited at
the Hilton.  

You're also right that BA is more like Dublin, as I attended both ICANN
Dublin and ICANN BA this past year, and the ICANN attendees where very
much split up between hotels.   In both places it worked well for ICANN
and they are 2-3 times as big as an IETF. Though I suspect they are much
less sensitive to event pricing as the IETF is.  The fact they don't
charge any attendance fee, reflects the difference in funding level ICANN
enjoys.   Attendees were spread out between many hotels in BA and it
seemed to work, at least I didn't hear much in the way of complaints.
There also seemed to be a lot of good hotels in the area at different
price points.  

In BA taxi's as John L. noted are cheap, though they can be hard to find
at times, and they didn't use Uber (there was some other taxi app that I
need remember before we go down there).  That should hopefully make it
much easier for people not at the venue hotel to move between their hotels
and the venue both quickly, comfortably, and safely.

Unfortunately for the IETF, there are a lot of competing large groups all
looking for meeting space+rooms, and it isn't likely to get better for a
while. Which is good because it means the economy is continuing to improve
and that will help support more participation, but it's bad because it
means pressure to pay more to make the IETF more attractive than other
groups, and a lot of those other groups are playing with much bigger
budgets.

Spending more isn't necessarily the only option, and it would change the
IETF.  Sure we could be more attractive, and get massive room blocks if we
just spent a lot more money, but one of things I think that helps the IETF
continue to be so successful, is that it's run as a pretty lean ship. It
has let the IETF continue during tough years like 2008-2010 without making
major changes.  People should look at the IAOC's presentation on the IETF
budget, there is a lot of good info there. Compared to nearly every
organization of it's size, the IETF is run very effectively on a lean
budget, THAT"S A GOOD THING and something that I know Ray and IAOC work
hard to maintain. Thank you BTW.    At the same time: We enjoy
comparatively cheap registration fees compared to most other week long
technical events. PLUS We get great cookies and even ice cream at breaks,
and cokes at breakfast! PLUS We have solid sponsors who have treated us to
great socials. PLUS BEST OF ALL: We get great meeting rooms and the best
network and wifi anywhere.  So like many things, all could be fixed by
spending more $$$$, but that's the easy way out, but not necessarily the
best way.

A couple of suggestions that may alleviate some of the frustrations:

1.  Can an weekly update count of block rooms and nights available be
maintained, perhaps weekly on the IETF meeting site?  This would help
people making reservations
    know when they can skip trying a hotel because it is sold out.   I
know this is hard to keep 100% current, because some people will book &
then cancel, but any info is helpful.

2. Is it cost feasible to run some sort of shuttle between the venue and
the overflow hotels?  It may not be, but it would help with the distance
and safety concerns.

3. A longer term solution is to perhaps increase our bargaining power by
building longer term relations with a few locations. I see that we do some
of this already, perhaps formalizing it might be an option.

So instead of spending more $$$, one solution might be to change from
having such a wide variety of places we hold meetings to having a small
set of locations, which have hotels which will give us enough space,
including nearby overflow hotels, and good meeting space.  That way we
could build multi-year relationships with individual venues and hotels.

We could still do the 1:1:1 rotation, but with perhaps 2 or 3 favored
locations in each of the 1:1:1 regions.  Having a couple in each region
gives us a fall back if something isn't available and a better negotiating
position.      

This might look like:   Vancouver/Japan/Honolulu/? for AP,
Berlin/Paris/Prague/? for EU and San Diego/Vancouver/Boston/Philadelphia/?
for NA.

-glenn

On 12/16/15, 3:24 PM, "Ray Pelletier" <[email-masked]> wrote:

>It's more like Dublin, if you recently attended the ICANN conference
>there.  
>
>> 
>That's not the case here, or anywhere we have negotiated agreements.
>
>We strive to contract for 600 on a peak night at the so-called
>headquarter's hotel,
>Typically if we don't get our target room block it's because there's
>another group
>at the hotel, or sometimes it's because the hotel is concerned about the
>risk 
>of setting aside 70 - 90% for a group they've never done business with.
>This concern

---

From: Christian Hopps <[email-masked]>
Date: Wed, 16 Dec 2015 19:16:03 -0500

> All of which is evident in our first meeting in Latin America and Buenos Aires.

I believe that IETF is probably trying as hard as it can to do the best job it can; however, I wonder about something from above:

I'm not sure if "all of which" included the "never done business with" or how exactly hilton works, but it seems to me that we've done quite a bit of business with hilton. Is one hilton not able to talk to our previous hilton's or corporate headquarters to get information on our group? This would be unfortunate as that information seems very useful for the new hilton to have.

Thanks,
Chris.

---

From: "John Levine" <[email-masked]>
Date: 17 Dec 2015 01:54:01 -0000

>But have we had over 1,000 rooms in one venue in recent memory? 

Not that I recall, but there's generally been reasonable alternatives
within a few blocks.  The B.A. Hilton is in a dramatic location, but
it's a long way from anything else.

>Seoul for IETF 97 has 434 rooms.

The last time we were in Berlin I stayed at the reasonably priced
Hotel Lützow which was about three blocks from the IC, and there were
plenty of other hotels in the immediate area.

R's,
John

PS: I am not faulting the IAOC for any of this.  Once we made the
commitment to go to Argentina, the options are pretty limited.  There
are for example lots of hotels close to each other in Mar del Plata
where ICANN met a decade ago, but I suspect the six hour bus trip
across the pampas from the B.A. airport might provoke a few complaints.

---

From: "John Levine" <[email-masked]>
Date: 17 Dec 2015 01:58:44 -0000

>previous hilton's or corporate headquarters to get information on our group? This would be unfortunate as
>that information seems very useful for the new hilton to have.

B.A. Hilton: tell us about the IETF

Headquarters: they usually have over 1000 people, nearly all of whom want to stay at the venue

B.A. Hilton: ¡madre de dios! our hotel only has 417 rooms and some of them are already booked

Headquarters: <OK, you take it from here>

---

From: Fernando Gont <[email-masked]>
Date: Thu, 17 Dec 2015 01:16:35 -0300

On 12/16/2015 07:43 PM, Livingood, Jason wrote:
> Yes, that is happening
> now. http://www.bloomberg.com/news/articles/2015-12-16/argentina-to-announce-lifting-of-currency-controls-today So
> things will be very different by the time of the IETF meeting (it is
> very fluid and hard for any of us to speculate how it will work, as much
> in Argentina seems to be changing).

What is the "much" that is changing?

-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Fernando Gont <[email-masked]>
Date: Thu, 17 Dec 2015 01:48:43 -0300

> out for people with limited mobility?

Never stayed at that area. But using www.booking.com you can find places
like 600m away from the Hilton (e.g.
<http://www.booking.com/hotel/ar/apartment-terrazas-del-dique.en-gb.html?aid=306395;label=buenos-aires%2Fpuerto-madero-Eelst*LpyP7U16n_myyEqgS25635933629%3Apl%3Ata%3Ap15%3Ap2%3Aac%3Aap1t1%3Aneg%3Afi%3Atikwd-3626240213%3Alp2218%3Ali%3Adec%3Adm;sid=6e7151c730296ad61e349d7d2aa7561d;dcid=1;checkin=2016-04-02;checkout=2016-04-09;dist=0;group_adults=1;group_children=0;no_rooms=1;room1=A;sb_price_type=total;srfid=67563b9f784cbdb5c96168ed26f4ad5e7f3eda09X5;type=total;ucfs=1&>)

Thanks,

---

From: Tim Chown <[email-masked]>
Date: Thu, 17 Dec 2015 09:35:13 +0000

> On 17 Dec 2015, at 01:54, John Levine <[email-masked]> wrote:
> 

Just a question of how far a “few blocks” is… some people prefer the
all-in-one option (for a variety of very valid reasons), others like to find 
a cheaper hotel and/or perhaps embed themselves in local culture a
little more than you can in a cookie-cutter Hilton.

Does the post-meeting survey ask people if they got the accommodation
of choice? Might be interesting to know how many people really wanted
to be in the main meeting hotel but couldn’t do so. It may be a small,
but vocal, minority, or it may be many more, but such a question would 
help inform the organisers on main hotel sizing.

> Hotel L�tzow which was about three blocks from the IC, and there were

I recall the options in Berlin are good. Just pointing out that the people 
hoping for a big change to current practice aren’t going to see it in the 
immediate future.

Tim

---

From: Jari Arkko <[email-masked]>
Date: Thu, 17 Dec 2015 14:36:21 +0200

Hi all,

I wanted to first apologise for the difficult situation with hotels (again) - I know
this is important, the IAOC knows this is important, and we are trying. But
I realise that we’re not doing as well as we should. For various reasons.
I refer to what Ray said and what Lou said and what several people
such as Glenn have said about hotel business in general. And thank
you Ray for providing numbers; we’d be happy to provide any other
numbers that people want to see. I also feel bad that I myself am
contributing to the problem as the meeting organisers do set up
a part of the room block marked for, for instance for the IESG
members or the secretariat.

From the IAOC and meeting organising staff point of view, one of the
parameters that we try to figure out is what block size works for us
in the main hotel. There are very good reasons to make that block
big; but there are also some reasons to not make it too big, risk
for us and the hotel, reduction in the number of hotels that we
can even consider, cost increases, etc. The IAOC wants to run
the IETF finances tight so we have been looking at the room
blocks, the number of small meeting rooms and other
factors that we ask for in our hotel contracts. Yokohama
and Buenos Aires have made at least me rethink this,
and maybe our balance isn’t where it should be. Unfortunately,
the hotel deals are made well ahead in time, so it is difficult to
course correct immediately.

I also liked Lou’s prioritised requirements effort.

A couple of other notes, mostly from a personal perspective.
I’ve been to Buenos Aires for a couple of meetings by now,
and I never stayed in the main meeting hotel, either because
I wanted my employer to have to pay less (100$ gets you
many hotels and I wasn’t going to have a vacation in my
room anyway, so a place to sleep was fine) or
once because the meeting host who had invited me had
me placed in one of the surplus hotels due to lack
of availability in their main hotel (which was the Sheraton).

But moving around in the city was easy. I walked everywhere.
I also used taxis, which are cheap and plentiful.

The waterfront area between the Hilton and the main city
center has plenty of good restaurants.

I’ve been to the Hilton, for a meeting, but not stayed at
it. Looked very classy and modern. Compared to the
Sheraton It is newer but also smaller and a bit longer
walk away from the area that the hotels are more plentiful.

Jari

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 17 Dec 2015 05:32:05 -0800

Jari,

Thanks for the explanation, but i had much simpler question i raised
on this thread, but i fear i am not getting them answered, so let me add
the IAOC mailing list:

1. How many rooms at the Hilton where blocked out at the rate of $209 ?
2. How many rooms at the Hilton where blocked out at the rate of $270 ?
3. Home many rooms at the hilton at the rate of $209 where available for
   booking at the time yesterday when the official announcement was made to the IETF mailing list ?

Thanks
    Toerless

On Thu, Dec 17, 2015 at 02:36:21PM +0200, Jari Arkko wrote:
> I realise that we?re not doing as well as we should. For various reasons.
> you Ray for providing numbers; we?d be happy to provide any other
> and maybe our balance isn?t where it should be. Unfortunately,
> I also liked Lou?s prioritised requirements effort.
> 
> I?ve been to Buenos Aires for a couple of meetings by now,
> many hotels and I wasn?t going to have a vacation in my
> I?ve been to the Hilton, for a meeting, but not stayed at

-- 
---
Toerless Eckert, [email-masked]

---

From: Jari Arkko <[email-masked]>
Date: Thu, 17 Dec 2015 15:59:13 +0200

>   booking at the time yesterday when the official announcement was made to the IETF mailing list ?

I don’t know the answer to this but we’ll get you one.

Jari

---

From: Jared Mauch <[email-masked]>
Date: Thu, 17 Dec 2015 09:04:05 -0500

> On Dec 17, 2015, at 8:59 AM, Jari Arkko <[email-masked]> wrote:
> 
> 

I know myself and several co-workers all got rooms at the hilton.

- Jared

---

From: Behcet Sarikaya <[email-masked]>
Date: Thu, 17 Dec 2015 11:00:51 -0600

> Berlin/Paris/Prague/? for EU and San Diego/Vancouver/Boston/Philadelphia/?

+1 for these suggestions.
I think IEEE 802 does this type of business. Their policy is to keep
people in the same hotel, by providing registration fee discount up to
$300. That means going to the same places every two years or so.
That's the price you pay, there is no silver bullet.

Behcet

---

From: Warren Kumari <[email-masked]>
Date: Thu, 17 Dec 2015 18:10:04 +0000

> That's the price you pay, there is no silver bullet.
Yup. I have been to a number of IEEE meetings and stayed in the IEEE hotel
in order to get the discount - the next 802 Plenary is at the Venetian in
Macau. The registration page (
http://802world.org/plenary/ieee-802-group-hotel/ ) says:
-------------------------------
Group Hotel Registration Policy
The combined hotel room stays of attendees are used to offset the cost of
the meeting. For this reason, anyone who is not a registered guest at the
Sands Venetian Macao Hotel for three (3) or more nights is not eligible for
the registration discount ($US 450.00).
------------------------------

The room rate starts at 1,550.00MOP (~$180USD). The discounted registration
fee is $600USD.
The non-discounted fee is $1050USD.

The IEEE seems to have many fewer requirements / expectations from their
attendees.
For example, one of the recent meetings was in the Estrel in Berlin, a
hotel which would not suit the IETF at all.
Perhaps if IETF attendees didn't demand everything that we do (lots of
breakout rooms, walking distance to bars and restaurants, no trains, the
ability to install and run our own network, not being in Minneapolis, large
cookies, specific price points, a willingness to keep going back to the
same N locations) we wouldn't have so much kvetching.

W

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 18 Dec 2015 08:36:27 +0000

On 2015-12-17, at 19:10, Warren Kumari <[email-masked]> wrote:
> Perhaps if IETF attendees didn't demand everything that we do (lots of breakout rooms, walking distance to bars and restaurants, no trains, the ability to install and run our own network, not being in Minneapolis, large cookies, specific price points, a willingness to keep going back to the same N locations) we wouldn't have so much kvetching.

+1

Lars

---

From: Leif Johansson <[email-masked]>
Date: Fri, 18 Dec 2015 15:49:48 +0100

+1

---

From: Tim Chown <[email-masked]>
Date: Fri, 18 Dec 2015 16:45:13 +0000

> On 18 Dec 2015, at 08:36, Eggert, Lars <[email-masked]> wrote:
> 
> On 2015-12-17, at 19:10, Warren Kumari <[email-masked] <mailto:[email-masked]>> wrote:

Indeed, Minneapolis is great!

Tim

---

From: Brian Rosen <[email-masked]>
Date: Thu, 17 Dec 2015 12:44:29 -0500

>> Berlin/Paris/Prague/? for EU and San Diego/Vancouver/Boston/Philadelphia/?
Minneapolis!!!!!!

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 18 Dec 2015 09:16:54 -0800

> http://802world.org/plenary/ieee-802-group-hotel/ ) says:

3000 rooms there. If every attendee could get a room at the $180 rate,
then this wouldn't be such a bad deal. Seems like the opposite of the current 
IETF problem: Large hotel with too many rooms that need to be filled. And
several other competing hotels in walking distance.

WOuld be interesting to know if IETF ever ran into the problem
of not filling up blocked rooms... 

> hotel which would not suit the IETF at all.

Why not ? 1150 room would be perfect size. Probably because IETF wouldn't
reserve enough of the conference space, so we wouldn't get a good deal with them ?

> same N locations) we wouldn't have so much kvetching.

+1 on almost everything. I think inexpensive helps diversity, and
number of breakout rooms is just at the core of our work.

And ++1 on Minneapolis in November again.

Cheers
    Toerless
> > >>>    Q- What??s the room block size we are getting at the recent venues
> > >>>        I??ve seen hotels do this for other events.  While the whole
> > block

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 18 Dec 2015 17:17:02 +0000

On 18/12/2015 16:45, Tim Chown wrote:
>> On 18 Dec 2015, at 08:36, Eggert, Lars <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>>> Perhaps if IETF attendees didn't demand everything that we do (lots 
>>> of breakout rooms, walking distance to bars and restaurants, no 
>>> trains, the ability to install and run our own network, not being in 
>>> Minneapolis, large cookies, specific price points, a willingness to 
>>> keep going back to the same N locations) we wouldn't have so much 
>>> kvetching.
>>
>> +1
>
Yes, as far as I am concerned Minneapolis is a great place to meet.

Stewart

---

From: Warren Kumari <[email-masked]>
Date: Fri, 18 Dec 2015 17:20:58 +0000

So, I suspect that at least part of this is caused by there being a number
of different incentives at work / people optimizing for a number of
different things[0].

1: Some people attend IETF purely for getting work done - this means that
they are perfectly happy to go back to the same N hotels, over and over.
They don't *really* care what the destination is like, they don't need
pretty beaches and holiday amenities. During they day they spend most of
their time in sessions, and in the evening they spend much of their time
hanging out with other IETFers. They will probably attend all / most
meetings, regardless of where they are. Minneapolis is fine with them.

2: Some people like to combine their IETF work with a vacation / only show
up for a session or two, and spend a significant amount of their time
sightseeing / wandering around, etc.
These folk do not like going back to the same destinations, they like
locations (like Hawaii, Paris, Prague, etc) where they can go do other
stuff. They are much less likely to attend a meeting in a location that
they don't like (e.g they may skip Minneapolis in the winter).

3: Some set of people are responsible for actually, you know, running the
meetings. They have many incentives, at least one of which is actually
having enough money to be able to run the meetings, and have people happy
with the meeting. Economies of scale makes it better to have lots of
people... Seeing as people from set 1 are likely to attend everywhere where
people from set 2 do, but not the opposite, it makes sense to optimize for
set 2.

4: Folk from set 1 seem to be more vocal then people from set 2, but a:
there are lots of people in set 2 / there is a significant overlap -- this
was demonstrated by: A: the amount of discussion about "Whee, let's go to
Minneapolis" versus B: the results of the meeting location survey done a
few meetings back.

5: Some set of folk think that planning a meeting and choosing a location
is easy. If you open a browser and type in "hotel" there are many, many
results. This *surely* means that we can easily find one that will take out
money. After all, we are a "prestigious" organization, no hotel is going to
give up a thousand or so guaranteed guests, and, well, the $confernce met
there a few years ago. Unfortunately $confernce is not the same as the
IETF. There are many many constraints and tradeoffs that the IETF meeting
planners have to take into account. I'm part of the NOC team and regularly
chat with someone who is involved in some of the site selection. One of my
favorite games is "Why don't we go to XXX?" - fairly much anywhere I
suggest has already been considered, and there are good reasons (often
surprising) that it has been disqualified.
Believe it or not, the meeting selection committee / IOAC is not sitting
around scratching their armpits and / or optimizing for ways to make people
sad...

W

---

From: Melinda Shore <[email-masked]>
Date: Fri, 18 Dec 2015 08:41:45 -0900

On 12/18/15 8:20 AM, Warren Kumari wrote:
> 5: Some set of folk think that planning a meeting and choosing a
> location is easy. 

I doubt anybody thinks it's easy.  I think most of us
who have a problem with the meeting situation are aware
that it's quite difficult.

Speaking strictly for myself, my problem with what's been
going on is that on the one hand we are de facto making meetings
the center of IETF activity for most participants and yet we're
not trying to optimize around the effectiveness of the meeting
space.  I think we possibly could do something around the
former but I don't think it's likely we will, based on both past
discussion and the behavior of very many very experienced
working group chairs (another discussion).  But we've got
two situations in tension with each other.

I don't think your characterization is correct or reasonable
and I found your comments a bit dismissive.

Melinda

---

From: Warren Kumari <[email-masked]>
Date: Fri, 18 Dec 2015 17:42:36 +0000

> reserve enough of the conference space, so we wouldn't get a good deal

I'm not sure of all of the reasons, but:
1: there is basically nothing within walking distance. When I wanted to get
out of the hotel for food I managed to find a (bad) gyro / curry wurst
vendor. IETF people have repeatedly demanded many nice food places within
walking distance.
2: There do not seem to be nearly enough break out rooms. A number of the
IEEE meetings I attended were in guest rooms that they had taken the beds
out of.
3: The food options in the hotel were quite limited. I really enjoyed
the Schweinshaxe, but after the N days many people would probably get bored
of it...
3: As far as I remember, their network infrastructure was, um, sad.

> And ++1 on Minneapolis in November again.

Yup. I too like the Minneapolis idea (as do a number of other people who
seem to be very active on the list / in the IETF), but it seem that we are
in the minority:
https://iaoc.ietf.org/documents/Venue-Preference-Survey-2014.pdf

 W

> > > >>>    Q - Are hotels artificially limiting availability of the IETF
> block
> > > >>>by
> > > >>sign up to, and/or when they start cutting back the number of rooms
> in
> > > >>the block.
> > > >>All of which is evident in our first meeting in Latin America and
> Buenos
> > > >>Aires.
> > > >>
> > > >>For those who managed to make a reservation outside the IETF block,
> but
> > > >>at a

---

From: Michal Krsek <[email-masked]>
Date: Fri, 18 Dec 2015 18:44:44 +0100

Let me add another point of view. And it is geographic.

I do not believe Internet is US effort (for decades) nor effort being 
funded by big global corporations (hopefully). If we want to open the 
IETF to "foreigners" we need to go there (with the price causing 
difficulties to incumbent geeks). Be sure - it is harder to visit IETF 
if you are male engineer living to India than female engineer living to 
San Francisco.

I'd strongly propose to go to new locations (as I saw a lot of new 
activities here after IETF came first time), and tune-up the remote 
participation systems.

And - it would be great if IOAC/AMS is providing as much information as 
possible, but I do not expect they are our babysitters. I'm always 
responsible to find a suitable hotel, tickets and cost of living each 
IETF according the budget (which is sometimes 0, so I'm not coming).

If we want to be closed group, let's select Minneapolis three times per 
year and we are done. Everything is a kind of compromise, do not be selfish.

No jokes here, please not be offended by ordinary english.

         Michal (speaking as "foreginer")

P.S: It is almost sure I'm not coming to BA as it is most likely over my 
budget, but I still think this location is good for the IETF and for the 
Internet.

Dne 18.12.2015 v 18:20 Warren Kumari napsal(a):
> So, I suspect that at least part of this is caused by there being a 
> number of different incentives at work / people optimizing for a 
> number of different things[0].
>
> location is easy. If you open a browser and type in "hotel" there are 
> many, many results. This *surely* means that we can easily find one 
> that will take out money. After all, we are a "prestigious" 
> organization, no hotel is going to give up a thousand or so guaranteed 
> guests, and, well, the $confernce met there a few years ago. 
> Unfortunately $confernce is not the same as the IETF. There are many 
> many constraints and tradeoffs that the IETF meeting planners have to 
> take into account. I'm part of the NOC team and regularly chat with 
> someone who is involved in some of the site selection. One of my 
> Believe it or not, the meeting selection committee / IOAC is not 
> sitting around scratching their armpits and / or optimizing for ways 
> to make people sad...
>
> W
>
>

---

From: Warren Kumari <[email-masked]>
Date: Fri, 18 Dec 2015 18:47:00 +0000

> and I found your comments a bit dismissive.
I became frustrated by the fact that there almost always seems to be a
thread where the tone is that the IAOC / meeting selection screwed up, and
that they should be optimizing for Y instead[0].

Meeting locations are always going to be a compromise, involving complex
(and shifting) tradeoffs and constraints. I believe that the IAOC / meeting
selection committee is trying very hard to meet the criteria, and it feels
to me that the tone of these discussions often doesn't give them enough
credit...

I was not intending to be dismissive (nor to imply any any specific person
was in any particular group); my frustration may have caused my to a:
oversimplify and b: word things poorly.

I apologize,
W

[0]: Where Y seems to change.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 18 Dec 2015 11:00:13 -0800

On 12/18/2015 9:44 AM, Michal Krsek wrote:
> If we want to open the IETF to "foreigners" we need to go there

The IETF already goes everywhere and always has.  Mailing lists are open
to all, free, accessible everywhere, and remain the formal base of
authority for working group decisions.

As for the recruiting benefits of having a face-to-face meeting in a new
area of the world, it is an entirely appealing idea, but I believe there
is no objective data demonstrating that this is effective, or at least
none within the last 15 years.

The basic rule is supposed to be that we go where the workers /already/
are (or, at least where they already are emerging from.)  The
alternative is to inconvenience 1200-1500 existing participants on the
very high-risk gamble that the IETF will pick up a few new ones.

d/

---

From: Ray Pelletier <[email-masked]>
Date: Fri, 18 Dec 2015 15:49:53 -0500

All;
> very high-risk gamble that the IETF will pick up a few new ones.

Let’s put this in the context the IAOC has been using for years now.

Every location is different.  It’s near impossible to get the perfect venue.

We also diligently try to meet the 1-1-1* objectives:

In 2010 after discussion with the community the IAOC set the Regions
3 years in advance following the 1-1-1* model per year.  1 meeting
in Europe, 1 in North America, and 1 in Asia/Pacific, with a
possibility of a meeting outside those regions, or for a Host to
volunteer for a particular region inside the 3 year planning period,
or beyond it.

The following are the objectives desired in the selection of an IETF meeting venue:

   1. Advancing standards development
   2. Facilitating participation by active contributors
   3. Encouraging new contributors
   4. Sharing the travel pain; balancing travel time and expense across the regions from 
	where IETF participants are based.

The is the list of the criteria that is evaluated when selecting a venue. 
This is not in any priority order.

	1.  An Optimal Facility for an IETF meeting is held under "One
	Roof", that is, qualified meeting space and guest rooms are
	available in the same facility.
	
	2.  The Meeting Space must be adequate in size and layout to
	accommodate meeting and participant networking where possible.
	
	3.  The Technology must be capable of supporting meeting network
	requirements.
	
	4.  The Guest Rooms at the headquarters hotel must be sufficient
	in number, have an adequate network, or the Hotel will accept
	the IETF network, and within close proximity to venue.
	
	5.  The Venue Surroundings must be surrounded by other hotels,
	restaurants and bars, and within walking distance; not isolated
	
	6.  The Cost of guest rooms, meeting space, food and beverage
	must be affordable.
	
	7.  The meeting should be Hostable and Sponsorable, that is, the
	Meeting is in a location and at a price that it is possible and
	probable to find a host and sponsors.  Host costs typically
	include the cost of the meeting space, network operation and
	circuits. [Note: ~$400,000 USD]
	
	8.  Overflow Hotels are hotels under contract typically within
	walking distance of the venue and have a variety of guest room
	rates.  They do not include the IETF network, but typically have
	free Internet.
	
	9.  Travel is a consideration, ease of access and number of
	hops.  Travel to the venue should be reasonably 	acceptable
	based on cost, time, and burden for participants traveling from
	multiple regions.   Also, are there other travel barriers to
	entry, e.g., visas?
	
	10.  Can the venue and hotels be put under contract?
	
	11.  Risk - Review risks associated with this venue over
	selected dates.
	
	12.  Are there health and safety concerns for the participants at this
	venue?
	
Ray
IAD

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 18 Dec 2015 21:31:06 +0000

Let me ask a question. I'm on the IAOC Meetings committee, which is an advisory committee that does some research (with AMS) and makes a recommendation to Ray, which he then takes to the IAOC. The IAOC sometimes agrees with us and sometimes doesn't. You will have just seen a note from Ray on this mailer detailing the IAOC's objectives in meeting planning; our committee, with strong involvement from AMS, does the investigative legwork to try to achieve those.

Right now, I am suggesting a model to Ray, based on a proposal that we have seen that would build a multi-meeting contract with a certain hotel. As with most business, matters, it would be inappropriate for me to discuss a contract below a certain level of detail. But in general terms, this proposal comes from a hotel that we have met in multiple times, had successful meetings, and as far as we know have met the objectives Ray outlined. We have list of places we have met in in which that wasn't true for one reason or another; we also have a set of locations that have worked better than the average, and done so on multiple occasions. Some of these are in Asia, some are in Europe, and some are in North America. Of probable interest to you: one of the sites I think mostly works is in Prague.

What I am suggesting to the IAOC is that, over the coming 9 years (27 meetings), we meet 9 times in Asia (and maybe that includes ANZ), 9 times in Europe (and maybe that includes Africa), and 9 times in the Americas. Of those, I am suggesting that we meet 3 of the 9 Asian times in a particular hotel that has worked well for us in that part of the world, 6 of the 9 European times in two hotels that have worked well for us in Europe, and in 9 of the 9 "Americas" times, meet in 3 hotels that have worked well for us in the past in the US and Canada. Our world tour would begin to have aspects of a rotation. For that to happen, I am suggesting that we ask these specific locations whether they, too, would be interested in a multi-meeting contract, and to propose terms for such meetings.

Folks from Latin America (e.g., South and Central, generally spanish-speaking and portuguese-speaking) will object on the grounds that they would like to be included in the rotation. I can respond to that in a couple of ways, one of which is that I honestly don't expect to get proposals for 3 meetings in 9 years from each of the 3 North American hotels on my little list. Also, we can probably expect a little flexibility in contracting that would allow us to insert a Latin American location by moving one of the venues out a little bit. I think the problem is solvable.

What this does is give us a set of locations, for as many as 18 of the coming 27 meetings, that we know work for the IETF and its purposes, because they have in the past. It also gives us at least 9 of the coming 27 meetings in which we can explore locations such as you advocate.

What will be the problems with placing those meetings? North America is frankly not too hard. Europe takes a little more effort, especially in finding a suitable host. Asia/ANZ - we put a lot of effort into that. The locations that can offer us the number of bedrooms and breakout rooms we need, can honestly discuss having 1500 people walk out of a meeting at 11:30 and return by 13:00, and are near major hub or regional airports in Asia is a little thin, and where we find them, they are expensive.

Let me ask, since you clearly have opinions on such matters - what would you think of such an arrangement? What am I missing in such a proposal?

---

From: Kathleen Moriarty <[email-masked]>
Date: Fri, 18 Dec 2015 16:45:59 -0500

Sent from my iPhone

> On Dec 18, 2015, at 4:31 PM, Fred Baker (fred) <[email-masked]> wrote:
> 

I think the poll or request for input should be broader than the IETF list.  Jari asked how many people follow this list at a plenary and the percentage seemed pretty low.  I suspect that's part of the difference from the meeting surveys to the discussions that take place here.  I know I'm stating the obvious.

Thanks,
Kathleen

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 18 Dec 2015 22:59:08 +0100

> Let me ask, since you clearly have opinions on such matters - what would you think of such an arrangement? What am I missing in such a proposal?

You addressed this message specifically to Michal.

But being a foreigner to most people on this list :-), I think I can
answer this as well.

Great plan!

There is additional value in repeat performances that is rarely being
discussed here.  And going to places that we know will work takes a lot
of uncertainty out of the planning, uncertainty that only generates more
uncertainty (will X come to the meeting in this strange place?  Is it
even worth having a meeting if X isn't there and Y will have visa
problems ...).

So please do go ahead with that plan.  (And please endure the kvetching
-- the majority on this will be a silent one.)

Grüße, Carsten

PS.: And, yes, the Prague meetings have been great repeat performances.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 19 Dec 2015 11:15:53 +1300

> But being a foreigner to most people on this list :-), I think I can

Not just a foreigner, but also an alien, in the US, just like me.

> Great plan!

I agree. We had a relatively good result with a previous multi-meeting
contract, which provided some leverage with a whole hotel chain.

Whatever we do will be wrong for some people; the IAOC has to make
its best guess knowing this. I had a boss once who defined his job
as "making the best possible distribution of unhappiness."

   Brian

---

From: Michal Krsek <[email-masked]>
Date: Fri, 18 Dec 2015 23:32:36 +0100

Hi Fred,
thank you for sharing the idea. Thank you for being polite to Prague 
(having ietf to Prague needs extra effort from my friends and me, so it 
is not personal benefit, you can imagine :-) ).

Your proposal seems to be fine to me, but if we set too many rules, we 
may find ourselves in the trap of numbers/rules (like distance from 
overflow hotel to primary hotel has to not be considered same - weather, 
safety, taxi availability, public transport has to be considered, and I 
believe they are). Looking for a good South/Latin America place is great.

One generic rule should be no more than new venue per year (two 
incumbent locations).

I'm not too in the hotel business, but first time in PRG everything just 
worked if scouting team came with Hilton representative from US. Before 
that happened, they almost overlooked us trying to get in with the 
fiber. After this visit, everything worked smoothly. So may the contract 
be with the hotel chain (instead of specific hotel) - if this works?

I see and really appreciate the work IAOC and AMS and volunteers put 
into meeting organization. I'm helping here in Prague and it is hard, 
and there are still some cultural misunderstandings.

I'm little bit disappointed by seeing notorious complaints about "we 
can't fit into the same hotel" or "walking distance is too long" or 
"hotel is expensive" or "I need a visa to visit that country" ...

BTW my dream is we have a kind of immersive remote participation, we are 
not there yet (in our laboratory environment, too).

             With kind regards
                     Michal Krsek

P.S: From my personal experience - it is great if there is a group of 
local volunteers helping (seemed to me that japanese team did very good 
work).

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 18 Dec 2015 17:51:29 -0500

Fred,

I very much like your suggestion. We’ve repeated at hotels before
(Minneapolis Hilton, Omni Shoreham in DC, Westin Bayshore in Vancouver,
Hilton Metropole in London, and of course the Prague Hilton all immediately
come to mind) and it usually seems to work well when we return to a known
venue.

Michal mentions working with a hotel chain rather than particular hotels. I
recall that the IETF partnered with Hilton for a period, and as a
participant that seemed to work very well.

Cheers,
Andy

On Fri, Dec 18, 2015 at 5:32 PM, Michal Krsek <[email-masked]> wrote:

> (having ietf to Prague needs extra effort from my friends and me, so it is
> not personal benefit, you can imagine :-) ).
>
> Your proposal seems to be fine to me, but if we set too many rules, we may
> find ourselves in the trap of numbers/rules (like distance from overflow
> hotel to primary hotel has to not be considered same - weather, safety,
> taxi availability, public transport has to be considered, and I believe
> they are). Looking for a good South/Latin America place is great.
>
> One generic rule should be no more than new venue per year (two incumbent
> locations).
>
> that happened, they almost overlooked us trying to get in with the fiber.
> After this visit, everything worked smoothly. So may the contract be with
> the hotel chain (instead of specific hotel) - if this works?
>
> I see and really appreciate the work IAOC and AMS and volunteers put into
> meeting organization. I'm helping here in Prague and it is hard, and there
> are still some cultural misunderstandings.
>
> I'm little bit disappointed by seeing notorious complaints about "we can't
> fit into the same hotel" or "walking distance is too long" or "hotel is
> expensive" or "I need a visa to visit that country" ...
>
>> Let me ask a question. I'm on the IAOC Meetings committee, which is an
>> advisory committee that does some research (with AMS) and makes a
>> recommendation to Ray, which he then takes to the IAOC. The IAOC sometimes
>> agrees with us and sometimes doesn't. You will have just seen a note from
>> Ray on this mailer detailing the IAOC's objectives in meeting planning; our
>> committee, with strong involvement from AMS, does the investigative legwork
>> to try to achieve those.
>>
>> Right now, I am suggesting a model to Ray, based on a proposal that we
>> have seen that would build a multi-meeting contract with a certain hotel.
>> As with most business, matters, it would be inappropriate for me to discuss
>> a contract below a certain level of detail. But in general terms, this
>> proposal comes from a hotel that we have met in multiple times, had
>> successful meetings, and as far as we know have met the objectives Ray
>> outlined. We have list of places we have met in in which that wasn't true
>> for one reason or another; we also have a set of locations that have worked
>> better than the average, and done so on multiple occasions. Some of these
>> are in Asia, some are in Europe, and some are in North America. Of probable
>> interest to you: one of the sites I think mostly works is in Prague.
>>
>> What I am suggesting to the IAOC is that, over the coming 9 years (27
>> meetings), we meet 9 times in Asia (and maybe that includes ANZ), 9 times
>> in Europe (and maybe that includes Africa), and 9 times in the Americas. Of
>> those, I am suggesting that we meet 3 of the 9 Asian times in a particular
>> hotel that has worked well for us in that part of the world, 6 of the 9
>> European times in two hotels that have worked well for us in Europe, and in
>> 9 of the 9 "Americas" times, meet in 3 hotels that have worked well for us
>> in the past in the US and Canada. Our world tour would begin to have
>> aspects of a rotation. For that to happen, I am suggesting that we ask
>> these specific locations whether they, too, would be interested in a
>> multi-meeting contract, and to propose terms for such meetings.
>>
>> Folks from Latin America (e.g., South and Central, generally
>> spanish-speaking and portuguese-speaking) will object on the grounds that
>> they would like to be included in the rotation. I can respond to that in a
>> couple of ways, one of which is that I honestly don't expect to get
>> proposals for 3 meetings in 9 years from each of the 3 North American
>> hotels on my little list. Also, we can probably expect a little flexibility
>> in contracting that would allow us to insert a Latin American location by
>> moving one of the venues out a little bit. I think the problem is solvable.
>>
>> What this does is give us a set of locations, for as many as 18 of the
>> coming 27 meetings, that we know work for the IETF and its purposes,
>> because they have in the past. It also gives us at least 9 of the coming 27
>> meetings in which we can explore locations such as you advocate.
>>
>> What will be the problems with placing those meetings? North America is
>> frankly not too hard. Europe takes a little more effort, especially in
>> finding a suitable host. Asia/ANZ - we put a lot of effort into that. The
>> locations that can offer us the number of bedrooms and breakout rooms we
>> need, can honestly discuss having 1500 people walk out of a meeting at
>> 11:30 and return by 13:00, and are near major hub or regional airports in
>> Asia is a little thin, and where we find them, they are expensive.
>>
>> Let me ask, since you clearly have opinions on such matters - what would
>> you think of such an arrangement? What am I missing in such a proposal?
>>
>
>

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 18 Dec 2015 18:36:45 -0500

Hi,

This is my personal opinion, and it's one that states the kinds of
things I'd want to think about rather than something that reflects my
opinion.  I mostly don't have an opinion yet; and I think it's just as
well to have this sort of discussion in public, so that's why I'm
replying.

On Fri, Dec 18, 2015 at 09:31:06PM +0000, Fred Baker (fred) wrote:
> 
> What I am suggesting to the IAOC is that, over the coming 9 years (27 meetings)

One thing that worries me a little bit in discussing that kind of plan
is how far out we're talking about, and how much it assumes that the
IETF meeting system and pace of meetings will not change in that time.

The IETF is changing and is working in an environment which is itself
changing a lot.  We are seeing quite a lot more remote participation,
and there's little reason to suppose that will slow down.  The
standards environment is changing a lot, too.  Many standards are now
developed basically as online repositories of code and standard at the
same time.  The complaint that the IETF moves too slowly and too
bureaucratically is heard more often than ever.  We are having more
interims (most of them virtual).  And our population is changing: we
are having new people come to the IETF and, of course, some people are
retiring.  In 9 years, will we still be meeting three times a year?
Will the 1-1-1* approach still apply?  (Perhaps the vast majority of
IETF participants will be coming from India and China, for instance,
in which case North America might not be a good regular destination.)

Moreover, 9 years is a _really really_ long time past the term of
anyone appointed by nomcom.  Three years was already past the nomcom
horizon, but at least there is an argument that people on the IAOC are
likely to come to at least some meetings the year after their term
ends.  I confess I have no way of forecasting whether I'll still be
participating in the IETF in 9 years (given my track record on
predicting my own career, frankly, it's astonishing to me when I even
have a clue about next week).  My first IETF meeting (in person) was
10 years and one month ago, and if you'd asked me then whether I'd
still be participating after 10 years I couldn't have guessed.  

Finally, there's been a great deal of worry about the lack of
cross-area review that we get these days, but it doesn't seem to me
that the meetings obviously help with that except by accident.  It
could be that we will conclude that three giant meetings a year (as
opposed to, say, area meetings or something like that plus, say, one
all-IETF meeting a year) is a bad idea because of the practical
constraints.

The above are each topics where I have poor knowledge, and where that
knowledge would make a difference to how I'd respond to an idea like
this.  (They're probably not all the topics, but ones that I can think
of now.)  I just don't know what to think about this.  I'd be
interested to hear views, especially including those who are perhaps
less-frequent contributors to this list, people who've started
participating in the IETF in recent years (both at meetings and also
not at meetings or only infrequently at meetings), and those who've
been arguing that we need to do something to make the meetings less
important and return the important work to lists and other such ways
of working.

I solicit your advice (either on-list or off).

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 19 Dec 2015 13:02:44 +1300

On 19/12/2015 12:36, Andrew Sullivan wrote:
<snip>
> cross-area review that we get these days,...

I'd want objective evidence that we get less cross-area review than
at some date in the past before worrying a great deal. It seems to me
that we are actually doing it more systematically than in the past.

>...  but it doesn't seem to me
> that the meetings obviously help with that except by accident.

But those accidents - hearing about something by chance that you would
never discover by chance on-line - are a major part of the advantage of
our plenary weeks. Being all in one place and time zone allows those
useful accidents to happen.

   Brian

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 18 Dec 2015 19:07:43 -0500

> that we are actually doing it more systematically than in the past.

I am prepared to believe that we are in fact doing it more
systematically.  If so, however, the argument you make for the
kismet-review effect of full IETF meetings is actually weaker, since
the system could be set up to encourage things via lists and so on.

> useful accidents to happen.

I agree with this, but like most cases it's a cost-benefit trade, and
we do rather need to keep in mind the very cost we've just been
talking about at length.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Wassim Haddad <[email-masked]>
Date: Sat, 19 Dec 2015 02:25:33 +0000

+1

Wassim H.

Sent from my iPad

> On Dec 18, 2015, at 14:52, Andrew G. Malis <[email-masked]> wrote:
> 
> Fred,
> 
> I very much like your suggestion. We’ve repeated at hotels before (Minneapolis Hilton, Omni Shoreham in DC, Westin Bayshore in Vancouver, Hilton Metropole in London, and of course the Prague Hilton all immediately come to mind) and it usually seems to work well when we return to a known venue.
> 
> Michal mentions working with a hotel chain rather than particular hotels. I recall that the IETF partnered with Hilton for a period, and as a participant that seemed to work very well.
> 
> Cheers,
> Andy
> 
> 
>> thank you for sharing the idea. Thank you for being polite to Prague (having ietf to Prague needs extra effort from my friends and me, so it is not personal benefit, you can imagine :-) ).
>> 
>> Your proposal seems to be fine to me, but if we set too many rules, we may find ourselves in the trap of numbers/rules (like distance from overflow hotel to primary hotel has to not be considered same - weather, safety, taxi availability, public transport has to be considered, and I believe they are). Looking for a good South/Latin America place is great.
>> 
>> One generic rule should be no more than new venue per year (two incumbent locations).
>> 
>> I'm not too in the hotel business, but first time in PRG everything just worked if scouting team came with Hilton representative from US. Before that happened, they almost overlooked us trying to get in with the fiber. After this visit, everything worked smoothly. So may the contract be with the hotel chain (instead of specific hotel) - if this works?
>> 
>> I see and really appreciate the work IAOC and AMS and volunteers put into meeting organization. I'm helping here in Prague and it is hard, and there are still some cultural misunderstandings.
>> 
>> I'm little bit disappointed by seeing notorious complaints about "we can't fit into the same hotel" or "walking distance is too long" or "hotel is expensive" or "I need a visa to visit that country" ...
>> 
>> BTW my dream is we have a kind of immersive remote participation, we are not there yet (in our laboratory environment, too).
>> 
>>             With kind regards
>>                     Michal Krsek
>> 
>> P.S: From my personal experience - it is great if there is a group of local volunteers helping (seemed to me that japanese team did very good work).
>> 
>>

---

From: Richard Shockey <[email-masked]>
Date: Sat, 19 Dec 2015 08:57:26 -0500

+1  

Vancouver or anywhere in Canada Forever. I appreciate the problem with US visas.  Though I must say most of us believed the Minneapolis Hilton actually had the optimal lay out of meeting rooms and no management type one ever believed it was a junket.  Berlin or Prague on the short list as well.

I’ve already made my decision NOT to go to Buenos Aries.

---

From: tom p. <[email-masked]>
Date: Sat, 19 Dec 2015 15:15:27 +0000

----- Original Message -----
From: "Richard Shockey" <[email-masked]>
To: "Fred Baker (fred)" <[email-masked]>; "Michal Krsek"
<[email-masked]>
Cc: <[email-masked]>
Sent: Saturday, December 19, 2015 1:57 PM

+1

Vancouver or anywhere in Canada Forever. I appreciate the problem with
US visas.

<tp>

I think that visas is the other factor, beside getting there and having
the facilities to get the work done, that keeps on surfacing(and appears
at the end of item 9 in Ray's recent list). As has already been said, in
these threads, the IAOC will not be able to acquire the fine detail of
what is required for participants from 192 countries to visit all the
places we go (just as the IAOC is unlikely to have much success in
persuading places where we would like to go to build bigger hotels with
adequate facilities for us).  But if the IAOC focussed on making it
simpler, more straightforward for people to get into the USA, then I
think there would be a case for holding more meetings in the USA, which
solves the problems of the lack of facilities since the USA is
well-endowed.

For myself, from Europe, I much prefer travelling to Mid to Eastern USA
as opposed to continental Europe or Asia but do see those e-mails from
people in places we would like to encourage about the difficulties they
encounter in understanding what they need to do.

At a slight tangent, I read this morning, in another context, of someone
being denied access to the USA because while they had the relevant
Electronic replacement for the visa waiver, they were expected to have a
printed copy (silly me, thinking that electronic means electronic; memo
to self - always print the electronic document and let's fell a few
trees).  It is that sort of detail - print it out, just in case - that
we can make a difference and is an area where the IAOC could make that
difference.

Tom Petch

Though I must say most of us believed the Minneapolis Hilton actually
had the optimal lay out of meeting rooms and no management type one ever
believed it was a junket.  Berlin or Prague on the short list as well.

advisory committee that does some research (with AMS) and makes a
recommendation to Ray, which he then takes to the IAOC. The IAOC
sometimes agrees with us and sometimes doesn't. You will have just seen
a note from Ray on this mailer detailing the IAOC's objectives in
meeting planning; our committee, with strong involvement from AMS, does
the investigative legwork to try to achieve those.
>Right now, I am suggesting a model to Ray, based on a proposal that we
have seen that would build a multi-meeting contract with a certain
hotel. As with most business, matters, it would be inappropriate for me
to discuss a contract below a certain level of detail. But in general
terms, this proposal comes from a hotel that we have met in multiple
times, had successful meetings, and as far as we know have met the
objectives Ray outlined. We have list of places we have met in in which
that wasn't true for one reason or another; we also have a set of
locations that have worked better than the average, and done so on
multiple occasions. Some of these are in Asia, some are in Europe, and
some are in North America. Of probable interest to you: one of the sites
I think mostly works is in Prague.
>What I am suggesting to the IAOC is that, over the coming 9 years (27
meetings), we meet 9 times in Asia (and maybe that includes ANZ), 9
times in Europe (and maybe that includes Africa), and 9 times in the
Americas. Of those, I am suggesting that we meet 3 of the 9 Asian times
in a particular hotel that has worked well for us in that part of the
world, 6 of the 9 European times in two hotels that have worked well for
us in Europe, and in 9 of the 9 "Americas" times, meet in 3 hotels that
have worked well for us in the past in the US and Canada. Our world tour
would begin to have aspects of a rotation. For that to happen, I am
suggesting that we ask these specific locations whether they, too, would
be interested in a multi-meeting contract, and to propose terms for such
meetings.
>Folks from Latin America (e.g., South and Central, generally
spanish-speaking and portuguese-speaking) will object on the grounds
that they would like to be included in the rotation. I can respond to
that in a couple of ways, one of which is that I honestly don't expect
to get proposals for 3 meetings in 9 years from each of the 3 North
American hotels on my little list. Also, we can probably expect a little
flexibility in contracting that would allow us to insert a Latin
American location by moving one of the venues out a little bit. I think
the problem is solvable.
coming 27 meetings, that we know work for the IETF and its purposes,
because they have in the past. It also gives us at least 9 of the coming
27 meetings in which we can explore locations such as you advocate.
frankly not too hard. Europe takes a little more effort, especially in
finding a suitable host. Asia/ANZ - we put a lot of effort into that.
The locations that can offer us the number of bedrooms and breakout
rooms we need, can honestly discuss having 1500 people walk out of a
meeting at 11:30 and return by 13:00, and are near major hub or regional
airports in Asia is a little thin, and where we find them, they are
expensive.
>
>Let me ask, since you clearly have opinions on such matters - what
would you think of such an arrangement? What am I missing in such a
proposal?

---

From: Jari Arkko <[email-masked]>
Date: Sat, 19 Dec 2015 20:17:19 +0100

I think that is a very good model, thank you Fred. Although I
too have some of the questions that Andrew had about
predictability of the future far out. My personal opinion
is that IETF-wide meeting continue to be a component
of the overall system for various reasons even with any
further arrangements that we might develop for cross-area
and other things. But their nature and size will most likely
change in 10 years :-)

Jari

---

From: "John Levine" <[email-masked]>
Date: 19 Dec 2015 19:33:56 -0000

>Let me ask, since you clearly have opinions on such matters - what would you think of such
>an arrangement? What am I missing in such a proposal?

I think it would be a fine idea to return to venues that have worked
well in the past.  On the other hand, in nine years I will be 70 years
old and it is not clear to me how involved I will be in the IETF in
2024.

We do need to think about outreach and bringing in new people, but
given the way the IETF works, where meetings are almost entirely
working sessions, people at the sessions are expected to have done
their homework and know something about the topic, and tutorials in WG
meetings are generally considered to be a poor use of limited time, it
doesn't seem to me that the physical meetings are a particularly good
way to bring in people who aren't already reading the mailing lists.

R's,
John

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 19 Dec 2015 20:20:59 +0000

> would you think of such an arrangement? What am I missing in such a

Returning to known-good venues is a good plan. Ones like Berlin
and Prague are especially good as they cater well for those who
hardly leave the venue all week as well as for those who are
involved only in specific things that allow them more free time.
(And that latter group is hugely important as I'd bet almost all
of the former group started out in the latter group.)

9 years is too far ahead to know for sure that we'll have the
same kind of 3 x 1400-people meetings per year. I'd be surprised
if that's the case in 2024 to be honest, but it could be. I'm
not sure how best to handle that but I'd be against an arrangement
that forced the IETF to keep doing the same kind of meetings that
far ahead. If there's an acceptable way to get changes agreed
every 2 years or so that'd be ok though. I'm fine that the IAOC
figure out what they consider acceptable, but would prefer that
as much as possible of that be visible to the community before
final agreements are reached.

S.

---

From: Jari Arkko <[email-masked]>
Date: Sun, 20 Dec 2015 09:45:46 +0100

> that we are actually doing it more systematically than in the past.

I think we do review in generally more systematically than in the past.
And bulk of that happens outside meetings. But some of it
also in meetings, e.g., BOF discussions.

And in my totally unscientific gut feeling, we have more review of
each output document from the IETF than in the past. Of course,
the Internet has also changed, and each of us knows much less about
everything than we did in the past. And the people have changed
too. It would be difficult to measure quality, except maybe by
observing IETF outputs actually work and be deployed in
the marketplace. (I think on that metric we’re doing broadly
fine, but of course the results vary great deal. But again, just
personal feelings rather than objective measurements.)

> useful accidents to happen.

I doubt that most people have "helping review other work" very
high on their list of motivations when the decide to attend an
event in any organisation. In the interest of providing even
more gut feeling-based data to this thread, I suspect  the
four top reasons are 1) driving their main own thing,
2) finding interesting new topics or collaboration that
they or their company should work on in the industry,
3) general learning of what is going on and 4) side
meetings. However, the three last reasons do lead to
helping review other work in many cases, so...

Jari

---

From: John C Klensin <[email-masked]>
Date: Sun, 20 Dec 2015 05:43:43 -0500

--On Friday, December 18, 2015 21:31 +0000 "Fred Baker (fred)"
<[email-masked]> wrote:

> Let me ask a question. I'm on the IAOC Meetings committee,
> which is an advisory committee that does some research (with
> AMS) and makes a recommendation to Ray, which he then takes to
> the IAOC. The IAOC sometimes agrees with us and sometimes
> doesn't. You will have just seen a note from Ray on this
> mailer detailing the IAOC's objectives in meeting planning;
> our committee, with strong involvement from AMS, does the
> investigative legwork to try to achieve those.

Fred,

I'm in general agreement with Andrew and several other comments,
but want to reinforce two of them.

First, we seem to be seeing an increase in virtual interim
meetings even in the last four or five years.   I don't know
whether that marks a change in the way the IETF does business
and the people who are attending (although I suspect it does)
nor do I know if there is any correlation between f2f plenary
meetings that are far away or expensive for some key
participants in some WGs and that increase or if it is due
entirely to other factors.  There are also considerable efforts
to make remote participation (as distinct from remote lurking)
more plausible.  To the extent to which they succeed, it may
have a side-effect of reducing the odds that someone who only
participates in one or two WGs will travel to the IETF meeting
site. 

Quite independent of factors we cannot predict [1], those
factors suggest a very strong possibility of major changes in
just a few years, much less than nine or more.

Second, I've heard a great deal of unrest about claims of
inability to make changes in meeting arrangements because those
arrangements are already set three years out.  Sometimes that
even seems to take the form of "we can't do that for the next
meeting, N+1, because it is within the three year window and we
can't do it for meeting N+10 because it it too far away to think
about".  Creating that situation for 27 meetings into the
future, with or without the sort of changes suggested by the
above, seems to be a recipe for the sense of an unresponsive and
unaccountable system that upsets the IETF community (whether
there is a substantive problem or not).

Part of that problem is a sense that the IAOC (and the meetings
committee) have been less than forthcoming with the community
about what is going on and what decisions are being made and
why.  Ray's note lists multiple criteria that we all know tend
to conflict but does not identify priorities.  That is
reasonable in some ways, problematic in others, especially if
one believes those decisions should be made by the community
rather than a small cluster of people in secret.  At the other
ends of the process, BCP 101 appears to require that contracts
(with no exclusion for hotel contracts) be posted, albeit with
commercially-sensitive details redacted.   My recollection is
that we were promised for some years that would be done, but it
hasn't been... and the hotel category doesn't even appear on the
IAOC contracts page.

It seems to me that your idea improves on that situation in some
ways but also vastly increases the potential for a plausible
perception of abuse (or at least a completely non-transparent
process).

So, while I think that stabilizing on a small number of
locations would be a good idea, especially if we can get more
favorable arrangements as a result, I think nine years is much
too long... and that even three years may be too long unless it
comes with a plan from the IAOC about how to make the whole
system more obviously transparent and responsive.

    john

[1] On the "unpredictable" scale, I'd hate to be the IAOC if
there were another significant industry crash that brought about
very significant pressures on travel budgets and a significant
drop in attendance for several years if one had firm hotel
contracts with large block commitments many years out.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 20 Dec 2015 09:56:57 -0800

> the system could be set up to encourage things via lists and so on.

Well...  That depends on how reviews are done and how the 
kismet-contacts affect that.

Massive numbers and types of problems get missed by the reviews that are 
currently done.  That's not because people don't care or are 
insufficiently knowledgeable.  I'm not sure what could reliably done to 
reliably improve the catching of serious problems, but currently we are 
reliably missing lots of stuff, lots of times.

Kismet contacts increase the range of brains and eyes looking at stuff. 
  In statistical terms, that makes it more likely that someone will 
catch one or another significant issue.  Who and when and what aren't 
predictable, but the overall odds get better.

d/

---

From: Randall Gellens <[email-masked]>
Date: Sun, 20 Dec 2015 16:21:49 -0800

At 5:05 PM +0000 12/16/15, Ted Lemon wrote:

>  ultimately each decision made sense when I understood the factors 
> that were considered.  Well, except maybe Hawaii

What was the problem with Hawaii?  The hotel had plenty of rooms, and 
it was the only welcome reception where I could hear people and 
didn't get hoarse from shouting (because it was outdoors rather than 
in a confined space).

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
Good night to spend with family, but avoid arguments with your
mate's new lover.

---

From: Ralph Droms <[email-masked]>
Date: Mon, 21 Dec 2015 13:36:26 -0500

> On Dec 20, 2015, at 12:56 PM 12/20/15, Dave Crocker <[email-masked]> wrote:
> 
> Well...  That depends on how reviews are done and how the kismet-contacts affect that.
> 
> Massive numbers and types of problems get missed by the reviews that are currently done.

Can you say more here?  How do you know these problems are being missed?  When you write "missed by the reviews that are currently done", do you mean missed altogether or missed by, say, the WG review and picked up in IESG review?

Given the number of problems I see in some documents during Int-Dir or Gen-ART, I'm sure there are more problems that I'm not picking up.  It's a reasonable inference that there are problems not caught by any of our reviews.  I'm curious about specifics...

- Ralph

>  That's not because people don't care or are insufficiently knowledgeable.  I'm not sure what could reliably done to reliably improve the catching of serious problems, but currently we are reliably missing lots of stuff, lots of times.
> 
> Kismet contacts increase the range of brains and eyes looking at stuff.  In statistical terms, that makes it more likely that someone will catch one or another significant issue.  Who and when and what aren't predictable, but the overall odds get better.
> 
> d/
> 
> --
> Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Mon, 21 Dec 2015 16:39:23 -0800

Ralph,

On 12/21/2015 10:36 AM, Ralph Droms wrote:
> Can you say more here?  How do you know these problems are being
> missed?  When you write "missed by the reviews that are currently
> done", do you mean missed altogether or missed by, say, the WG review
> and picked up in IESG review?

      1. We have errata.

         We wouldn't have them if published RFCs did not /regularly/ 
have problems.  So for all the reviewing we do, we still choose to have 
an institutional mechanism for recording errors in the published specs.

         And note that the current rather odd policy for accepting 
errata is that they can only mean "the rfc didn't properly state was 
intended."  That is, if the wg intended the wrong thing and people now 
understand what the right thing is, that is not allowed to be recorded 
as an errata. So the only errata are straight-up documentation errors.

      2. I've directly seen published RFCs that had gaping errors

         (and I don't have special specification-reading skills, nor are 
the specs I read all that special.)

         By way of example, the original DKIM RFC was massively reviewed 
and implemented and yet the explicit specification of algorithms were 
wholly deficient, while the accompanying prose was pretty good.  Nobody 
had ever done a careful correlation audit.

      3. We have no discipline to reviews and no documentation to give 
careful guidance.

         Reviews are a random walk of well-intentioned people who have 
varying skills, varying attention-spans, etc.  The results are, 
therefore, random.

In other words, it is merely a certainty that serious problems will slip 
through.  Every review is good to do, but no specific review is the 
savior of us all.  (The ultimate fallacy of a savior model is requiring 
expenditure of scarce, strategic area director resources on late-stage 
reviews...  Expensive resources, frustrating delays, minimal benefits. 
Who wouldn't want all that?)

The main benefit of a cross-area review is that it is a cold reading by 
someone with no history with the effort.  Fresh eyes.

> Given the number of problems I see in some documents during Int-Dir
> or Gen-ART, I'm sure there are more problems that I'm not picking up.

Guaranteed.

> It's a reasonable inference that there are problems not caught by any
> of our reviews.  I'm curious about specifics...

Before going from Proposed to Full standard, we expect some breadth of 
community deployment.  That includes the expectation that fresh eyes of 
fresh implementers have tried to take the spec and build something 
interoperable.  We don't actually require this.  But at least it's 
implicit as an evaluation criterion of maturity.

We could move that requirement to Proposed, but that would merely make 
Proposed essentially the same as Full.  And the barrier to Proposed is 
already too high.

Instead we should understand that we cannot and should not try to demand 
or expect documents that are perfect.  We should demand 'good enough' 
and let the outside world evaluate and feed the results back to us.

The way they already do.

The problem is our mythology before that and the misguided expectations 
and barriers created as a result.

d/

---

From: John C Klensin <[email-masked]>
Date: Mon, 21 Dec 2015 20:59:27 -0500

One small addition...

--On Monday, December 21, 2015 16:39 -0800 Dave Crocker
<[email-masked]> wrote:

>       3. We have no discipline to reviews and no documentation
> to give careful guidance.
> 
>          Reviews are a random walk of well-intentioned people
> who have varying skills, varying attention-spans, etc.  The
> results are, therefore, random.

Perhaps worse, some of the area reviews are not random but are
from people who, when they are engaging in full disclosure,
admit that they didn't understand the spec and therefore
reviewed what they did understand and maybe engaged in a bit of
nit-picking to show that they really tried to read it.  I've
occasionally seen almost exactly the same behavior from a few
ADs.  Maybe more than "occasionally" depending on how critically
one evaluates the reviews.

Beyond a certain point, those types of reviews make us feel
good, but may make a negative contribution to document quality
by convincing others, who might have less time (or not draw the
short straw) and/or better expertise and insight to conclude
that they don't need to read the document because lots of others
are doing so.

> In other words, it is merely a certainty that serious problems
> will slip through.  Every review is good to do, but no
> specific review is the savior of us all.  (The ultimate
> fallacy of a savior model is requiring expenditure of scarce,
> strategic area director resources on late-stage reviews...
> Expensive resources, frustrating delays, minimal benefits. Who
> wouldn't want all that?)

Exactly.

> The main benefit of a cross-area review is that it is a cold
> reading by someone with no history with the effort.  Fresh
> eyes.

I would have said "no history with the effort and at least some
substantive knowledge and interest in the subject matter".   "I
don't have a clue but was designated as the Foobar Area
Reviewer" reviews are usually not very helpful (although
sometimes we get lucky).

>...
> We could move that requirement to Proposed, but that would
> merely make Proposed essentially the same as Full.  And the
> barrier to Proposed is already too high.
> 
> Instead we should understand that we cannot and should not try
> to demand or expect documents that are perfect.  We should
> demand 'good enough' and let the outside world evaluate and
> feed the results back to us.

I'm not certain we are getting to "good enough" quite often
enough.  Moreover, what was "good enough" when the expectation
was that people would not deploy Proposed Standards in products,
at least without understanding that was a risk and treating it
as such, may not be "good enough" when Proposed Standards are
not only deployed but the community's attitude seems to be that
a piece of specification that turns out to be a bad idea has to
be truly catastrophic to modify the spec in an incompatible way,

>...
> The problem is our mythology before that and the misguided
> expectations and barriers created as a result.

Very likely... at least a major part of the problem.

    john

---

From: Dave Crocker <[email-masked]>
Date: Mon, 21 Dec 2015 20:52:56 -0800

> not only deployed but the community's attitude seems to be that

This nicely summarizes a common bit of mythology in the IETF.

First it presumes that folk out there in develop-and-deploy land have no 
ability to assess what they are developing and deploying.

Second is that it presumes that there has been some sort of major change 
in the way IETF specs are processed pre- and post- Proposed status 
assignment.

Both are fundamentally wrong.

The IETF is a collaborative community venture, not a grand parental 
oversight commission.  Folk out their in product-land have been able to 
deal with immature, flakey and changing IETF specs productively for more 
than 25 years.

d/

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Dec 2015 00:36:50 -0500

--On Monday, December 21, 2015 20:52 -0800 Dave Crocker
<[email-masked]> wrote:

>> was that people would not deploy Proposed Standards in
>> products, at least without understanding that was a risk and
>> treating it as such, may not be "good enough" when Proposed
>> Standards are not only deployed but the community's attitude
>> seems to be that
> 
> 
> First it presumes that folk out there in develop-and-deploy
> land have no ability to assess what they are developing and
> deploying.
> 
> Second is that it presumes that there has been some sort of
> major change in the way IETF specs are processed pre- and
> post- Proposed status assignment.
> 
> The IETF is a collaborative community venture, not a grand
> parental oversight commission.  Folk out their in product-land
> have been able to deal with immature, flakey and changing IETF
> specs productively for more than 25 years.

Dave, I can (and do) agree with the comment in your last
paragraph without agreeing with the previous two (although we
could debate what changes are "major" or what "fundamentally"
means). 

Perhaps I was just unclear because I wasn't talking about "the
way IETF specs are processed" but about how IETF specs at
Proposed Standard are interpreted in the community and how the
IETF responds to those interpretations. 

I'm personally very sensitive to the change in the latter case
because, when I first took over as an Apps AD (you might recall
it was a bit mid-term -- information that is long enough ago to
be irrelevant to most others), the very first task I had to deal
with involved a major vendor who had deployed a product based on
a either a very late I-D or a proposed standard spec (don't
remember; could go look it up) after which the IETF changed its
mind, decided the original spec was a bad idea, and specified
something different and very incompatible with that original
idea.  They felt that their having deployed the original spec
should require the IETF to stick with it, bad idea or not.  I
got to explain the IETF's conclusion that the original was a bad
idea even though they had gotten it to work, tell them that the
situation was their tough luck for deploying at scale that
early, and that, unless they made changes, they simply were not
going to interoperate with anyone who did follow the spec.

For better or worse, I have a lot of trouble believing that
scenario could play out the same way today.  Again, whether that
is a major change or not could be debated endlessly, but it is a
change.

    best,
     john

---

From: Jari Arkko <[email-masked]>
Date: Tue, 22 Dec 2015 18:52:16 +0200

> scenario could play out the same way today.

Mumble. I’m not sure. I seem to recall many cases
where somebody has to explain to $vendor that
things have changed and they can’t do what
they wanted to do. In more cases than just
draft changes.

Results of course vary. But, ultimately it is an IETF
community decision on what a working group does.
Do they weigh the existing implementation base
or some other factors more? You or me might
disagree with some of those decisions… perhaps
on good grounds. I’m not sure the overall situation
would be better if we somehow got to override
the wg opinions. (Except of course on clear
issues that fulfil Discuss criteria.)

Jari

---

From: Jari Arkko <[email-masked]>
Date: Tue, 22 Dec 2015 19:01:39 +0200

> Instead we should understand that we cannot and should not try to demand or expect documents that are perfect.  We should demand 'good enough' and let the outside world evaluate and feed the results back to us.

One view is that we already are in that mode. I don’t think any reasonable person could claim that any specification from a standards body of any sort is perfect, including from the IETF. More interestingly, the question is whether our community, directorate, and IESG reviews and associated practices reach the ‘good enough’ level or under- or overshoot. But one person’s egregiously unnecessary fine-tuning is another person’s major threat to the Internet.

Personal opinion: we overdo it, a lot of the time.

But I think we are agreeing that we actually shoot for ‘good enough’ but do too little follow-up and revision. I could cite many counter examples where that follow-up does happen. But in many cases there is no follow-up. Why is that? Specification turned out to be uninteresting, so need to follow-up? Close enough, no business need to waste time to get to perfection? Remaining details hammered out in interops, code already runs, no need to revise? Aside from the few errata, no need for bigger changes? Worst specs are revised others are good enough? IETF process too complicated for the update? Probably a mixture of these reasons.

Jari

---

From: Eric Burger <[email-masked]>
Date: Tue, 22 Dec 2015 19:51:12 -0500

We’ve got the existence proof of the counter example in SIP. I cannot count how many times I heard, “Yes, I know foo is broken, but it has been in 2543bis-02 and now we are at 2543bis-06 and we could not possibly make a change because so many vendors implemented to bis-02.” Egad! We could not change what was in an Internet *Draft*??? Imagine the pushback to changing an RFC.

> On Dec 22, 2015, at 11:52 AM, Jari Arkko <[email-masked]> wrote:
> 
>

---

From: tom p. <[email-masked]>
Date: Wed, 23 Dec 2015 11:17:44 +0000

----- Original Message -----
From: "Brian E Carpenter" <[email-masked]>
To: "Andrew Sullivan" <[email-masked]>; <[email-masked]>
Sent: Saturday, December 19, 2015 12:02 AM

Brian

I think that we are doing more, in the shape of Directorate, Gen-ART and
such like reviews but that they may not make the I-D better.

Any one operating any sort of system will likely know that change
introduces faults which introduce change which ....   The art is to get
it right first time and when these late stage reviews introduce changes,
then I see the I-Ds getting worse, certainly in terms of the coherence
and consistency of the language, probably in terms of technical accuracy
too.  Outside people suggest new text which makes sense within their
mindset but may not in the context of what the WG has so far produced;
and by this stage, the WG's energy may be waning, so something that may
have
been corrected in a -00 or -01 gets left in in the I-D-(RFC minus one).

The best I-D, in some senses, is often the first to be produced, before
adoption by a WG, but that almost always has to be changed and the
longer that that goes on, commonly years, then the less coherent the I-D
is.  I am thinking particularly of I-Ds that are too large to be held in
one's
mind all at once (which most are).

Tom Petch

---

From: Dave Crocker <[email-masked]>
Date: Wed, 23 Dec 2015 08:33:50 -0800

On 12/22/2015 4:51 PM, Eric Burger wrote:
> We’ve got the existence proof of the counter example in SIP. I cannot count how many times I heard, “Yes, I know foo is broken, but it has been in 2543bis-02 and now we are at 2543bis-06 and we could not possibly make a change because so many vendors implemented to bis-02.” Egad! We could not change what was in an Internet*Draft*??? Imagine the pushback to changing an RFC

Eric,

Your anecdote demonstrates that some industry folk make choices entirely 
independently of how the IETF labels things.  Frankly, that 
substantiates my point.  The fact of such choices has always been with 
us.  We, and the world, cope with it.

The IETF has an obligation to provide clear documentation about the 
meaning of our labels and to assign the labels according to those 
criteria.  That means the process of producing the document and the 
process of approving publication.

We do /not/ have an obligation to anticipate people's using our 
documents in ways that go beyond that labeling.  And the fact that some 
people make such choices puts us under no obligation.

It is a fact of life that specifications get changed or thrown away.  A 
company that makes an independent -- as opposed to community-based -- 
decision to make itself dependent on something that fragile so that it 
can't cope with this sort of change has very basic problems and it is 
not the IETF's responsibility to protect them.

The IETF's fundamental decision mechanism is community rough consensus. 
  No individual or organization has veto or approval authority.  If the 
/community/ wants us to retain something in an I-D or RFC, then we must. 
  If only an individual or organization does, we don't.

And none of this dictates the details of our quality assurance process. 
  What /should/ dictate that process is cost/benefit tradeoffs, not an 
misguided, absolutist effort at perfection (which fails in any event.)

AD review appears to miss far more than it catches.  That's seems to be 
difficult for some folk to accept, and so they point to the individual 
exceptions rather than the longer pattern of misses.  (By way of some 
very strategic examples, consider IPv6 and DNSSec and email security and 
key management and...)

And AD review is extremely expensive.  ADs are second-level managers and 
reviewing is the task of an individual contributor.  Having ADs do 
late-stage reviews and having ADs spend significant time discussing 
their views and preferences about the quality of this or that 
specification is a strategic waste of valuable resources that should be 
spent doing second-level management oversight.

In direct terms it adds time and frustration to IETF processes, produces 
only random benefit while usually missing major problems, and reduces 
the population of AD candidates due to the time-bloat of the job.

d/

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 24 Dec 2015 08:05:01 +1300

>From what I've seen, mainly as a GEN-ART reviewer, *changes* are not what
happens at that stage. Corrections - yes. Clarifications to remove ambiguity
- yes. But changes of the intent of the protocol, or changes in choices that
were a matter of taste or WG preference - no.

The IESG is also explicitly supposed to avoid changes for reasons of taste
or preference:
http://www.ietf.org/iesg/statement/discuss-criteria.html#stand-undisc

> have been corrected in a -00 or -01 gets left in in the I-D-(RFC minus one).

Yes, that can happen, depending on the shepherd and authors. But it mainly
happens in the WG I believe, during the long grind to achieve a WG Last Call
with no dissenters.

> one's mind all at once (which most are).

Sure. And that is where another point comes in: the running code point.
I watched the DNCP/HNCP sausage being made. Guess what? Those are two complex
documents but most of the later changes were in direct response to cross-area
review comments *and* comments from implementors. The most powerful response
to a review comment is one based on implementation experience.

    Brian

---

From: "Pat (Patricia) Thaler" <[email-masked]>
Date: Wed, 23 Dec 2015 20:11:51 +0000

I'm not sure of all of the reasons, but:
1: there is basically nothing within walking distance. When I wanted to get out of the hotel for food I managed to find a (bad) gyro / curry wurst vendor. IETF people have repeatedly demanded many nice food places within walking distance.
2: There do not seem to be nearly enough break out rooms. A number of the IEEE meetings I attended were in guest rooms that they had taken the beds out of.
3: The food options in the hotel were quite limited. I really enjoyed the Schweinshaxe, but after the N days many people would probably get bored of it...
3: As far as I remember, their network infrastructure was, um, sad.

<pat> Actually, IEEE 802 meetings uses more break out rooms. Looking at one example, there were 24 meetings in the same slot. Because of the number of simultaneous meetings, sometimes the smallest ones meet in hotel suites.

It has been difficult to get hotels with enough meeting space outside the US so for Berlin we did sacrifice on nearby food options and ambiance (though with some exploring of the side streets I did find a decent coffee/tea place and some other cafes and bars). The hotel did have easy access to the S-Bahn, but even then one had to change to get to the heart of Berlin. <pat>

> same N locations) we wouldn't have so much kvetching.

<pat> IEEE 802 also requires the hotels to let our vendor install our network for the meeting space (but they usually don’t take over the network for the hotel sleeping rooms and our attendees aren’t as bandwidth hungry), our attendees want cookies and nearby places to eat, hotels that are reachable without a lot of extra travel time, and are sensitive about hotel price though perhaps not quite as cost sensitive as some of the IETF.

We do have a bit more willingness to go back to a place where we had a successful meeting in the past but we get kvetching too if we do it too much.

I’d rather have a hotel with enough room space that I could book a room in the hotel within a week after the bookings went live even if it meant that the nearby food options weren’t quite as good. And I don’t mind repeating successful meeting venues including Minneapolis. <pat>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 24 Dec 2015 01:46:13 +0000

> very strategic examples, consider IPv6 and DNSSec and email security and

Do you have data on this?   This doesn't match my experience.   I've found AD review to be very helpful.   It's certainly inconvenient, but I've seen AD reviews catch lots of things that were worth catching.   I would not go so far as to claim that my anecdote is more valid than yours, but I think that if we are going to make claims like this, we should probably back them up with data.   The current AD review process didn't just happen.

---

From: Eric Burger <[email-masked]>
Date: Wed, 23 Dec 2015 20:54:10 -0500

I would posit there is another dynamic I think we are missing. A lot of my academic work around Cyclomatic Complexity is about relating the complexity of code to latent defects in programs AFTER PEOPLE THINK THEY FOUND ALL OF THE BUGS.

I think the analogy here is that AD review, cross-area review, and sacrifices to Amaterasu will not find ALL of the bugs in a specification. The best we can hope for is to do best practices to keep the bugs to an acceptable, low level.

> On Dec 23, 2015, at 8:46 PM, Ted Lemon <[email-masked]> wrote:
> 
> Dave Crocker wrote:

---

From: Dave Crocker <[email-masked]>
Date: Wed, 23 Dec 2015 18:21:29 -0800

On 12/23/2015 5:46 PM, Ted Lemon wrote:
> Dave Crocker wrote:
>> AD review appears to miss far more than it catches.  That's seems
>> to be difficult for some folk to accept, and so they point to the
>> individual exceptions rather than the longer pattern of misses.
>> (By way of some very strategic examples, consider IPv6 and DNSSec
>> and email security and key management and...)
>
> Do you have data on this?

Anecdotal. Mine. Over enough years to represent a pattern. (I'm not
alone in this, but I'm reporting my own experience) In 25 years, not one
single RFC I've worked on had a serious problem caught by an AD, though
many were eventually discovered to have serious problems. Some were
delayed by large numbers of non-substantive or flat-out-wrong AD
Discusses, however. So we got significant costs with insignificant
benefits and significant damage.

Yes, there are legitimate issues caught.  Again, my point is not enough
(and by the way, not nearly soon enough.)

> This doesn't match my experience.   I've found AD review to be very
> helpful.

I find /all/ reviews "helpful".  But that marginal benefit does not
justify the strategic costs of consuming scarce resources, calendar
delay, participant frustration, etc. by imposing them at the end of a
long development cycle.

> It's certainly inconvenient,

Inconvenient is such a mild word. The aggregate effect of these kinds of
hassles is decisions by potential participants to take their
specifications elsewhere.

> but I've seen AD reviews catch lots of things that were worth
> catching.

Again, that's not the issue.  Every review by every reviewer tends to be
performed honestly and well and to provide useful feedback.  However
that, unfortunately, does not justify the particulars of AD reviews.
Please go back and consider the /balance/ of costs and benefits I cited.

The benefit of periodically finding something significant needs to be
balanced against the costs of resource consumption, significant issues
missed, delays, and participant frustration.

> I would not go so far as to claim that my anecdote is more valid than
> yours, but I think that if we are going to make claims like this, we
> should probably back them up with data.   The current AD review
> process didn't just happen.

25 years ago, the IETF was a very, very different place.  We've kept the
formal structures put in place then, without adapting to the changes.
Simply put, what we put in place doesn't scale.  We're supposed to know
something about scaling issues, but we apply none of it to
administration of the IETF...

> On 12/23/2015 5:54 PM, Eric Burger wrote:
>> I think the analogy here is that AD review, cross-area review, and
>> sacrifices to Amaterasu will not find ALL of the bugs in a
>> specification. The best we can hope for is to do best practices to
>> keep the bugs to an acceptable, low level.

The last sentence if quite reasonable, as long as it include the balance 
clause, "at an acceptable cost" and as long as we then fairly consider 
costs.

d/

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 24 Dec 2015 00:47:23 -0500

I note in your earlier description you commented that these reviews 
(whose value you doubt) take up AD time when they should be serving as 
managers.  I note that many areas now routinely turn the primary review 
over to a directorate.  The ADs decide how much careful combing they do 
after the directorate.  That seems to enable the kind of delegating that 
can help, without mandating it, and without having rules that 
micro-manage how ADs do their job.

Yours,
Joel

---

From: Marco Davids <[email-masked]>
Date: Thu, 24 Dec 2015 09:20:13 +0100

Hi,

For IETF95 I booked a room in the InterContinental BA.

On the IETF hotel-information page it says:

"At the time a reservation is made, Hotel shall charge guest one night's 
room charge and tax (non-refundable)."

However, the hotel charged my credit card for the full amount of the 
stay (which is a considerable sum of money). Isn't this unusual?

Anyone else with a similar experience?

--
Marco

---

From: Keith Moore <[email-masked]>
Date: Thu, 24 Dec 2015 10:24:39 -0500

On 12/23/2015 09:21 PM, Dave Crocker wrote:
> many were eventually discovered to have serious problems

When I was an AD, I caught several serious problems with drafts after WG 
or IETF Last Call.

The problem I saw wasn't the inability to catch things so much as the 
inability to get them fixed.    By the time a draft got past WG Last 
Call, it was generally so exhausted that it could not agree on how to 
make any significant change to a document, even in response to a 
legitimate and serious concern.   In many cases, if there was a simple 
fix, it was quickly and happily implemented.   But in far too many cases 
where there was a serious problem, the most that an IESG member could 
hope for was to get some weasel words inserted in the text or prepend an 
IESG note - neither of which really addressed the problem.

It was entirely too much work to do all of that review.   After being on 
IESG for four years, it was about ten years before I could stomach a 
detailed review of any lengthy technical document.   While on IESG I 
tried getting others to do reviews for me, but I found that I still had 
to read the documents under review to make sense of the reviews I was 
getting.   If I had to do that again today, I'd probably try to give the 
reviewers more guidance.

But mostly I think that WGs need to have their work subject to formal 
external review much earlier in the process than Last Call, particularly 
when the WG's work has the potential to impact other WGs or other areas.

Keith

---

From: "John Levine" <[email-masked]>
Date: 24 Dec 2015 23:04:37 -0000

>For IETF95 I booked a room in the InterContinental BA.

It's a little late now, but the distance info on the IETF hotel page
for the Intercontinental is wrong.  It's not 2 km from the Hilton,
it's 9 km away, about 20 mins by taxi if there's no traffic or half an
hour by subway and walking.

R's,
John

---

From: Melinda Shore <[email-masked]>
Date: Thu, 24 Dec 2015 15:28:37 -0900

On 12/24/15 2:04 PM, John Levine wrote:
> it's 9 km away [ ... ]

What's your basis for saying this?  I don't know Buenos
Aires but the maps are pretty clear that it's more or less
2km.

Melinda

---

From: "John Levine" <[email-masked]>
Date: 25 Dec 2015 00:11:14 -0000

In article <[email-masked]> you write:
>for the Intercontinental is wrong.

Sorry, my bad.  There's two Intercontinentals, this one is indeed
about 2 km away, 25 minute walk or 7 to 10 mins in a taxi if there's
no traffic.

---

From: Fernando Gont <[email-masked]>
Date: Fri, 25 Dec 2015 05:10:53 -0300

> Aires but the maps are pretty clear that it's more or less

While I certainly don't mean to provide advice wrt acommodation options,
looking at sites like booking.com there were way cheaper and good
options at 1km from the Hilton -- you may want to take a look.

Thanks,

---

From: Ted Lemon <[email-masked]>
Date: Fri, 25 Dec 2015 16:26:27 +0000

> benefits and significant damage.

Dave, this may not have occurred to you, but there is another correlation
here that may be the one that matters: _you_ have not had any AD reviews
catch significant issues.   Perhaps you are exceptional.   I am not being
facetious--I suspect that this is in fact the case.

> specifications elsewhere.

If they don't want AD review, they can publish through the ISE!   I don't
think many people realize this is an option, but AFAIK that's the whole point
of having an ISE: to publish things that really are requests for comments.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 26 Dec 2015 07:39:08 +1300

> of having an ISE: to publish things that really are requests for comments.

But they do not get published without review; it's just that the criteria
are different, and sometimes the ISE says no. (Speaking as a member of the
Independent Submissions Editorial Board, as a published author in the Independent
stream, and as a co-author of a draft currently submitted to that stream.)

    Brian

---

From: Ted Lemon <[email-masked]>
Date: Fri, 25 Dec 2015 20:56:04 +0000

> stream, and as a co-author of a draft currently submitted to that stream.)

Correct, but they do get published without the review process that is used for IETF documents, and the process is quite nicely tuned for stuff that you want to publish without that review process.   I suspect it's quite a bit like the IETF review process was before it got to the stage Dave is complaining about, but since I never tried to publish a document during that era I can't say for sure.   I by no means intended to cast aspersions on the ISE process.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 26 Dec 2015 10:51:49 -0500

On Thu, Dec 24, 2015 at 12:47 AM, Joel M. Halpern <[email-masked]> wrote:
> I note in your earlier description you commented that these reviews (whose
> value you doubt) take up AD time when they should be serving as managers.  I
> note that many areas now routinely turn the primary review over to a
> directorate.  The ADs decide how much careful combing they do after the
> directorate.  That seems to enable the kind of delegating that can help,
> without mandating it, and without having rules that micro-manage how ADs do
> their job.

Well yes and no. If I am doing an early review of a doc it might be a
substitute for AD involvement. but if the doc is in IESG last call and
the Security Considerations is a back reference to one in a ten year
old RFC that doesn't actually say anything substantive, SECDIR review
is only going to be a failstop for helping make sure the document gets
attention.

What frustrates me in the reviews is that since we lack a current
description of the Internet architecture, we lack useful guidelines as
to what interactions need to be addressed and which do not.

Where a technology lives in the stack should be a guide to what other
areas it might affect and which security criteria are relevant.

For example, if we have an application specification that depends on a
particular link layer configuration, we have a layering violation. The
cure for that is not 'cross area review', it is for someone to go
learn the principles of modular architecture.

The security requirements that can be met change depending on the
layer you are working at as well. You are not going to be able to
address 'traffic' analysis at the application layer where the
architectural model does not have a concept of network topology. You
are not going to be able to address meta-data analysis at the packet
layer where there is no notion of content.

It is also worth pointing out that directorates have been abused on
occasion. A directorate should never be a cabal that second guesses
the work of working groups behind closed doors.

A lesson that it has taken us longer to understand in security than it
should have perhaps is that implementers have a choice and if we
demand too much, they leave our stuff out completely.

---

From: Abdussalam Baryun <[email-masked]>
Date: Sat, 26 Dec 2015 22:36:19 +0200

On Thu, Dec 24, 2015 at 5:24 PM, Keith Moore <[email-masked]>
wrote:

> inability to get them fixed.    By the time a draft got past WG Last Call,
> it was generally so exhausted that it could not agree on how to make any
> significant change to a document, even in response to a legitimate and
> serious concern.   In many cases, if there was a simple fix, it was quickly
> and happily implemented.   But in far too many cases where there was a
> serious problem, the most that an IESG member could hope for was to get
> some weasel words inserted in the text or prepend an IESG note - neither of
> which really addressed the problem.
>
> detailed review of any lengthy technical document.   While on IESG I tried
> getting others to do reviews for me, but I found that I still had to read
> the documents under review to make sense of the reviews I was getting.   If
> I had to do that again today, I'd probably try to give the reviewers more
> guidance.
>

Easiest way for cross reviews are in meetings, maybe a method can be that
cross-reviews need to be prepared for meetings by WG-chair or ADs. Usually
the ADs are available and responsible, but they need to organise this in
new methods maybe. I think the best way is to have AD feedback in each
meeting session related to all WG-IDs. I think the AD should point to the
cross-WG issues while meetings for each work. The AD feedback will
encourage others to think of the integration of IDs which makes problems
less to occur.

AB

---

From: Keith Moore <[email-masked]>
Date: Sat, 26 Dec 2015 16:13:24 -0500

On 12/26/2015 03:36 PM, Abdussalam Baryun wrote:
>
>
>     But mostly I think that WGs need to have their work subject to
>     formal external review much earlier in the process than Last Call,
>     particularly when the WG's work has the potential to impact other
>     WGs or other areas.
>
>
> Easiest way for cross reviews are in meetings

I disagree.   Though some amount of cross-area review does happen at 
meetings (and if it weren't for meetings, there would be considerably 
less), there are several problems with relying on meetings for 
cross-area review.  The first is that the meetings tend to be scheduled 
in area tracks without much regard for the potential for one WG's output 
to cause problems in a different area.   The second is that the way we 
conduct meetings is itself hostile to cross-area review.   Though people 
from other areas do bring up concerns at meetings, there is a strong 
tendency to view those people as "outsiders" who "haven't done their 
homework" (perhaps because they don't use the same technical terms as 
the WG participants, and/or they just learned of the problem the group 
was causing a few minutes before and haven't had the time to read the 
group's dozen internet-drafts).   And those outsiders are also viewed as 
taking up too much precious meeting time which could otherwise be 
devoted to more PowerPoint presentations.

(Which is a related problem - IETF working groups are producing too many 
documents for the number of participants, resulting in many documents of 
low quality and/or low relevance which IESG mostly feels obligated to 
publish.   And IETF treats that as a sign of success).

Keith

---

From: Dave Crocker <[email-masked]>
Date: Sun, 27 Dec 2015 14:48:07 -0800

Joel,

On 12/23/2015 9:47 PM, Joel M. Halpern wrote:
> (whose value you doubt)

Unfortunate that you'd toss that in as a distracting aside, especially 
since it is not an accurate summary of what I said.

>       take up AD time when they should be serving as
> over to a directorate.

You are implying that the use of directorate reviews is instead of AD 
reviews.  While sometimes it might be, at other times it isn't.

Since my core point is that AD reviews are a strategic problem, not a 
strategic benefit, the fact that there are other reviews done is largely 
unresponsive to my point.

 >  The ADs decide how much careful combing they do

Ahh, good. Another convenient IETF cliche. "Micro-manage" has become 
such a facile way of dismissing substantive comments.  So is the cliche 
to leave everything up to someone else's discretion.

There are strategic issues, here, Joel.  The "AD under lampposts" focus 
is about requiring or encouraging or expecting area directors to do work 
that is not (very) productive, takes (significant) time, and 
(substantially) reduces the pool of area directors, by virtue of making 
the job's footprint larger than it needs to be.

Worrying about the community's basic requirements and expectations on 
ADs is as far from micro-managing as one can get.

Looking at AD Discuss text and listening to IESG Telecons[*] shows 
well-intentioned people who often indulge in pursuit of fine-grained 
detail that is possibly appropriate from an individual contributor and 
never appropriate from a second-level manager.  Alas, it also often 
shows ADs who are more inclined to be swayed by their personal 
preferences than by the formal constraints they are supposed to be 
operating under.

The current Nomcom's 'general' AD job description includes:

> For other Internet-Drafts an AD needs simply to be satisfied that
> adequate review has taken place, though many ADs personally review
> these documents as well.

That looks entirely compatible with your characterization, and my point 
is that having /any/ expectation that /any/ AD will review most or all 
I-Ds that come before the IESG is a mistake.  A strategic mistake.

We need to do a much better job of describing the AD job in ways that 
encourage and require doing specifically the work that the IETF needs to 
have an AD do, and to very carefully exclude tasks that bloat the job 
with time-consuming, low-benefit efforts. The efforts do not serve the 
community well and they reduce the population of AD candidates.

The view that reviews by ADs, themselves, are essential requires 
ignoring trade-offs and, worse, cherry-picking the data.  Let me repeat 
that even in those very rare cases that an AD catches something major, 
we should view that as documentation of a very deep and very serious 
error in a process that let's such basic problems get that far.

Even expecting an AD to be deeply familiar with the documents they are 
'sponsoring' is questionable; it's predicate is that the AD should be 
the primary representative for the work, in spite of not having been a 
principal in the work.  If IESG handling of a specific document is not 
controversial -- ie, no Discusses -- then the AD doesn't need deep 
knowledge.  If the IESG handling involves controversy, Discusses, or the 
like, then get some principals to be involved in the discussion, not the 
second-hand efforts of the AD.i

ADs should worry about basic wg functioning, about community 
participation in the work, about community assessment of the work, and 
about the likelihood of community update of the work.  Lines those ducks 
up and the spec will quack, independent of the AD review.

d/

[*] Jari and the rest of the IESG deserve very considerable community 
thanks for opening the formal telecon meetings to this community 
observation.

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 28 Dec 2015 18:14:00 -0500

> of not filling up blocked rooms...

I believe Ray has said in public before (it'd in any case be easy to
spot in the financial reports) that we have never, at least, paid out
because we've missed our commitments to the hotel.  Note that such a
payment would be quite bad for us, because it'd be in effect an
unbudgetted expense the IETF would have to cover somehow.  It's part
of the reason (as I noted in my other posting on this topic) I'm a
little uncomfortable with making hotel deals so far in the future.  It
is a risk we take in contracting a hotel no matter how far in the
future, however (and I think the people negotiating the deals in this
case are doing a good job of managing that risk).

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: John C Klensin <[email-masked]>
Date: Mon, 28 Dec 2015 18:23:32 -0500

--On Monday, December 28, 2015 18:14 -0500 Andrew Sullivan
<[email-masked]> wrote:

>  Note that such a
> unbudgetted expense the IETF would have to cover somehow.
> It's part of the reason (as I noted in my other posting on
> this topic) I'm a little uncomfortable with making hotel deals
> so far in the future.

And another strong argument against making plans nine years in
advance rather than three, at least IMO.

   john

---

From: Theodore Ts'o <[email-masked]>
Date: Tue, 29 Dec 2015 10:29:21 -0500

> advance rather than three, at least IMO.

When I was treasurer, Usenix used to make hotel contracts five in
advance, because that was the only way the Executive Director could
get desirable hotel contracts for its bigger conferences (which are
roughly the size of IETF these days, BTW).

This hurt a _lot_ after the 2008 crash.  As more than a million of
dollars.  On the plus size, Ellie negotiated with the hotel to convert
the penalties for missing the room block into increased minimum
payments for catering, so we ate really well at the in-hotel reception
for the next couple of years.  So the money wasn't _entirely_ wasted,
but it did make a pretty big dent into Usenix's endowment.

Oh, the IETF doesn't have millions of dollars of endowment banked
away?   Hmm....

						- Ted

---

From: Dave Crocker <[email-masked]>
Date: Tue, 29 Dec 2015 10:58:04 -0800

> of having an ISE: to publish things that really are requests for comments.

Well, ummm...  The Independent Stream has changed substantially.

It now includes anonymous reviews that attempt substantive quality 
assessment and content revision demands, or outright rejections based on 
those reviews and without discussion -- that is, without an attempt to 
ensure the accuracy or relevance of the review comments.

So while the brief published statement of policy that the Independent 
Stream uses rather liberal criteria, my two recent experiences with it 
demonstrate otherwise.

In effect, the Independent Stream has moved from:

      "Is this at all relevant to the general IETF community and is it 
written in a relatively competent/clear manner (and does not not 
conflict with current IETF work)?"

to:

      "Do we think this is something the IETF should be associated with 
and do we agree that it is useful content?"

My description sounds pretty harsh.  I wish I could formulate more 
palatable language, but my two, recent experience really do point 
towards what I've described.

d/