Subject: [71attendees] 2001:0df8:0:5::6 down?

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 14 Mar 2008 10:44:51 -0400

Looks like the first IPv6 DNS resolver 2001:0df8:0:5::6 is down...

I'll remove it from my settings now, but for future reference: it's  
nice to keep the nameservers running at least until the end of the  
last session.
_______________________________________________
71attendees mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/71attendees

---

From: Michael Graff <[email-masked]>
Date: Fri, 14 Mar 2008 10:48:58 -0400

Ok don't believe it was intentional. A noc guy here says the entire  
machine is misbehaving.

--Michael

---

From: Bill Fenner <[email-masked]>
Date: Fri, 14 Mar 2008 11:28:03 -0400

> Looks like the first IPv6 DNS resolver 2001:0df8:0:5::6 is down...

It is back up now.  postgresql hung oddly, with 38 processes waiting  
for each other's locks, and this caused the rest of the system to  
start failing oddly.  Our apologies for the outarge.

> nice to keep the nameservers running at least until the end of the

For future reference: all of the network services are officially  
maintained until noon on Friday, except for APs in rooms that have no  
more meetings in them.

   Bill

---

From: Jim Martin <[email-masked]>
Date: Fri, 14 Mar 2008 10:54:09 -0400

We're seeing two network events at this time. The primary "100G" link  
has gone down and the primary internal services box (unix server) is  
having challenges. We've failed over to the backup links and are  
working on either getting the primary DNS/DHCP working or switched to  
backup servers.

	Sorry about the outage!

	- Jim

---

From: Tony Tauber <[email-masked]>
Date: Sat, 15 Mar 2008 00:00:11 -0400

On Fri, Mar 14, 2008 at 10:54:09AM -0400, Jim Martin wrote:
> 	We're seeing two network events at this time. The primary
> 	"100G" link  has gone down and the primary internal services

Just to clarify what happened on the WAN side of things:

Due to a mis-communication about the shutdown plans for the last day
of the meeting, at 10:23 ET, the 100G service was deleted from the
transport system.  Since this was not done gracefully, traffic appears
to have been black-holed for a few minutes until the BGP hold-timer
expired and things failed over to the 10G backup wave on the same fiber.

The NOC crew labored for several minutes more to deal with the sick
DHCP server which, from some of things I was experiencing, had been
getting a bit ill starting earlier in the morning from what I could
see (periodic loss of DHCP service in the RRG room).  

I believe these events are completely coincidental.

Tony

> 	box (unix server) is  having challenges. We've failed over to
> 	the backup links and are  working on either getting the
> 	primary DNS/DHCP working or switched to  backup servers.
> 
> 	Sorry about the outage!
> 
> 	- Jim
>