Subject: Re: [testlist] Temporary Datatracker Service Change

From: Glen <[email-masked]>
Date: Wed, 21 May 2008 15:26:29 -0700

Marc Blanchet wrote:
> it would be interesting to know what was causing the problem. this would 
> be valuable for the community in the run to move to Ipv6. maybe you 
> could provide (as you do very well) an update when you have such findings.
> Marc.

Marc -

Thanks for the compliment.  Yes, I'm trying to be as communicative as 
possible.  We've gotten many requests for changes recently from the 
IETF, and that has caused a few bumps.  We're confident that these will 
only be temporary, and I've found that detailed communication is the 
best answer to bumps like this.

In this case, what we have discovered - and what we currently have - is 
a situation where Apache2 is querying itself on an ongoing basis.  From 
the logs for the datatracker zone:

2001:1890:1112:1::21 - - [21/May/2008:10:55:23 -0700] "GET /" 400 1097
2001:1890:1112:1::21 - - [21/May/2008:10:55:24 -0700] "GET /" 400 1097
2001:1890:1112:1::21 - - [21/May/2008:10:55:25 -0700] "GET /" 400 1097
2001:1890:1112:1::21 - - [21/May/2008:10:55:26 -0700] "GET /" 400 1097
2001:1890:1112:1::21 - - [21/May/2008:10:55:30 -0700] "GET /" 400 1097

Because the source address was one of the server's own addresses, 
because it was not the primary address but an alias, because it was 
coming in rapidly, and because I'd been working on the server for 18 
hours without sleep, I made the assumption that the source address was 
invalid.  Because it was coming in only on the datatracker, and only on 
IPV6, I focused there.  Because we've had problems with TMDA overloading 
recently, and because I thought that perhaps -this- might cause an 
overload, I took the datatracker off of IPV6 just for the night, until I 
could do more research.  (Hence the clarification that this was NOT an 
IPV6-related issue - at least I'm pretty sure it's not - see below.)

This morning, I put the IPV6 datatracker back online, and the queries 
started up again.  I tried various trace methods, and followed numerous 
suggestions from the community, but it was lsof that finally showed me 
that apache was, in fact, doing something I'd never seen before, and 
querying itself!  Not only that, but it was the -parent- process, not 
one of the child worders, that was initiating the queries.  So my 
initial killing of that process did stop the queries... but it also 
stopped apache processes from respawning.  So that wasn't going to solve it.

Although this doesn't seem to be causing any immediate problems, the 
potential for a runaway/overload is what I'm worried about.

But since I have no idea why Apache is doing this, I don't have any 
additional answers.  Bill Fenner mentioned to me that he's seen similar 
behavior to this before, but was never able to ascertain which module or 
part of Apache was causing the behavior.  I'll keep looking into it, in 
my copious free time.  ;-)  For now, I've blocked these queries with an 
ip6tables rule, and that seems to be working fine.  Apache keeps trying 
to query, the queries are blocked, and that does not seem to be 
interfering with Apache in any way.

So at this stage, it appears to be an Apache anomaly, not specifically 
related to IPV6 or anything else, and something I'll continue to look 
into as I can.

Hope this helps!

Best,
Glen Barney
_______________________________________________
Testlist mailing list
[email-masked]
https://mail.ietf.org/mailman/listinfo/testlist

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 22 May 2008 01:31:38 +0200

Hi,

> related to IPV6 or anything else, and something I'll continue to look 

Finally found some time to search a bit closer for what this is all about,
and found this thread:

  http://mail-archives.apache.org/mod_mbox/httpd-bugs/200606.mbox/[email-masked]%3E

which has a second post that in passing mentions:

   "The dummy requests are made by httpd to wake up all the children."

while the first post mentions that this seems to happen when a signal (such as
for instance prompted by a logfile rollover) is sent to the parent process.

This seems to make sense to me.

	Henrik

---

From: Glen <[email-masked]>
Date: Wed, 21 May 2008 16:50:19 -0700

> This seems to make sense to me.

Agreed.  I wish there was more info, because we only signal apache once 
per day for logrotation, and that's it.  I'll keep looking as well!

THANKS!
Glen