Subject: [weirds] Redirect Loop Detection - comments on draft-lacnic-weirds-restwhois-redirects-00

From: Wil Tan <[email-masked]>
Date: Wed, 19 Sep 2012 14:37:01 +1000

In section 2.4 of draft-lacnic-weirds-restwhois-redirects-00:

   When redirection is used there is always the risk that bogus user-
   agents and applications or malicious user can create loops that in
   turn may become Denial of Service attacks.

In our use cases where the registries (e.g. the RIRs) are federated,
how could a malicious user create loops? I would imagine that loops
are most likely caused by administrative errors, e.g. wrong entries in
a registry's database. Am I missing something?

The document suggests that the server modify the URI prior to
redirecting, but it's less than ideal since it breaks caching. I
realize that there's no ideal solution here, but feel that this is
perhaps something that's best solved using a combination of:

- knowing the other weirds servers that would redirect to you; and
- if the request needs to be redirected, and the Referer header
indicates that the UA came from another weirds server, refuse to
redirect; and
- we should mandate that UA's send the Referer headers

Also, in 3.1.1.

   Operators that support the redirect URI MUST never process redirects
   that contain a step value greater that their locally set threshold.

If the query can be satisfied locally without responding with a
redirect, shouldn't the operator just process it even if the step
value exceeds threshold?

.wil

---

From: "Carlos M. martinez" <[email-masked]>
Date: Wed, 19 Sep 2012 02:06:55 -0300

Thanks Wil ! Will review.

On 9/19/12 1:37 AM, Wil Tan wrote:
> _______________________________________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/weirds
>

---

From: Arturo Servin <[email-masked]>
Date: Wed, 19 Sep 2012 08:04:43 -0300

Wil 

	Thanks for the comments.

> a registry's database. Am I missing something?

	Yes, basically it is because administrative errors pointing to wrong information.

	In name spaces there could be some bad cases, imagine that registrar X says that example.com is register in Y, then Y said that is W, and then W says that is X. Because is coming from W now, X has no idea that it was the original query

> - knowing the other weirds servers that would redirect to you; and

	That is very complicated, there could be tons. For RIRs could be easy but for names I do not think it is possible.

> indicates that the UA came from another weirds server, refuse to

	Well, then that is basically forbid redirection, isn't it?

> - we should mandate that UA's send the Referer headers

	That is ok, it help to identify simple redirection from A to B, but it does not help with the above case of multiple redirection. 

> redirect, shouldn't the operator just process it even if the step

	Well, perhaps it is language there. We meant to generate another redirection. What about:

"Operators that support the redirect URI MUST never create a new redirect
  that contain a step value greater that their locally set threshold. However 
  if the Operator has an authoritative response to the agent it MUST respond
 regardless to the threshold value."

Regards,
as

---

From: Wil Tan <[email-masked]>
Date: Wed, 19 Sep 2012 21:42:48 +1000

Hi Arturo,

On Wed, Sep 19, 2012 at 9:04 PM, Arturo Servin <[email-masked]> wrote:

>         In name spaces there could be some bad cases, imagine that
> registrar X says that example.com is register in Y, then Y said that is
> W, and then W says that is X. Because is coming from W now, X has no idea
> that it was the original query
>
>
I can't really see the name registries doing redirects the way that the
RIRs do. There are too many TLDs to begin with, and more coming. The only
two use cases I can think of in the name registries are:

1. thin registries like COM/NET who refer to the authoritative server
(operated by a registrar). In that case, though, the registry is still
authoritative for part of the data (like who the registrar of record is,
creation/expiry dates, etc. - all except contacts) so it would still return
a full response, including a referral URL to another server. That is, it
won't be doing a 301 redirect.

2. in a TLD with multiple second level domains that are administratively
separated from the top level, e.g. ZA, and if there isn't a way for clients
to bootstrap the WEIRDS server for the second level namespaces, perhaps a
query for xyz.co.za to the top level .za WEIRDS server could result in a
redirect to the .co.za WEIRDS server. In this case, though, it'd be
terribly inefficient if every single query results in a redirect.

>         That is very complicated, there could be tons. For RIRs could be
> easy but for names I do not think it is possible.
>
>
Right, but as I said above, I doubt it'd be very useful for names, unless
I'm missing something.

>         Well, then that is basically forbid redirection, isn't it?
One level of redirect is allowed in this case. If the query had no Referer
header, or one that isn't identified as a WEIRDS server, a redirection
would occur. In other words, the premise is that, if a query was redirected
to me, but I'm not authoritative for the object, then the previous server
that you asked either got it wrong, or wasn't specific enough.

>         That is ok, it help to identify simple redirection from A to B,
> but it does not help with the above case of multiple redirection.
>
> >
> > Also, in 3.1.1.
> >
> However
Works for me :)

.wil

---

From: Arturo Servin <[email-masked]>
Date: Wed, 19 Sep 2012 08:52:01 -0300

Wil,

	If I understand well your proposal is to only allow just one redirect by using Referer headers, right?

Regards,
as

On 19 Sep 2012, at 08:42, Wil Tan wrote:

> I can't really see the name registries doing redirects the way that the RIRs do. There are too many TLDs to begin with, and more coming. The only two use cases I can think of in the name registries are:
> 
> 1. thin registries like COM/NET who refer to the authoritative server (operated by a registrar). In that case, though, the registry is still authoritative for part of the data (like who the registrar of record is, creation/expiry dates, etc. - all except contacts) so it would still return a full response, including a referral URL to another server. That is, it won't be doing a 301 redirect.
> 
> 2. in a TLD with multiple second level domains that are administratively separated from the top level, e.g. ZA, and if there isn't a way for clients to bootstrap the WEIRDS server for the second level namespaces, perhaps a query for xyz.co.za to the top level .za WEIRDS server could result in a redirect to the .co.za WEIRDS server. In this case, though, it'd be terribly inefficient if every single query results in a redirect.
> 
>  
> Right, but as I said above, I doubt it'd be very useful for names, unless I'm missing something.
> 
> 
> One level of redirect is allowed in this case. If the query had no Referer header, or one that isn't identified as a WEIRDS server, a redirection would occur. In other words, the premise is that, if a query was redirected to me, but I'm not authoritative for the object, then the previous server that you asked either got it wrong, or wasn't specific enough.
> 
>

---

From: Wil Tan <[email-masked]>
Date: Wed, 19 Sep 2012 22:52:45 +1000

> using Referer headers, right?
Yes. The drawback I could see is that I suspect most HTTP libraries don't
send Referer when following redirects. I just checked Python's urllib and
it doesn't.

.wil

---

From: Arturo Servin <[email-masked]>
Date: Wed, 19 Sep 2012 10:07:15 -0300

In fact, the drawback is that one redirect is not enough for RIRs and perhaps for some corner cases in names.

	Example RIR:

	User queries IANA for space X, IANA redirects to ARIN but it does not have the space because it was issued to APNIC because a transfer. 

	Example names:

	User queries .example for example.com. .Example does not about .com and it redirects to IANA. IANA does not have .com and shouldt redirect to Verisign.

	One solution could be instead of doing a second redirect (or even a first one) to return a "redirect object" which in turns would have the information to query the next WEIRDS server:

{
	"redirect-server": "weirds.example.com",
	"redirected-by": "weirds.example",
	"redirect-times": 2
}
	
	Or something like that.

Regards,
as

On 19 Sep 2012, at 09:52, Wil Tan wrote:

> Yes. The drawback I could see is that I suspect most HTTP libraries don't send Referer when following redirects. I just checked Python's urllib and it doesn't.
> 
> .wil

---

From: Wil Tan <[email-masked]>
Date: Thu, 20 Sep 2012 00:03:23 +1000

On Wed, Sep 19, 2012 at 11:07 PM, Arturo Servin <[email-masked]> wrote:

> User queries IANA for space X, IANA redirects to ARIN but it does not have
> the space because it was issued to APNIC because a transfer.
>
>
Would there be any circumstances where ARIN would want to return more
information about a transferred block, such as the size of the block, or
date transferred? If it does, then perhaps we'll need to return a "redirect
object" as you proposed below anyway, because in a 301 you can't convey
anything much other than the target URL.

> Example names:
>
> User queries .example for example.com. .Example does not about .com and
> it redirects to IANA. IANA does not have .com and shouldt redirect to
> Verisign.
>
>
I think in the case of names, this sort of redirection wouldn't be
supported by registries because it's *should* be quite straightforward for
a client to know that it's querying the wrong server (by comparing the
TLD). Also, I doubt IANA would run a generic redirector service. That's
just my view, and it'd be great if others can weigh in.

> One solution could be instead of doing a second redirect (or even a first
> one) to return a "redirect object" which in turns would have the
> information to query the next WEIRDS server:
>
> {
I suppose that could work, at the expense of more complexity for clients to
support redirects. What we gain is cacheability.

.wil

---

From: Andy Newton <[email-masked]>
Date: Wed, 19 Sep 2012 14:17:50 +0000

>
>User queries IANA for space X, IANA redirects to ARIN but it does not
>have the space because it was issued to APNIC because a transfer.

I believe at that point ARIN would redirect to APNIC, right? What am I
missing?

>date transferred? If it does, then perhaps we'll need to return a
>"redirect object" as you proposed below anyway,
> because in a 301 you can't convey anything much other than the target
>URL.

Not that I like this idea, but I thought HTTP allowed a entity body even
for redirects. But I could be wrong.

>Also, I doubt IANA would run a generic
> redirector service. That's just my view, and it'd be great if others can
>weigh in.

I believe ICANN could be easily convinced to run such a service.

-andy

---

From: Arturo Servin <[email-masked]>
Date: Wed, 19 Sep 2012 11:28:33 -0300

Andy,

> I believe at that point ARIN would redirect to APNIC, right? What am I

	Wil is suggesting to do just one redirect. If that were the case, in the IANA-ARIN-APNIC example you would need 2 redirects. One for IANA to ARIN and another from ARIN to APNIC. A similar case happens when you have ReferralServers (I think just ARIN has them at least for now).

Regards,
as

---

From: Andy Newton <[email-masked]>
Date: Wed, 19 Sep 2012 14:50:31 +0000

>
>Wil is suggesting to do just one redirect. If that were the case, in the
>IANA-ARIN-APNIC example you would need 2 redirects. One for IANA to ARIN
>and another from ARIN to APNIC. A similar case
> happens when you have ReferralServers (I think just ARIN has them at
>least for now).

So one redirect is ok, but two redirects is not? I do not understand the
concern.

-andy

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 19 Sep 2012 15:10:22 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Wil Tan
Sent: Wednesday, September 19, 2012 10:03 AM
To: Arturo Servin
Cc: [email-masked]
Subject: Re: [weirds] Redirect Loop Detection - comments on draft-lacnic-weirds-restwhois-redirects-00

On Wed, Sep 19, 2012 at 11:07 PM, Arturo Servin <[email-masked]<mailto:[email-masked]>> wrote:

Example RIR:

Would there be any circumstances where ARIN would want to return more information about a transferred block, such as the size of the block, or date transferred? If it does, then perhaps we'll need to return a "redirect object" as you proposed below anyway, because in a 301 you can't convey anything much other than the target URL.

Example names:

User queries .example for example.com<http://example.com>. .Example does not about .com and it redirects to IANA. IANA does not have .com and shouldt redirect to Verisign.

I think in the case of names, this sort of redirection wouldn't be supported by registries because it's *should* be quite straightforward for a client to know that it's querying the wrong server (by comparing the TLD). Also, I doubt IANA would run a generic redirector service. That's just my view, and it'd be great if others can weigh in.

IANA might not, but others might (and probably will).

Scott

---

From: Wil Tan <[email-masked]>
Date: Thu, 20 Sep 2012 01:09:59 +1000

> So one redirect is ok, but two redirects is not? I do not understand the
My concern is that modifying the URI just to curb redirect loops is not
cache-friendly. In trying to come up with an alternative, I thought that if
there were no real use cases for supporting multiple redirects, then we
could stop redirect loops by allowing only a single redirect by checking
the Referer header, assuming clients send them.

The proposal was: upon receiving a request, if the server is not
authoritative for the object requested, it would check to see if the
Referer header value, if present, is one of the other weirds servers that
would redirect to it (IANA and the other RIRs), and if so, it will refuse
to redirect the client.

In any case, I think Arturo had just debunked my wrong assumption of not
needing multiple redirects.

.wil

---

From: Arturo Servin <[email-masked]>
Date: Wed, 19 Sep 2012 12:22:58 -0300

I think that we could conclude:

	- More that one redirect is required
	- Currently approach to detect loops is not cache-friendly
	- We need to decide:
		- If we require to detect the loop, if so
			- a cache-friendly approach
			- or just live with the current one

	So, my first question to the group is:

	Do we need to detect in the server that a loop has occurred or we just leave it to the user agent to deal with it?

Regards,
as

On 19 Sep 2012, at 12:09, Wil Tan wrote:

> My concern is that modifying the URI just to curb redirect loops is not cache-friendly. In trying to come up with an alternative, I thought that if there were no real use cases for supporting multiple redirects, then we could stop redirect loops by allowing only a single redirect by checking the Referer header, assuming clients send them.
> 
> The proposal was: upon receiving a request, if the server is not authoritative for the object requested, it would check to see if the Referer header value, if present, is one of the other weirds servers that would redirect to it (IANA and the other RIRs), and if so, it will refuse to redirect the client.
> 
> In any case, I think Arturo had just debunked my wrong assumption of not needing multiple redirects.
> 
> .wil

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 19 Sep 2012 15:26:53 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Arturo Servin
Sent: Wednesday, September 19, 2012 11:23 AM
To: Wil Tan
Cc: [email-masked]
Subject: Re: [weirds] Redirect Loop Detection - comments on draft-lacnic-weirds-restwhois-redirects-00

As a server operator I like the idea of being able to detect and abort a loop.

Scott

---

From: "Carlos M. martinez" <[email-masked]>
Date: Wed, 19 Sep 2012 12:36:36 -0300

I agree with Arturo, and definitely I see use cases for more than one
redirect.

The caching concern is a valid one though, so it is back to the drawing
board for that part :-)

~Carlos

On 9/19/12 12:22 PM, Arturo Servin wrote:
> Do we need to detect in the server that a loop has occurred or we just
> leave it to the user agent to deal with it?
> 
> Regards,
> as
> 
>> On Thu, Sep 20, 2012 at 12:50 AM, Andy Newton <[email-masked]
>> <mailto:[email-masked]>> wrote:
>>
>>     >
>> My concern is that modifying the URI just to curb redirect loops is
>> not cache-friendly. In trying to come up with an alternative, I
>> thought that if there were no real use cases for supporting multiple
>> redirects, then we could stop redirect loops by allowing only a single
>> redirect by checking the Referer header, assuming clients send them.
>>
>> Referer header value, if present, is one of the other weirds servers
>> that would redirect to it (IANA and the other RIRs), and if so, it
>> will refuse to redirect the client.
>>

---

From: Andy Newton <[email-masked]>
Date: Wed, 19 Sep 2012 16:02:58 +0000

Thanks. Now I understand.

As far as I can tell, there are two use cases:

  1.  Client asks for IP or ASN at ARIN. ARIN was never authoritative for it but knows that RIPE is, so ARIN redirects to RIPE. RIPE has transferred the IP or ASN to APNIC, so RIPE redirects to APNIC. And from there it stops. There will only ever be three levels deep at most because all subsequent parceling out of that space has to be done by RIPE.
  2.  Client asks for example.biz at VeriSign. VeriSign is not authoritative for .biz, so it redirects to Neustar. It might end here or Nuestar may redirect to the registrar (though I doubt it).

In both cases, a loop is a condition that is an error and is caused by a misbehaving client because the servers never would have redirected them into a loop. Therefore we should treat this condition like any other of a misbehaving client. Even if we standardized loop detection parameters, there's nothing to say the client will still do the right thing.

-andy

---

From: "John Levine" <[email-masked]>
Date: 19 Sep 2012 20:27:30 -0000

>In both cases, a loop is a condition that is an error and is caused by a
>misbehaving client because the servers never would have redirected them into a
>loop.

I don't think we can assume that.  No matter what we do, there's no
way that we can predict every possible server misconfiguration that
could lead to a redirect loop, particularly a loop that goes through
multiple servers.  Example: client queries ARIN, ARIN redirects to
RIPE, RIPE redirects to an LIR, LIR redirects to a hosting company
that provides SWIP-like info for its clients, company hasn't yet
configured the range and helpfully provides a default redirect back to
ARIN.  Oops.  Yeah, they shouldn't do that, but they will.

People are going to do their WEIRDS lookups using existing http
clients, or new clients will use existing http libraries, all of which
use counters to break redirect loops.  (They've been a problem for
over 15 years, after all.)  If there's a loop, it's much, much more
likely to be due to multiple servers disagreeing about who's
responsible for something than client bugs.

Even if someone does manage to write a client that loops, the only way
you're going to know that it's looping is that it's hammering on your
server at a high rate.  So as you suggest, the defenses you already
have against unreasonable clients will deal with this rather unlikely
scenario.

R's,
John

---

From: Andy Newton <[email-masked]>
Date: Wed, 19 Sep 2012 20:33:35 +0000

>have against unreasonable clients will deal with this rather unlikely

So we've come to the same conclusion from a different starting point.
Excellent! :)

-andy

---

From: Edward Lewis <[email-masked]>
Date: Thu, 20 Sep 2012 12:35:21 -0400

Commenting on this thread in general...

I'd drop any assumptions about why a query arrives at a server. 
Intended or not, a protocol should define what happens when it 
accepts a query that it has no data to match (and may know a better 
source).  This is just basic protocol engineering.  The DNS has a 
similar problem and there are a few lessons to be taken from there. 
(I'm not going to say DNS's model is the way to go, it's just a 
working example.)

1. In DNS there is a record called a CNAME which causes a query to 
"restart" with a new (arbitrary) name, hence there is no guarantee 
that a loop is avoided.  The solution used in DNS is to have the 
querier keep a count and cut off searching at 5.  Yes, "5" because, 
well, why not, it's a cool number.  We don't want a scared number 
like 6, who is afraid of 7 because "7 8 9".  (Those teaching children 
English numbers get that joke.)  What is architecturally significant 
here is that the burden is placed on the querier to "police" this. 
The assumption is that answering with a CNAME is not a burden to the 
server.

2. The DNS has coined the term "lame" for any server that is asked a 
question about data not in its knowledge base.  (I.e., a "lame 
server".)  Transferring that term here, we are concerned about WEIRDS 
protocol servers that are lame for queries and how do they react - 
regardless of why the query was sent.

"Lameness" and the CNAME loop are unrelated topics in DNS, they both 
have lessons learned to apply here.

FWIW, even if from DNSSEC alone, we have seen that operational 
miscues are far more damaging than a security incident.  Although the 
latter might be a bigger problem for the victim, the former is 
"across the board" pain.

3. In the DNS there was no original indication of lameness.  Over 
time the convention was to "go ask upwards" (the root - or IANA was 
the target of the redirection).  After decades of this as accepted 
practice this got turned into a DDoS abuse point because the referral 
was larger than the query.  Emphasizing this is DNS specific, the DNS 
protocol is highly susceptible to abuse when the answer is more bytes 
than the question.  Today servers return a response code indicating 
there's no response and it is up to the client to figure out where to 
go.

(Plus - in the 90's a particular buggy OS implementation, in the 
sense what it gullibly accepted the upwards referral, caused a 
material problem that led to at least one ARIN policy to look for 
lame servers.)

When it comes to this topic, there is a difference between names and 
numbers.  With names, it is "obvious" who the appropriate server 
(provider) should be, given the TLD and the sublabels from there. 
For numbers, the RIR federation has apportioned the address space in 
a non-obvious way.  The use case of ARIN responding with a redirect 
to LACNIC is far more anticipated than Verisign redirecting to 
Neustar.  Although this is a difference I don't think it is 
significant as queriers still have to know how to navigate and 
servers still have to know how to respond.  Servers for names as well 
as numbers might get a query for "257.12.34.82" and they will all 
have to tell the client to go somewhere else.

Are loops possible - yes.  Should a querier be smart about them - 
yes.  Does a server need to worry about them - it shouldn't.  Does a 
server have to indicate it doesn't have the answer - yes.  Does the 
server need to tell the client where to ask next - for queries for 
names it shouldn't, for queries for numbers a solution is needed.

If generating a redirect is easy, then placing the burden to avoid 
them on the client will diffuse the problem.  If knowing where to 
tell the client to go is hard, redirecting isn't easy.  Even a names 
server needs to know what is the right numbers server.

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

2012...time to reuse those 1984 calendars!

---

From: "Linlin Zhou" <[email-masked]>
Date: Fri, 21 Sep 2012 11:30:58 +0800

> 1. In DNS there is a record called a CNAME which causes a query to
"restart" with
> a new (arbitrary) name, hence there is no guarantee that a loop is
avoided.  The
> solution used in DNS is to have the querier keep a count and cut off
searching at 5.
> Yes, "5" because, well, why not, it's a cool number.  We don't want a
scared
> number like 6, who is afraid of 7 because "7 8 9".  (Those teaching
children
> English numbers get that joke.)  

I found that RFC2616 10.3 also recommend maximum of five redirections. Do we
need to put this recommend in this section to avoid the threshold is set too
big or too small?
If the redirection times greater than the threshold, I suggest have a
response error code defined here, such as 40x or 50x.

>What is architecturally significant here is that
> the burden is placed on the querier to "police" this.
server.

Linlin