Subject: Re: [alto] Some questions for alto-xdom-disc

From: Sebastian Kiesel <[email-masked]>
Date: Mon, 4 Dec 2017 22:16:46 +0100

Dear xin wang, all,

please see below

On Fri, Dec 01, 2017 at 04:05:47AM +0000, xin wang wrote:
> Dear authors of alto-xdom-disc and all,
> 
> 
> Do you have any new updates on the draft of alto-xdom-disc?

we are working on a new version of the draft, which will give a better
specification of the discovery procedure as such (i.e., section 2).
We are not planning to make a change on how the procedure is supposed
to work, just give a better explanation.

> I know that the draft intends to address the IRD discovery issue in
> the cross-domain setting, but I find that the cross-domain itself
> arouse my great interest.

Well, the immediate outcome of the procedure is one or more IRD URIs.

However, nobody is interested in discovering IRDs as such - it is just
an intermediate step, so you can lookup the ECS or EPS in the IRD.
In fact, our procedure is intended to be used with the
Endpoint Property Service and the Endpoint Cost Service.

> Considering that each domain has one ALTO server that can give a
> useful cost value (not default) between any two endpoints in the
> domain for the ECS. Then, the ALTO server discovery works when a
> client asks for the cost value between any two endpoints in the same
> domain, as it will direct to the right ALTO server which can give cost
> values for some endpoint-pairs. It might involve multiple ALTO servers
> to answer a single query of ECS service but the requirement of the
> client can be satisfied.
> 
> 
> However, if a client asks for a cost value between two endpoints which
> locate in different domains, then who should be able to give the cost
> value?

We believe that in many scenarios, the best source of information
for costs between source IP address S and destination address D,
is the network operator that runs the network in which S is located.
Consequently, we believe that this network operator should be able
to announce "if you want to do ECS(S,D) please ask the ALTO server
at http://...some.uri..."

> This should be a common case if we target to deploy ALTO across
> the public Internet as you listed as one of the requirements for ALTO
> cross-domain server discovery.

Indeed.

> There are basically two approaches for the issue above: one is to
> depend each ALTO server itself to compute cost values across domains
> (e.g., recursive sending queries to other servers); the other is to
> set up a hierarchy structure to relay the query to an upper ALTO
> server which is able to support ECS across domains. In either way,
> there need substantial efforts to consider/design a protocol between
> two ALTO servers (which is discussed a little in the Sec. 1.1.2 in
> your draft).
> 
> Do you think there are potential solutions can resolve the issue
> without depending too much on the inter-ALTO-server information
> exchanging? Or design inter-ALTO-server protocol is the best
> direction?

We do a classification and discussion of several approaches.
However, we then focus on one approach, which does not need any
communication between ALTO servers, neither using the regular ALTO
protocol nor a to-be-defined inter-ALTO-server protocol.

Instead, we let the ALTO client do the work. 

If an ALTO client wants to query the ECS with specific source and
destination addresses, it has to discover an appropriate ALTO server
first.  Then, it can ask this server, and this server is supposed to
answer without any consultation of other servers.

Furthermore, we want to avoid that a new "rendezvous point" or
Internet-wide directory of ALTO servers would have to be established.
We want to build on an existing infrastructure.

The idea is as follows:

Conceptually, the ECS does a query on a large N x N matrix, 
where the column headers are labeled "from IP address (or prefix)" and 
the row headers are labeled "to IP address (or prefix)".
However, we believe that it is unlikely that a single ALTO server will
ever accumulate so much data that it can give reasonable values for
every element of the matrix, in an Internet-wide deployment scenario.

Instead, we split our large matrix into many small 1 x N "column vectors".
Each of them indicates the cost from one specific IP address (or prefix)
to all possible IP addresses (or prefixes).  Then, we can install each
of these column vectors on a different ALTO server. So, in total we can
have up to N ALTO servers, each with one stripe of the overall matrix.
Of course, one ALTO server can also host more than one column vector, so
we may work with fewer than N servers.

Those who control the reverse DNS for S (i.e. the mapping from IP
address S to a host name) can put a record in the DNS pointing from S
to the URI of that ALTO server that knows the column vector
"from S to all possible IP addresses (or prefixes)".  
This is how we use an existing infrastructure (the DNS) for the
discovery job.

As I wrote above, we are working on an update to section 2.
Could you please review section 3, which explains the interaction
of the procedure with ECS and the other ALTO services?  Thanks!

best regards,
Sebastian

---

From: Qiao Xiang <[email-masked]>
Date: Tue, 5 Dec 2017 04:26:59 +0000

Hi Sebastian,

Tony and I just had some offline discussion on the cross-domain draft. We
feel that some parts in Sec.2.1 and Sec. 3 should be updated in the newer
version.

In Sec. 2.1, it is written that "... It performs DNS lookups and returns
one or more URI(s) of information resources related to that IP address or
prefix,...". Tony pointed out that the word "related" is vague, and I agree
with him. From the remaining of Sec. 2, I can infer the following workflow:
given an ALTO server A, if it can provide ALTO information on an IP address
or prefix "X", it *can* choose to prepare several domain names, so that the
ALTO client can discover that "server A can provide information on
address/prefix X" via reverse DNS lookup. We both feel that this workflow
should be clearly stated to replace the vague "related to" sentence quoted
above.

However, even with a clearer description on how each ALTO server publishes
what kind of IP addresses/prefixes it can provide information about, in my
opinion, the EPS and ECS use cases given in Sec. 3 are still questionable.
It is unclear to me what points we want to express in Sec. 3. Is it a
standard procedure specifying how ALTO client should use this discovery
process, or only a suggested, incomplete guideline?

in Sec. 3.3, it is stated that "If the ALTO client wants to do a similar
Endpoint Property query for a different IP address or prefix "Y", the whole
procedure has to be repeated,". However, assume the ALTO client first wants
to discover which ALTO server can provide information of IP 198.51.100.3,
and finds an entry with key100.51.198.in-addr.arpa. Then if it wants to get
EPS information of IP 198.51.100.0/24, it can still use the same ALTO
server.

In Sec. 3.4., take the one source S1 and multiple destination D1, D2, D3,
... as an example. your design asks the client to issue a lookup in the
format of XDOMDISC(S1, "ALTO:https"), and then query the returned IRDs. But
what if there exists an ALTO server A that has endpoint cost information
for (S1, D1), but only publishes a domain name entry
D1(reversed).in-addr.arpa, and A is the only server that has such
information? In this way, the ALTO client cannot get the endpoint cost of
(S1, D1) from any other ALTO servers discovered via XDOMDISC(S1,
"ALTO:https"), but it can discover server A by a query XDOMDISC(D1,
"ALTO:https").

To address these issues, I agree with Richard's comments from IETF99, which
says that we should state the design objectives, principles and
requirements of cross-domain servery discovery more clearly.

Looking forward to your thoughts on these issues. Thank you very much.

Best
Qiao

On Mon, Dec 4, 2017 at 9:16 PM, Sebastian Kiesel <[email-masked]>
wrote:

> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto
>

-- 
Qiao Xiang
Postdoctoral Fellow,
Department of Computer Science,
Yale University

---

From: xin wang <[email-masked]>
Date: Tue, 5 Dec 2017 05:30:20 +0000

Dear Sebastian and all,

Thank you for the clarification!

First, I agree with your point that “the best source of information for costs between source IP address S and destination address D, is the network operator that runs the network in which S is located”, which is quite natural and reasonable. My consideration is that what is the best source of information really depends on the implementation of reverse DNS where network operators put records for ALTO server discovery. Therefore, it is possible that different network operator may have different strategies to choose the best source of information. I think we should not restrict these network operators with the same strategy. Instead, the protocol should define what the best source (return of XDOMDISC(X,"ALTO:https")) would look like, I think. The following is my comments on Sec. 3:

For Sec. 3.1, as the list of IRD URI(s) is computed by XDOMDISC(X,"ALTO:https”), I think it is acceptable that the result is very sparse. We can see that the result basically captures all the cost values related to “X” which satisfies the requirement of the ALTO client invoking the service for “X”.

However, one thing not sure is that do we have a formal definition of “related” in Sec. 2.1 where the definition of the interface is given? From your draft, my understanding of “related” (i.e., the return of XDOMDISC) is the following:

If an IRD contains an ALTO server which can give at least one useful cost value either with “X” as a source or destination, then the URI of the IRD can be returned for XDOMDISC(X,"ALTO:https”).

This definition can match the case 2 in Sec. 3.4 where the input of XDOMDISC is the destination address. However, the definition would degrade the return of XDOMDISC in the case 1 in Sec. 3.4 as in the extreme case for each destination address Di, there is an ALTO server “related” to Di. Hence, the return of XDOMDISC(S1,"ALTO:https”) would have n + 1 entities where n is the number of destination addresses. Then, how to find the best one among these n + 1 entities becomes a new issue.

One direct solution I can think of is that can we also add the “position” of “X” as an additional input of XDOMDISC? The “position” field has three possible values: “source”, “destination”, and “both" which means the discovered ALTO server can give at least one useful cost value with “X” as “source”, “destination”, and “both" respectively. Then, the interface would be XDOMDISC(X, Position, "ALTO:https”). If this sounds reasonable, we can then generalize the “position” field in a more abstract way if possible.

Best,

X. Tony Wang

________________________________
From: Sebastian Kiesel <[email-masked]>
Sent: Monday, December 4, 2017 16:16
To: xin wang
Cc: [email-masked]; IETF ALTO
Subject: Re: Some questions for alto-xdom-disc

Dear xin wang, all,

please see below

Indeed.

The idea is as follows:

best regards,
Sebastian

---

From: Sebastian Kiesel <[email-masked]>
Date: Wed, 13 Dec 2017 21:26:58 +0100

Hi Qiao,

thanks for your feedback! please see below.

On Tue, Dec 05, 2017 at 04:26:59AM +0000, Qiao Xiang wrote:
> with him. 

This vague wording is kinda intentional, as a hint that the procedure
specified in section 2 as such is rather generic: you call it with an IP
address or prefix and a service identifier, and you get an URI.  All the
ALTO-specific stuff is in section 3.

> From the remaining of Sec. 2, I can infer the following workflow:

The workflow from am ISP's perspective is (maybe we should write that
down into a new section):  

If (and only if) you control the "reverse DNS" for a given IP address
or prefix X (i.e., if you can make entries in x.x.x.x.in-addr.arpa,
usually because you are an ISP or company IT department), then
you should think about who has the best "ALTO information" for traffic
from X to any other destination (i.e., who has the best "column vector"
as I called in my mail queted below).

In many cases this will be you. Put your "column vector" in your ALTO
server and add entries to the "reverse DNS" that point to it.

However, in some cases, someone else may have better information, e.g.
your upstream wholesale ISP.  Or, someone else's information is good
enough and you are too lazy to run an own ALTO server.  In these cases,
add entries to your "reverse DNS" that point to their ALTO server.

If you do not control the reverse DNS, this whole approach is not
suitable for you, in order to publish who has good ALTO information
for your addresses.

Does this explanation make sense?  Would it be helpful to add it to the
document?

> process, or only a suggested, incomplete guideline?

see above. It is intended as normative specification how to use the
rather generic procedure of section 2 with ALTO.

> EPS information of IP 198.51.100.0/24, it can still use the same ALTO

Well, there are prefix delegations that are smaller (longer prefixes)
than /24's.  Or maybe 198.51.100.(2-253) are your clients while 
198.51.100.1 is a server that needs special treatment.  Given that
the DNS has quite efficient caching mechanisms, a new lookup for every
IP address won't hurt.

> "ALTO:https"), but it can discover server A by a query XDOMDISC(D1,

If we assume the scenario with one source S1 and multiple 
destinations D1, D2, D3,  we could do three lookups

ALTOSERVER1=XDOMDISC(D1, "ALTO:https")
ALTOSERVER2=XDOMDISC(D2, "ALTO:https")
ALTOSERVER3=XDOMDISC(D3, "ALTO:https")

In a large scale scenario, i.e., if D1, D2, and D3 are in different
ISP's networks we will probably end up with 
ALTOSERVER1 != (not equal) ALTOSERVER2 != ALTOSERVER3

Then, we could ask ALTOSERVER1 for c1=ECS(S1,D1,routingcost)
and ask ALTOSERVER2 for c2=ECS(S1,D2,routingcost)
and ask ALTOSERVER3 for c3=ECS(S1,D3,routingcost)

The problem is now that we cannot compare c1 with c2 and/or with c3, as
routingcost has no unit of measurement.  We only know "smaller is better
WITHIN the context of the same ALTO server".

That's why it is much more useful in this scenario to do only one lookup
ALTOSERVER=XDOMDISC(S1, "ALTO:https")   and ask this single ALTO server
all three questions.

If we would like to query for a different ALTO cost metric than "routing
cost", one with a well-defined metric such as "router hops", "ASN hops"
or "air miles between the geolocations", both approaches would work.
However, we still belive that ISPs should put entries in the DNS such
that our preferred query method will succeed.

> Looking forward to your thoughts on these issues. Thank you very much.

Thank you for your feedback!
Sebastian

---

From: Sebastian Kiesel <[email-masked]>
Date: Wed, 13 Dec 2017 21:39:19 +0100

Hi Tony,

please see below.

On Tue, Dec 05, 2017 at 05:30:20AM +0000, xin wang wrote:
> First, I agree with your point that “the best source of information
> for costs between source IP address S and destination address D, is
> the network operator that runs the network in which S is located”,
> which is quite natural and reasonable. My consideration is that what
> is the best source of information really depends on the implementation
> of reverse DNS where network operators put records for ALTO server
> discovery. Therefore, it is possible that different network operator
> may have different strategies to choose the best source of
> information.

You are right.  Let me be more precise: Our assumption is not that those
who control the reverse DNS for a prefix ALWAYS have the best ALTO
information. Often it will be them, sometimes it could be someone else.
We just assume that those who control the reverse DNS for a prefix
always know who has the best information, and they can put a pointer
to it in their DNS.

> I think we should not restrict these network operators
> with the same strategy. Instead, the protocol should define what the
> best source (return of XDOMDISC(X,"ALTO:https")) would look like, I
> think. 

For further discussion about the word "related" please see my other
mail to qiao.

Thanks,
Sebastian

> The following is my comments on Sec. 3:
> 
> 
Viele Gruesse,
Sebastian