Subject: [Idr] Rtgdir early review of draft-templin-atn-bgp-07

From: Stewart Bryant <[email-masked]>
Date: Tue, 24 Jul 2018 04:09:46 -0700

Reviewer: Stewart Bryant
Review result: Has Issues

This is basically a well written draft, although it has has a lots of spelling
mistakes and needs to passed through a spell checker before a further version
is uploaded.

However, I have a number of issues that the chairs and authors should consider
before deciding how to proceed.

The first question to ask is whether the work belongs in RTGWG or in IDR since
with the exception of the reference to  the optimization described in
draft-templin-aerolink-82  this is a pure BGP solution. This is something that
the RTGWG chairs should discuss with the IDR chairs. They should also discuss
whether the draft needs to be looked at by a BGP specialist before RTGWG adopts
it.

The approach of building an overlay to separate the network from the main BGP
system is a sound one, indeed from both the numbers and a security point of
view, it would appear to be  essential in a safety critical application such as
this. What bothers me is whether the author is underestimating the risk of
running an application such as this over the public Internet. Whilst as they
explain in the security section, high profile civil users do understand how to
mitigate risk and minimize the effect of attack, none of these organizations
are as large a target as civil aviation flight safety, and thus I would expect
considerable resources, even to nation state level, to be directed at the
attachment points. Hopefully ICAO fully understands the risks in running this
on the public Internet as proposed. If ever there were an application for
inter-domain network slicing, this is it.

There is a suggestion in section 3 that for reasons of cost, globally unique
ASNs would not be used. It is difficult to believe that cost is an issue in an
SoF system. What is surely needed is the most robust approach, which in the 
long term is usually the cheapest anyway. Using global identifiers minimizes
the possibility of error and issues if ever there is a routing leak, and thus
the decision on whether to use private or global identifiers needs some careful
thought beyond that expressed in this version of the draft.

I am worried about the text towards the end of section three which proposes
splitting the network into two or more disjoint systems, since that will surely
lead to operation and integration issues.

In section 6 consideration is given to scaling the core, which looks basically
under control for the existing flight profile, but with relatively little
headroom for expansion.  Given the rise in UAVs, that will surely need to be
integrated into a common flight safety system, I am concerned as to whether the
authors have allowed sufficient room for expansion. Again hopefully the flight
specialists have taken this into account and this is not a problem.

Related to the above, presumably flight density if not uniform and the authors
have satisfied themselves with the scaling of the stub areas.

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 24 Jul 2018 19:53:55 +0000

Hello Stewart,

Thank you for your effort in reviewing the document. Please see below for
follow-up discussion:

Fred

> -----Original Message-----
> From: Idr [mailto:[email-masked]] On Behalf Of Stewart Bryant
> Sent: Tuesday, July 24, 2018 4:10 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
> Subject: [Idr] Rtgdir early review of draft-templin-atn-bgp-07
> 
> Reviewer: Stewart Bryant

Certainly we will fix that; apologies if it impacted the readability.

> whether the draft needs to be looked at by a BGP specialist before RTGWG adopts

The document has been presented at RTGWG at three different IETF meetings
now and seems to have been well received there. So, we would like to see it
become an RTGWG working group item so we would not have to start the
socialization process over again, but your point is well taken. I do not have
enough experience with routing area working groups to understand all of
the implications, so any guidance would be appreciated. 

> inter-domain network slicing, this is it.

For civil aviation, this is indeed a critical consideration. On the open public
Internet, for example, although IPsec could be used to establish secure
tunnels other threat vectors such as DDoS need to be considered. In my
understanding, ICAO is thinking about an underlay network comprising
secured physical links (e.g., fiber) and/or constructs like SD-WAN. But,
the construction of the underlay I think is something that has been
taken for granted and as you observe is pivotal to the design security.

> thought beyond that expressed in this version of the draft.

I don't mind dropping the assertion that private AS numbers could be used.
As you say, there is probably no downside to procuring real global AS
numbers up front even if the overlay is never connected to the global
public Internet.

> lead to operation and integration issues.

Looking at that text with fresh eyes, I can see that a rewrite would help
bring across the point we are trying to make. What we want to have is
a means for supporting multiple independent RIBs and bounding the
numbers of prefixes each RIB will be sized to carry. So, if each RIB
supported up to 1M prefixes and we had 1K RIBs, we would be able
to service 1B prefixes or even a bit more. Does that help clarify? If so,
I will consult with co-authors to improve that text.

> specialists have taken this into account and this is not a problem.

The upper bound in terms of scaling we were targeting was up to 1B
prefixes carried as BGP routes in order to allow for expansion. You are
right to observe that we are at the advent of wide-scale deployment of
UAVs, but this network would be only for those UAVs that fly to, from
or through airspace controlled by Air Traffic Controllers (ATCs) - meaning
it would only be for UAVs that take off and land at airports. For the future,
the types of UAVs that will eventually do that include unmanned cargo
delivery freighters and the like. But, we would only be expecting a few
thousands, ten-thousands, or hundred-thousansds of those; certainly
not millions and millions.

For small UAVs that stay away from the airports, however, there is
potential for many millions of those or more. For them, the FAA and
NASA have a concept known as "Unmanned (air) Traffic Management"
or "UTM". The UTM would need to be a separate network unto itself,
but we believe these same BGP principles could apply there.

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 25 Jul 2018 10:13:31 +0100

> the implications, so any guidance would be appreciated.

Actually it might also be a candidate for BESS (BGP enabled services).

This is a chair/AD decision as much as anything, and they are normally 
guided by where
the required expertise is.

> taken for granted and as you observe is pivotal to the design security.
Indeed, I would have expected this to be on a secure network of some 
sort either purely
private or some form of VPN. However, I am sure I read in your text that 
you were
considering using the Public Internet much in the way of SD-WAN.

> I will consult with co-authors to improve that text.

Wow, are you sure that such a system is buildable?

What you seem to be building is a huge unaggregated routing system. 
People have considered that before and found that it was 
unimplimentatble. Remember that for every unaggregated prefix in the RIB 
you need an entry in the FIB. The FIB in a core router is high speed 
(expensive) memory and will be replicated on every line card.

I think that you really need to discuss your scaling requirements with 
some hardware
vendors to verify that the design you propose can support the required 
aircraft
population over the expected lifetime of the system.

What I suspect that you need is some sort of formal hierarchy or 
layering in the system to support
long term scaling, and that is going to be a lot easier if it is 
explicitly designed in on day one.

> it would only be for UAVs that take off and land at airports.
Well, you are the application expert here, but that seems to be a 
critical assumption
that you need to test in multiple geographies. Thinking about the London 
airports, that
is quite a restriction.

> For the future,
I am always reminded that when IPv4 was designed, and (in much the same 
era Ethernet)
their address sizes were an approximation to infinity. They are now a 
long way from infinity.
This will be an expensive system to replace so it needs migration path 
to its infinity.

> but we believe these same BGP principles could apply there.

Best regards

Stewart

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 25 Jul 2018 14:13:25 -0700

RTGWG chair hat on

I believe we have reached the agreement to adopt the draft in RTGWG, as it doesn’t bring any changes to the protocol and describes architectural aspects of the solution.
Fred - please work with Stewart on addressing all the comments raised and if there's consensus in wg we will be proceeding with the adoption. 

Cheers,
Jeff

On 7/25/18, 02:13, "Stewart Bryant" <[email-masked]> wrote:

    

    

    

    

    Best regards
    
    Stewart

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 25 Jul 2018 22:22:19 +0000

Hello Stewart,

See below for further follow-up:

Thanks - Fred

> From: Stewart Bryant [mailto:[email-masked]]
> Sent: Wednesday, July 25, 2018 2:14 AM
> To: Templin (US), Fred L <[email-masked]>; [email-masked]
> Subject: Re: [Idr] Rtgdir early review of draft-templin-atn-bgp-07
> 
> 
> 

Jeff Tantsura posted a message on this subject that I will defer to.

> considering using the Public Internet much in the way of SD-WAN.

This is an area that could certainly use more discussion, as we have been
seeing on the list. It occurs to me that, if critical services like online banking
can be made secure and robust while using the public Internet as an
underlay, then it may be that carrying air traffic management services via
secured tunnels over the Internet can also be possible. Failing that, it
seems like the only other option is to run dedicated private fibers with
no connections to the Internet - or are there other methods I may be
unaware of?

> Wow, are you sure that such a system is buildable?

I have proven that a single set of core ASBRs can carry 1M BGP routes
even on lightweight linux virtual machines running Quagga. And, only
a few cASBRs (perhaps as many as 10) need to carry the full routing
table. The stub ASBRs only keep track of their associated Clients, and
configure a default route with a cASBR as the next hop for other prefixes.
So, this is a very leaned-down BGP service in comparison to the way BGP
works in the global public Internet.

> (expensive) memory and will be replicated on every line card.

I have implemented it, and tested with up to 1M unaggregaed prefixes.
And, again, only a few nodes would need to maintain a full RIB/FIB.

Remember also that the global public Internet has some 600K or more
unaggregated prefixes and seems to still be doing fine, so there is
precedence for a single RIB supporting such large numbers. So, if we
deploy 1K independent RIBs then our 1B target seems feasible.

> explicitly designed in on day one.

I understand your concern, but I think we are OK. But, I will bring this
discussion point up with co-authors to make sure we are in agreement.

> that you need to test in multiple geographies. Thinking about the London

It is true that there are lots of airplanes out there, but probably not as many
as most people think. I think the numbers of airplanes deployed worldwide
is on the order of 10^4 - 10^5 - not millions upon millions. So, I think we
could live with a system that scales up to 1B unaggregated prefixes for
a long time. 

> This will be an expensive system to replace so it needs migration path

It is a good point, but up to 1B unaggregaed prefixes already seems pretty
robust for state of the art dynamic routing protocols. So, to go to something
that supports even greater scaling would require either extending our BGP
based architecture (e.g., by adding another tier in the hierarchy) or introducing
a new architecture that scales in a similar fashion as the global DNS.