Subject: Re: [homenet] dst/src routing drafts (for IETF-91 rtgwg)

From: David Lamparter <[email-masked]>
Date: Wed, 22 Oct 2014 21:06:53 +0200

On Mon, Oct 20, 2014 at 10:40:33PM +0200, David Lamparter wrote:
> https://datatracker.ietf.org/doc/draft-lamparter-rtgwg-routing-extra-qualifiers/?include_text=1
> https://datatracker.ietf.org/doc/draft-lamparter-rtgwg-dst-src-routing/?include_text=1

rtgwg & homenet:

So, these drafts describe the general router behaviour for D/S, without
heading into homenet specifics.  There are a few points that might be
worth going into detail:

- the first draft describes adding random kinds of attributes into route
  lookup.  That's very generic, and it didn't add significant
  complexity, but I don't think we're expecting to insert things in the
  middle?

- we haven't discussed some of the boundaries to other functions.
  Recursive route handling is unlikely to be relevant for homenet, but
  might warrant feedback from rtgwg.  And the situation is rather
  complex, the draft has an open "multiple-choices TBD" there.

  Interesting for both rtgwg & homenet are behaviour with uRPF filtering
  and multicast RP lookups.  The draft says:
   for uRPF, flip dst & src and match both, if the system supports it.
     If it doesn't, match packet source against all possible route dst.
     (This probably needs better wording in the draft)
   for mcast reverse-path lookup, ignore all D/S routes.  Multicast has
     only (group, source) available, no destination to match against the
     route source prefix.

So, to be explicit on this, D/S routing is conceptually incompatible
with multicast reverse path lookups.  But that can be dealt with (cue to
homenet multicast discussion here.)

Hoping for feedback (including, possibly, "I don't care" or "I have no
opinion"),

-David

P.S.: there is an unfortunate scheduling collision between homenet and
rtgwg for IETF 91... please convert to mail as much as possible!

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 22 Oct 2014 20:16:29 +0000

>> https://datatracker.ietf.org/doc/draft-lamparter-rtgwg-dst-src-routing/?include_text=1

Speaking strictly for myself, I’m not sure why homenet is relevant. The technology is related to networks that have different routing depend on on one’s use case. A class of solutions for it has been called the “fish” problem, and built using multi-topology routing. In homenet, it’s called SADR, and is primarily about egress routing (routing to an egress to an upstream ISP that gave you a PA prefix). While one doesn’t really want to confuse theory with practice, in theory it could be used between points of a network, to prevent folks using one set of prefixes to talk with another set, or to force routing of some sessions in some ways.

Personally, those are a class of problems I associate with campus networks more than residential networks.

---

From: Ole Troan <[email-masked]>
Date: Tue, 28 Oct 2014 14:06:26 +0100

Fred,

> Personally, those are a class of problems I associate with campus networks more than residential networks.

why homenet is relevant?
isn't multi-prefix multi-homing one of the most obvious use cases for source address dependent routing? that's not restricted with homenets, but also any small network. I'm assuming large networks will continue with PI addresses and BGP based multihoming.

cheers,
Ole

---

From: Gert Doering <[email-masked]>
Date: Tue, 28 Oct 2014 14:18:03 +0100

Hi,

On Tue, Oct 28, 2014 at 02:06:26PM +0100, Ole Troan wrote:
> isn't multi-prefix multi-homing one of the most obvious use cases
> for source address dependent routing? that's not restricted with
> homenets, but also any small network. I'm assuming large networks
> will continue with PI addresses and BGP based multihoming.

I see the distinction somewhere in the "what defines policy in a network",
as in:

 - in a "big" company network, there usually is some sort of network access
   policy, which is defined and enforced by the network people - so you'll
   see BGP policies and PI, or in a "dual /48" style shop, I expect to see
   dual-NPT66-with-ULA devices where you can configure your policy ("surfing
   goes out via ISP A, mail goes out via ISP B") in the network device

 - in a homenet, small company ("barber shop") network, there is no "admin",
   and to try to enforce network policy ("surfing via cable, bittorrent via
   DSL") in the network device is futile, because "no admin".  So here you
   need SADR to empower the end device - and by that means, the user - to
   define policy.  Have the bittorrent client use the source address from
   the DSL ISP's /48, the web browser use the source address from the 
   Cable ISP, and things work the way the user wants it.

   ... and this is why I really really like dual-/48 multihoming for the
   "not really managed" SOHO case, as it puts policy decisions where they
   can be made (= user's device) - and of course, why dual-/48 is not going
   to fly for enterprise networks (= can't have the user decide that).

(In case this wasn't obvious, I'm agreeing with Ole here, I just try to
shed light from a slightly different angle on that.)

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Tim Chown <[email-masked]>
Date: Tue, 28 Oct 2014 19:44:45 +0000

> Personally, those are a class of problems I associate with campus networks more than residential networks.

I suspect most campus networks will have a single /48 from their NREN, or a /48 PI. I’m not aware of any campuses in the UK which are multi-homed with dual /48, though I understand some Internet2 campuses may be. 

(As an aside we’ve just run out of our v4 /16…)

Homenet has ‘designed in’ future multihoming, which is rare today. In the homenet case the multihoming may presumably be more commonly for different services to the home?

Tim

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 29 Oct 2014 08:52:18 +1300

> isn't multi-prefix multi-homing one of the most obvious use cases for source address dependent routing? that's not restricted with homenets, but also any small network. I'm assuming large networks…

If you mean by "large" the few ten thousand largest networks in the
world, that will not cause significant BGP4 bloat by having PI, then
yes, I fully agree. Any medium to small network should expect to have
multiple prefixes in the long run (we are not talking about today, but
a time when all we old folks who remember the IPv4 world have retired
and gone fishing).

   Brian

---

From: David Lamparter <[email-masked]>
Date: Wed, 29 Oct 2014 07:28:37 +0100

> a time when all we old folks who remember the IPv4 world have retired

What I'm personally wondering most in this regard is: to what extent
will larger networks deploy multiple prefixes to the hosts?

I can certainly see it in small to medium businesses that maybe just got
their second uplink at their headquarters and opened 3 shops or
whatever.  But at some point I'd guess policy/enforcement replaces
multiprefix deployments, as Gert wrote in his mail.  Then again, some
shiny new things like MPTCP would benefit from multiprefix deployments,
which might be an argument for some medium-sized users to deploy this?

FWIW, I understand Fred's question for the relevance of homenet as
asking whether these drafts should explicitly target homenet or not, and
I agree that they shouldn't.  Even if 90% of SADR use-cases end up to be
"bread and butter" homenets, the remaining 10% are valid too.

-David

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 29 Oct 2014 06:50:17 +0000

> will larger networks deploy multiple prefixes to the hosts?

Well, define “larger”. Any network that gets a PI prefix is unlikely to deploy multiple prefixes. The question is at what size network is makes sense to obtain an AS number and a PI prefix, and use BGP to talk with one’s upstream. Wherever that boundary is, below that networks will use PA prefixes. The question then becomes: will they multi home?

And I think the answer today is that we don’t know the answer.

---

From: Ray Hunter <[email-masked]>
Date: Wed, 29 Oct 2014 13:05:01 +0100

Fred Baker (fred) wrote:
> On Oct 28, 2014, at 11:28 PM, David Lamparter<[email-masked]>  wrote:
>
> Well, define “larger”. Any network that gets a PI prefix is unlikely to deploy multiple prefixes. The question is at what size network is makes sense to obtain an AS number and a PI prefix, and use BGP to talk with one’s upstream.
I don't agree with this statement for the following reasons.

Availability: There are many enterprises that have very numerous 
far-flung sales-office type locations which do not host any critical 
data or applications, but which could benefit from higher availability 
than that provided by a single ISP provider (some of which are currently 
served by a specialised box offering a Very Small Office Service running 
dual IPSec tunnels to a central site, which then performs the break out 
to the corporate intranet/Internet)

Latency: There are many sites which could benefit from local Internet 
breakout to regional cloud services, where you don't want to suffer the 
latency associated with a back haul from an office in Australia to a 
regional hub in Hong Kong, or even East coast US to West coast US and 
back. You'd still also want some back up via the central breakout if the 
local breakout failed.

Cost: There are cost savings to be made in many countries where private 
network services are still many orders of magnitude more expensive than 
plain old Internet. So Internet offload for non-mission-critical traffic 
can be very attractive. If you could achieve this via direct host-server 
connections using address selection rules or multipath TCP; rather than 
via PBR, GRE tunnels + NAT, that would be a lot simpler.

>   Wherever that boundary is, below that networks will use PA prefixes. The question then becomes: will they multi home?
>
This I agree with.

-- 
Regards,
RayH

---

From: Jim Gettys <[email-masked]>
Date: Wed, 29 Oct 2014 11:13:37 -0400

On Wed, Oct 29, 2014 at 8:05 AM, Ray Hunter <[email-masked]> wrote:

>> Well, define “larger”. Any network that gets a PI prefix is unlikely to
>> deploy multiple prefixes. The question is at what size network is makes
>> sense to obtain an AS number and a PI prefix, and use BGP to talk with
>> one’s upstream.
>>
> Availability: There are many enterprises that have very numerous far-flung
> sales-office type locations which do not host any critical data or
> applications, but which could benefit from higher availability than that
> provided by a single ISP provider (some of which are currently served by a
> specialised box offering a Very Small Office Service running dual IPSec
> tunnels to a central site, which then performs the break out to the
> corporate intranet/Internet)
>

​
​
​And we are now deploying home networks that are being used for home health
medical monitoring; having full fail-over to another ISP is in the process
of becoming a "life" issue​.
​
​

> regional hub in Hong Kong, or even East coast US to West coast US and back.
> You'd still also want some back up via the central breakout if the local
> breakout failed.
>
> connections using address selection rules or multipath TCP; rather than via
> PBR, GRE tunnels + NAT, that would be a lot simpler.

​

>
>
>    Wherever that boundary is, below that networks will use PA prefixes.
>> The question then becomes: will they multi home?
>>

​Ditto.  Though I have a personal opinion that they will... How else can
you be able to test that things are actually going to "work" in the face of
loss of one ISP? Anything not being tested on an ongoing basis is unlikely
to work in the case of failure.  If availability really matters, you care...

Jim

​

>
>
>
> --
> Regards,
> RayH
>
>
> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet
>

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 29 Oct 2014 16:09:15 +0000

On Oct 29, 2014, at 5:05 AM, Ray Hunter <[email-masked]> wrote:

> Availability: There are many enterprises that have very numerous far-flung sales-office type locations which do not host any critical data or applications, but which could benefit from higher availability than that provided by a single ISP provider (some of which are currently served by a specialised box offering a Very Small Office Service running dual IPSec tunnels to a central site, which then performs the break out to the corporate intranet/Internet)
> 
> Latency: There are many sites which could benefit from local Internet breakout to regional cloud services, where you don't want to suffer the latency associated with a back haul from an office in Australia to a regional hub in Hong Kong, or even East coast US to West coast US and back. You'd still also want some back up via the central breakout if the local breakout failed.
> 
> Cost: There are cost savings to be made in many countries where private network services are still many orders of magnitude more expensive than plain old Internet. So Internet offload for non-mission-critical traffic can be very attractive. If you could achieve this via direct host-server connections using address selection rules or multipath TCP; rather than via PBR, GRE tunnels + NAT, that would be a lot simpler.

I have a hunch we’re talking past each other. We would agree, but we’re discussing different cases. I was discussing a case in which a company operates a single addressing plan. I think you’re discussing a company that operates multiple addressing plans, one for each of several locations.

Cisco might be an example of a “larger” company. We have an IPv6 /32, more locations than you can shake a stick at, and maybe 20 DMZs, places where our network interconnects with the Internet. I don’t know exactly what we advertise or where, but I could imagine that a few of our DMZs advertise the /32 and each of our DMZs advertises a /40, or something along that line. For us, PI is “duh, obvious”, and the only case in which we might - *might* - have a second prefix would be a ULA in a lab.

I suspect the company you are discussing might have a number of small offices in as many cities, and as many PA prefixes as it takes. The company might also have a PI prefix, but I would be surprised if it used the PI prefix in all of its little offices; if it did, it would essentially use it for internal traffic, and have some sort of VPN connecting the offices on which it was used. It would, however, use the PA prefixes from the offices when they need to talk outside the house. If it is using the PI prefix plus a PA prefix in any given office, it would depend on RFC 6724’s Rule 8 (longest match) to prefer a PI address when talking to another address within the prefix, and a temporary address from PA space otherwise.

In any event, in the context of my comment above, I am thinking of the many small offices as separate networks. They don’t justify their own PI spaces; the PI in context acts more like PA from the central corporation than PI in the traditional sense of the term. They might well get PA from one or more ISPs, and they might have something akin to PA from their central office or whatever.

Coming back to the question - “would they put multiple subnet prefixes on any given LAN” - the question really comes back to their requirements and stomach for complexity. If they require multihoming, such as Jim suggests, I would hope they would do that. Among my “NAT == Security” customers and SEs, getting them there requires their network to work well (egress routing being one of many important parts of that), and a good security solution that doesn’t involve NAT. Until they have one AND they are convinced, they tell us that the lack of a NAT solution is a reason to delay deployment. I don’t think that last is about the number of prefixes around; it’s about security policy and mindset.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 29 Oct 2014 14:24:03 -0400

Tim Chown <[email-masked]> wrote:
    > Homenet has ‘designed in’ future multihoming, which is rare today. In
    > the homenet case the multihoming may presumably be more commonly for
    > different services to the home?

It's true that real multihoming (in the home...) is rare.

There are a multitude of off-the-shelf SOHO routers which support multiple
uplinks via NAPT.  I have setup this kind of thing up for a few dozen
friends' companies over the years... usually via a cablemodel + DSL (from
different suppliers) because we are protecting from a combination of
backhaul events and "customer service".

All of the downtime I've seen in the past ten years in my home office has
been due to "initiatives" from "customer service".

So I am very happy with the features that we have built-in, and I think with
the growth of 3G we will see additional mechanisms.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: David Lamparter <[email-masked]>
Date: Thu, 30 Oct 2014 01:39:33 +0100

[plucking a paragraph from the middle]
On Wed, Oct 29, 2014 at 04:09:15PM +0000, Fred Baker (fred) wrote:
> I suspect the company you are discussing might have a number of small
> offices in as many cities, and as many PA prefixes as it takes. The
> company might also have a PI prefix, but I would be surprised if it
> used the PI prefix in all of its little offices; if it did, it would
> essentially use it for internal traffic, and have some sort of VPN
> connecting the offices on which it was used. It would, however, use
> the PA prefixes from the offices when they need to talk outside the
> house. If it is using the PI prefix plus a PA prefix in any given
> office, it would depend on RFC 6724’s Rule 8 (longest match) to prefer
> a PI address when talking to another address within the prefix, and a
> temporary address from PA space otherwise.

If there is an overlap of a company-wide numbering plan with local
connectivity, that might actually be use-case for a SADR route whose
destination isn't ::/0.  Though you could always do that with a simple
destination route for the company-wide prefix, what you can now do is
signal the correlations between routes.  (Except we don't have a
protocol to communicate this to the host yet.)

I'm imagining a route table like this:
(:cccc: being the company-wide PI)

::/0 from 2001:db8:1::/48 via PA-provider-1
::/0 from 2001:db8:2::/48 via PA-provider-2
::/0 from 2001:db8:cccc::/48 unreachable
2001:db8:cccc::/48 from 2001:db8:cccc::/48 via IPsec-gateway
2001:db8:cccc::/48 from ::/0 unreachable

Where the last route would prevent accidental leaking of packets onto
the internet in case the IPsec gateway malfunctions.  (The 3rd route is
redundant if there's no "::/0 from ::/0")

But - apart from ease of use for multiple prefixes, this can be done
without SADR just fine, the only advantage is that there's full
information regarding which source addresses work with which
destinations.  If we get that to hosts, and into their source address
selection, then we won something.

(And this is really the same as homenet walled-garden scenarios, where
an ISP uses a separate prefix for some [IPTV, whatever] service and
expects clients to use a distinct source prefix to get to that service.
Then again, "secret gardens are better than walled gardens.")

-David

---

From: Ole Troan <[email-masked]>
Date: Thu, 30 Oct 2014 08:29:37 +0100

David,

> selection, then we won something.

in a simple triangle topology:

   |            |
CE1      CE2
  \             /
   \           /
       IR1
         |
-----------------
   |
 Host

with PA1 and PA2 doing BCP38 filtering and a non SADR capable network, I would assert that you cannot do this without SADR.
IR1 load balances, and you could end up in a situation where IR1 load balancing would pick CE1 exit for PA2 source address, and CE2 exit for PA1 source address. with the result that whatever the host tried to do, all traffic would be black-holed by the ISP ingress filters.

with regards to getting SAS policies into hosts, that isn't necessary for the multi-homed to congruent networks case. it is needed in the walled garden cases (but there are some many other issues with walled gardens that I'm not sure the IETF should try to fix a broken business model). we do have a DHCP option already to configure a host's SAS table for this purpose. RFC7078.

cheers,
Ole

---

From: Ray Hunter <[email-masked]>
Date: Thu, 30 Oct 2014 23:10:51 +0100

> Fred Baker (fred) <mailto:[email-masked]>
> 29 October 2014 17:09
> On Oct 29, 2014, at 5:05 AM, Ray Hunter<[email-masked]>  wrote:
>
>> Fred Baker (fred) wrote:
Quite possibly.
> Cisco might be an example of a “larger” company. We have an IPv6 /32, more locations than you can shake a stick at, and maybe 20 DMZs, places where our network interconnects with the Internet. I do…
Well I think it's instructive to look at the current IPv4 unicast 
routing table and ask whether replicating this 1:1 into IPv6 is what 
enterprises really want.

Multihoming is far more than Internet.

External "3rd Party" networks are often as big as, or even bigger than, 
an enterprises own Intranet.

If I look at some customer's current IPv4 routing tables, they contain: 
3rd party management companies for IT, speciality partners, per-project 
collaboration partners, private cloud, public cloud, merger demerger and 
assimilation targets, payment companies, 3rd party management companies 
for specialist equipment .... hundreds of prefixes that aren't even 
managed by the enterprise in question, and with high churn rates too.

If I ask myself: do all product divisions of the business really want 
all of these differing communication requirements to get funneled 
through exactly the same handful of 3rd party touchpoints/DMZs to the 
outside world on a per region basis, I think the answer would 
emphatically be "no".

IMHO Destination based routing (and ensuring balanced/ symmetric return 
routes for your own prefixes) imposes unnatural constraints on these 
touchpoints/DMZs, compared to what the business expects.

Why do you want to share a route to your crown-jewels /32 prefix, also 
used for your sales/finance platform, with someone who just manages some 
print servers for one product division on a number of sites via a 
dual-redundant 1Mbps link, whilst you have 10Gbps to the Internet? What 
happens if this 3rd party print server management company leaks your /32 
elsewhere via BGP into another AS? e.g. to another customer where they 
also manage their print servers, and who in turn are an important 
customer of your own products and want to connect to your front end 
sales site (via another higher-speed link).

It remains to be seen whether multiple-prefixes and SADR would help 
reduce this complexity, or just shift it around.

e.g. having a different prefix M assigned to a logical management 
interface on all print servers managed by party M, in addition to a 
prefix C used by the internal customers to connect to the business 
interface of the same servers.

You could have a default route for traffic sourced from prefix M 
pointing to the server management company via one DMZ to company M, and 
a different default route for customers prefix C out of another e.g. 
Internet. The management company M would only need a return route to the 
specific M prefixes from their network located in your network, and 
would not need to learn your /32 C prefix at all. They in turn also 
wouldn't have to advertise their crown jewels /32 into your network. And 
firewall rules and route filter rules become "allow M to M on one DMZ 
gateway", rather than a spaghetti of 10000 individual rules from 
prefixes A-Z to prefix C that no-one understands any more.

I do know that trying to encode all of the regions/countries/business 
units requirements in the 32 bits available today in IPv6, whilst 
attempting to filter using BGP, just doesn't get anywhere near to 
expressing the current natural level of business complexity and 
availability requirements (assuming /32 RIR allocation and /64 LANs). 
And maintaining the related firewall rules and route filters is 
currently a nightmare.

-- 
Regards,
RayH