Subject: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

From: Toerless Eckert <[email-masked]>
Date: Thu, 22 Sep 2016 08:55:40 +0200

On Wed, Sep 21, 2016 at 12:27:42PM +0200, IJsbrand Wijnands wrote:
> Hi Xiaohu,
> 
> I would say that one of the big benefits is that an IPv6 BIER encoded packet can be translated/transformed into a set of IPv6 Unicast packets (just one bit set) at the egress of the network very easily. This is very beneficial when running BIER between servers and avoids the need for MLD and DR/DF election between the Server and the router.

Is this the example pitch: ?

                                        |-- Rcv1
                            --- BFER1 --|
                           /            |-- Rcv2
      Src ---- Rtr --- BFIR 
                           \            |-- Rcv3
                            --- BFER2 --|
                                        |-- Rcv4
					|
					|-- Rtr2  ---- Rcv5

Src sends one IPv6 unicast packet indicating (Rcv1,Rcv3,Rcv4). It will have a prefix
owned by BFIR. There it will be replicated by BIER. Copies arriving
at BFER1,BFER2 will be converted back to IP unicast packets, one copy to Rcv2 from
BFER1, one from BFER2 to Rcv3, one from BFER2 to Rcv4.

I like the value proposition, but i thought that this end-to-end BIER mode was
still out of charter scope ? (would be great if that was not true anymore).

How about supporting remote receivers like Rcv5 ? Sources can be remote so
it would make sense to allow bidirectional support. Might need some signaling
to BFER2 to map a remote IP address to a bit.

Cheers
    toerless

> 
> Thx,
> 
> Ice.
> 
> > On 21 Sep 2016, at 12:09, Xuxiaohu <[email-masked]> wrote:
> > 
> > Hi Ice,
> > 
> > If I understand your draft correctly, it talks about embedding BIER within IPv6 rather than transporting BIER over IP. The beauty of this approach is that all information needed for BIER forwarding is now directly contained in the received BIER packet. As a result, there is no need to retrieve them from the MPLS-BIER label and distribute the MPLS-BIER label mappings. However, what's the major advantage over the generic BIER header as defined in https://tools.ietf.org/html/draft-xu-bier-encapsulation-05 from your point of view?
> > 
> > Best regards,
> > Xiaohu
> > 
> > -----????????????-----
> > ?????????: BIER [mailto:[email-masked]] ?????? IJsbrand Wijnands
> > ????????????: 2016???9???20??? 22:30
> > ?????????: Eric C Rosen
> > ??????: [email-masked]; Pierre Pfister
> > ??????: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt
> > 
> > Hi Eric,
> > 
> > Many thanks for your comments. A few responses inline.
> > 
> >> Sometimes an idea that seems clever at first just doesn't work out when the details are examined.  The idea of using addresses that are unicast addresses in one context but multicast addresses in another falls into this category.  Inevitably a host will put a multicast address into an IP source address field, because the host
> > 
> > Maybe we can consider registering a new IPv6 address space for BIER. I think a lot of your concerns would be addressed. That way we don???t get in the way with already defined unicast behaviour and still leaves it open to treat the packet as unicast in some specific cases, like with tunnelling through non-BIER nodes.
> > 
> >> Any BIER encapsulation needs to be able to support the features that are in the architecture.  
> > 
> > And we should not be afraid to allow exceptions and/or updates to the architecture draft if necessary.
> > 
> >> An implementation that doesn't support SI is not compliant with the architecture.
> > 
> > By allocating part of the ???prefix??? for the SI/SD identifier we can make this work just fine IMO. Since the SI/SD here is manually defined and the BitString length is fixed, I think there is really only need for one of them. If we call that SI or SD (or something else) we can argue about :-)
> > 
> > Thx,
> > 
> > Ice.

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 23 Sep 2016 20:17:37 +0200

On Thu, Sep 22, 2016 at 03:22:35PM -0400, Eric C Rosen wrote:
> Toerless, nice example.  But let's look at it more closely.
> 
> >                                         |-- Rcv1
> >                             --- BFER1 --|
> >                            /            |-- Rcv2
> >       Src ---- Rtr --- BFIR
> >                            \            |-- Rcv3
> >                             --- BFER2 --|
> >                                         |-- Rcv4
> >					|
> >					|-- Rtr2  ---- Rcv5
> >
> >Src sends one IPv6 unicast packet indicating (Rcv1,Rcv3,Rcv4). It will
> >have a prefix owned by BFIR.
> Actually, in this example, I think Src is the BFIR, what you call
> BFIR is actually just a BFR at the border of a BIER domain.

I assume your argument is that in the current BIER arch, Bits
are allocated to BF(E)R, and in this model Bits are allocated to Rcv's,
therefore those have to be called BF(E)R, and accordingly Srcs 
have to be called BFIR. Yes ?

The current BIER arch is a bit like MPLS: It does not have a strong
differentiation of host vs. router (ES vs. IS). Ingres/egres LSR/BFR
still need to operate most/all of the IS functions.

In this proposal, BIER becomes end-to-end and Src/Rcv are really only
ES/hosts, which is why i'd rather not want to call them BFR. Rather call them
BIER-Hosts. But they are the entities to which Bits are assigned.

> What does "owned by BFIR" mean?  Is the prefix jointly "owned" by
> all the BFRs bordering the BIER domain?  What about non-BFRs
> bordering the BIER domain?  What keeps the packet from going to them
> instead?

I'd call a BFIR in this model a BFR that can receive packets with
a BIER destination IPv6 address from routers unaware of BIER or from
hosts.

"owned" just means that the BFIR has set up IPv6 unicast routing,
so that the prefix will be routed by non-BFRs to this router. Aka:
prefix is assigned to some loopback interface on the BFIR, announced
into IGP/BGP and arriving packets trigger BIER processing.

In the most simple case, the "BIER IPv6 prefix" is "owned" by all
the BFIR of the BIER domain, aka: it is anycast, so that packets
will be routest to the nearest BFIR and will then become BIER
forwarded inside the BIER domain.

In a more complex case, one could consider that BFIR would "own"
the "BIER IPv6 prefix, SI bits" prefixes (aka: announce prefixes including
the SI bits differently, and each BFIR decides which of these prefixes
to announce.

> If Src thinks that the prefix is on the local network, it won't send
> the packet to Rtr, it will instead try to figure out the unicast MAC
> DA corresponding to the IP destination address.  This DA doesn't
> exist, so the packet will get dropped.  But if Src doesn't think
> that the prefix is on the local network connecting it to Rtr, it may
> send the BIER packet out a different interface.  Where will the
> packet end up then?

Right. Practical example: Rcv1 wants to send a BIER packet
that should reach Rcv2 and Rcv3. We have one Prefix = "BIER IPv6 prefix / SI",
every Src/Rcv in the picture has a bit in the Bitstring following that
Prefix. How to forward ?!:

Every Src/Rc (Host) will have a /128 address allocation for
it's address 'Prefix,My-Bit'. Therefore any packet sent to a different
BitString will just follow the default-route == go to the Default-router,
and from there on (remote case) to the closest BFIR. Assuming the
default-router is not a BFIR. Result: BFER would receive packet, do
BIER replication, create one copy bck to Rcv2, and one down to BFIR to
reach Rcv3.

> What if Src is doing load balancing among the default routers on the
> LAN, or among its interfaces?  For a given flow, the "IPv6
> destination address" will change from time to time, which means the
> load balancing hash for that flow will change from time to time.
> This is the "non-deterministic ECMP" problem discussed in the
> architecture.  But since Src doesn't actually support BIER, it
> cannot use the solution described in the architecture draft.

I have not tried to track the relevance of the ECMP requirements,
so i would appreciate the most simple relevant example to discuss better.

Quick thoughts:
  Non-BIER ECMP: between SRc...Rtr..BFIR and between BFER...(Rtr)...Rcv:
  It's just like any othre IP ECMP problem, eg: if necessary do
  (Src,Dst, Proto,{SrcPort,DstPort}) ECMP ?

  BIER-ECMP: No entropy field, so definitely a good question.

> What if there are receivers on the same LAN as the Src?  How will
> those receivers get the BIER packets sent by the Src?  Since Src
> thinks the BIER packet is a unicast packet, it will encapsulate it
> in a unicast ethernet frame whose MAC DA is a MAC address of Rtr.
> Since Rtr is not a BFR, Rtr will remove the ethernet header and
> forward the IPv6-BIER packet towards (let's say, for the sake of
> argument) the node you call BFIR.  How will BFIR get any copies of
> that packet back to the receivers that are on Src's subnet?  Sending
> the BIER packet (with modified BitString) back to Rtr will cause a
> loop.
                                |--- Host1 (Src/Rcv)
              BFR/  ---- Rtr ---|
	(eg: BFIR/BFER)         |--- Host2 (Src/Rcv)
	                        |--- Host3 (Src/Rcv)

Host1 sends BIER packet, Host2 and Host3 bits are in the BitString.
Prefix of the BIER packet gets routed by Rtr to BFR. BFR performs
BIER, creates separate copies to Host2 and Host. On Rtr, we need /128
routes for Host2 and Host3.

Host1 sends BIER packet with ONLY Host3 as receiver. E voila, it
gets turned around by Rtr because it matches the /128 route.

> Suppose Src had an IP multicast packet to send, and encapsulated it
> in an IPv6-BIER header.  In that case, a BFER needs to decapsulate
> the packet and multicast the payload to the LAN on which the
> receivers reside.  How does a BFER know whether to (a) decapsulate
> and multicast or (b) whether to replicate, overwrite the destination
> address, and unicast?   I hope the answer isn't that the BFER needs
> to inspect the payload and guess what the hosts are expecting.

Well, i like the idea of addressing the hosts directly, also as he
draft outlines, so no IP multicast packets involved.

A more pragmatic option is of course to allocate bits to BFER. When
the BFER sees its own bit, it just continues normal BFR operations:
pass packet to the protocol indicated in the protocol field. Aka: IP-in-IP:
decap, pass IP multicast encapsulated packet to IP multicast routing.

Not sure how much value is left in this case over the more traditional
options of having a full fledged separate BIER header.

> BFER2 needs to know which BFR-ids represent hosts on any particular
> LAN to which it is attached.  A BFR-id is just a shorthand for a
> BFR-prefix, i.e., for an IP address.   If BFER2 wants to use unicast
> replication to transmit on the last hop, why not just use the actual
> IP addresses of the receivers?

Except for the "BIER-IPv6-address" as the destination address in a
received packet, there is no indication for a recevier that this 
is a BIER packet instead of a unicast packet.

On Linux i would probably want to configure the network stack so that
this address is not part of the default network context, but would only
be useable to BIER apps. This would also prohibit that someone trying
to attack BIER, eg: TCP packets with BIER destination address would
NOT be able to reach application code that was not meant to handle BIER
packets.

If we had another way to recognize BIER packets on receivers, then i
probably would also prefer to only use hosts unicast destination addresses
from BFER to receiver. Should make things simpler.

> I don't see how BFER2 can get a BIER packet to Rcv5.  Rtr2 will at
> best loop the packet back to BFER2, and at worst the packet will get
> forwarded (through a route not necessarily shown in the above
> picture) all the way back to "BFIR", which "owns the prefix".

See above. Different prefix-length routes (prefix vs. /128).

But thats just me, Pierre didn't include any such remote receivers.
If we figure we don't need to support them, that would make things
simpler.

> Each BFR and non-BFR can have a host route for every possible
> receiver, and some magic can be used to associate a BFR-id with each
> host route.  That would eliminate some of the routing and looping
> issues.    There might be a few scaling issues though.

Right.

> BTW, if one wants the BFR-ids to represent receiver hosts, one is
> going to need a lot more BFR-ids than if the BFR-ids represent
> BFERs.  So one would certainly  want to use an encapsulation that
> allows more than a 64-bit BitString.  One could try to compensate
> for the very short BitString by having lots of sub-domains.  Then
> for each application (or perhaps each flow) one could try to assign
> the BFR-ids densely.   But we've heard that sub-domains aren't
> necessary for this encapsulation.  More importantly, if a short
> BitString leads to more sub-domains, we will see an increase in the
> operational complexity and signaling load of the scheme.

Right. This can best be vetted with explicit examples. 

Toerless