Subject: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

From: Pierre Pfister <[email-masked]>
Date: Fri, 2 Sep 2016 09:33:30 +0200

Hello Bier addicts,

Ice and I just published a draft detailing a possible way to do BIER over IPv6.
The purpose of this draft is to be used as input to the BIER interim meeting and 
feed discussions regarding the future BIER encap. under discussion.

Questions, comments and flames are, as always, very welcome.

Cheers,

- Pierre

> 
> 
> A new version of I-D, draft-pfister-bier-over-ipv6-00.txt
> has been successfully submitted by Pierre Pfister and posted to the
> IETF repository.
> 
> Name:		draft-pfister-bier-over-ipv6
> Revision:	00
> Title:		An IPv6 based BIER Routing Underlay
> Document date:	2016-09-02
> Group:		Individual Submission
> Pages:		6
> URL:            https://www.ietf.org/internet-drafts/draft-pfister-bier-over-ipv6-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-pfister-bier-over-ipv6/
> Htmlized:       https://tools.ietf.org/html/draft-pfister-bier-over-ipv6-00
> 
> 
> Abstract:
>   This document specifies a routing underlay which enables transport of
>   IPv6 payloads to multiple IPv6 destinations using the Bit Index
>   Explicit Replication (BIER).  The BIER bits are stored within the
>   low-order bits of the IPv6 destination address while the high-order
>   bits are left untouched and used in order to decide whether an IPv6
>   packet is a regular IPv6 packet or an IPv6 BIER packet, and if so, to
>   identify the BIER set identifier.
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
>

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 12 Sep 2016 15:26:07 -0400

Here are some comments and questions on draft-pfister-bier-over-ipv6.

Despite the title and abstract, the draft has nothing to do with the 
routing underlay.  It's about procedures for the BIER layer. Section 4 
is about BIER layer functions at the BFIR, and section 5 is about BIER 
layer functions at other BFRs.

The draft says in some places that BIER packets are "regular" IPv6 
packets, but it says in other places that they are not "regular" IPv6 
packets.  "Regular" is never defined, but it should be used consistently.

The term "BIER bits" occurs in contexts where it seems to mean "BIER 
BitString".  (The bits representing the SI are also "BIER bits", but 
don't seem to be included as "BIER bits" in this draft.)

The draft says that the high-order 64 ("p+i") bits of the IPv6 
destination address field are "left untouched".  But it proposes using 
the high-order 56 ("p") bits to hold a special prefix meaning "BIER", 
and it proposes using the next 8 ("i") bits to represent the SI.  That 
doesn't seem like leaving the high-order bits untouched. But perhaps 
"untouched" means "the bits are not modified by the forwarding plane" 
rather than "the functionality of the bits is not modified".

It is worth emphasizing that, unlike other proposed encapsulations, this 
encapsulation only supports BitStrings of length 64.  Well, 128-p-i.  I 
guess one could use a /40 prefix and a single SI to get a 
BitStringLength of 88 if one doesn't need any more than 88 BFR-ids in 
the domain.  It still seems rather restrictive when compared to the 
other encapsulations that have been proposed.

I don't see how the sub-domain is identified in this encapsulation.

The draft says: "It makes use of a typical IP longest match in order to 
decide whether a packet is a BIER packet or not, which means hardware 
and software existing solutions may be used for that purpose."  But 
existing/deployed hardware and software solutions cannot do the BIER 
forwarding!  So I don't see what advantage is being claimed here.  Or is 
this just a way of saying "at least we don't use IPv6 extension headers" ;-)

The draft seems to suggest that if the payload is an IPv6 packet, one 
can use BIER to multicast it without encapsulating the payload at all.  
Rather, all one has to do is overwrite the enduser's destination address 
with an "address" value that contains the BIER prefix, SI, and 64-bit 
BitString.  Do I understand that correctly? If so, this seems to create 
a number of issues that are not discussed.

Suppose, for example, that the original payload has an IPv6 multicast 
address in its destination address field.  If that is overwritten, then 
when the packet reaches a BFER, the BFER will have no way of knowing 
what the original destination address was.  But the multicast flow layer 
needs that information in order to further process the packet!

If an enduser's UDP packet is carried as the payload of an IPv6 BIER 
packet, and the UDP packet has non-zero checksum, then, as the draft 
says, "the UDP checksum must be recomputed when the BIER bits are 
changed."  So the BIER forwarding plane now has a new function -- it has 
to look to see what the payload is, and whether the the payload has a 
checksum that needs to be adjusted.  Does that sound like a good idea?

The draft says: "It is possible to configure a host with an address 
which corresponds to a BIER address with a single bit set.  From the 
host perspective, such address is not different from a regular IPv6 
address."  Well, what will then stop the host from using that address as 
an IPv6 source address?

Packets carrying such an IPv6 source address may get dropped for using a 
prefix that has not been delegated to the host's subnet.

If a host uses a "BIER address" as its source address in a given packet, 
and the packet doesn't get dropped, the packet may elicit responses that 
put the BIER address in their destination address fields.  This seems 
like a security issue, as it creates an attack vector that can create 64 
responses to a single probe.

Hopefully hosts will not treat the BIER prefix as a delegated prefix 
that they can use to compute their own "privacy addresses"; if any of 
those computed addresses make it to the source address field of an IPv6 
packet, and the packet does not get dropped, chaos could ensue.

The draft suggests that the BIER prefix can be treated by BFRs outside a 
given domain as a routable unicast prefix, presumably leading towards 
one or more BFRs that serve as border routers for a given BIER domain.  
(At one point, the draft even says that BIER packets are IPv6 unicast 
packets, which seems a bit misleading. ;-)) That's interesting, but 
putting this feature to use requires one to work out quite a few details 
that are not mentioned.  And I don't think it will work for non-BFRs 
within the BIER domain.  To pass BIER packet through non-BFRs within the 
destination BIER domain, one still needs additional encapsulation.

The draft talks about "configuring" a BIER router with the rules needed 
to interpret a given IPv6 destination address field.  Does this mean 
that IGP/BGP extensions for signaling BFR-ids and other BFR parameters 
will not be used?  Why?  Or is it allowable for these parameters to be 
signaled as well as configured?  And if we are going to talk about 
inter-domain BIER, how is the inter-domain signaling done?

It is not clear from the draft whether the value of i (number of bits in 
the SI field) and the set of BFR-ids is specific to a given prefix, or 
whether it is the same for all prefixes.  (Or for all prefixes 
corresponding to a given domain.  Or perhaps sub-domain?)

Section 6 gives the "advantages" of the proposal, but it is not clear 
what the proposal is being compared to.  For instance, one of the 
advantages is claimed to be that the proposal doesn't use IPv6 extension 
headers.  Is the proposal being compared to another proposal that does 
use IPv6 extension headers?  If so, where is that proposal described?  
(None of the proposals I've seen so far use IPv6 extension headers.)

Another "advantage" is that the proposal "may be used for transporting 
IP multicast packets, but also for sending IP payloads directly to 
multiple destinations".  Does this "advantage" distinguish it from some 
other proposal?  I'd have thought that any multicast encapsulation can 
cause any packet to sent to multiple destinations.

Of course, maybe the first question is whether we really have a need for 
an encapsulation that is IPv6-specific.  The draft doesn't actually seem 
to address this question.

---

From: Pierre Pfister <[email-masked]>
Date: Tue, 13 Sep 2016 10:39:46 +0200

Hello Eric,

Thanks for those comments.

You found a few quite interesting ambiguities that I will clarify in the next iteration.
But the good news is that those nits did not led you to not understand something.

See the rest inlined.

Cheers,

- Pierre

> Le 12 sept. 2016 à 21:26, Eric C Rosen <[email-masked]> a écrit :
> 
> Despite the title and abstract, the draft has nothing to do with the routing underlay.  It's about procedures for the BIER layer. Section 4 is about BIER layer functions at the BFIR, and section 5 is about BIER layer functions at other BFRs.

Indeed. This is all about BIER Layer. It will get fixed.

> 
> The draft says in some places that BIER packets are "regular" IPv6 packets, but it says in other places that they are not "regular" IPv6 packets.  "Regular" is never defined, but it should be used consistently.

The point is to explain that such packets look like IPv6 packets, but are not to routers that have been configured to perform BIER operations on the BIER packet (And it knows that by matching the destination address to one of the BIER prefixes).
I will try to find a better wording than 'regular'. 

> 
> The term "BIER bits" occurs in contexts where it seems to mean "BIER BitString".  (The bits representing the SI are also "BIER bits", but don't seem to be included as "BIER bits" in this draft.)

I looked at the architecture draft but didn't find any place where SI value is referred as 'bits'.
If your point is that 'BIER bits' is not defined, I can live with it and change it to "BIER BitString" in the document. For the sake of clarity.

> 
> The draft says that the high-order 64 ("p+i") bits of the IPv6 destination address field are "left untouched".  But it proposes using the high-order 56 ("p") bits to hold a special prefix meaning "BIER", and it proposes using the next 8 ("i") bits to represent the SI.  That doesn't seem like leaving the high-order bits untouched. But perhaps "untouched" means "the bits are not modified by the forwarding plane" rather than "the functionality of the bits is not modified".

They are left 'untouched' as in 'not modified after initialization'. Of course the bit is touched by the sender of the packet. But if you find it unclear, I will also try to clarify.

> 
> It is worth emphasizing that, unlike other proposed encapsulations, this encapsulation only supports BitStrings of length 64.  Well, 128-p-i.  I guess one could use a /40 prefix and a single SI to get a BitStringLength of 88 if one doesn't need any more than 88 BFR-ids in the domain.  It still seems rather restrictive when compared to the other encapsulations that have been proposed.

IMHO, what really matters is the ratio of packets sent over the number of reached destinations. From that regard, a case with 1024 long BitString and 1M destinations with random access will not be very efficient. In some cases, 64 destinations are far enough. As you noted, we already have an encapsulation which can handle longer BitString, so I guess we'll have to evaluate whether other use cases, 1., need more bits, 2., can use the existing MPLS encap.

> I don't see how the sub-domain is identified in this encapsulation.

It is not addressed by the document indeed. Just for the sake of making a short document.
I guess we could use the IPv6 prefix for that as well.

> 
> The draft says: "It makes use of a typical IP longest match in order to decide whether a packet is a BIER packet or not, which means hardware and software existing solutions may be used for that purpose."  But existing/deployed hardware and software solutions cannot do the BIER forwarding!  So I don't see what advantage is being claimed here.  Or is this just a way of saying "at least we don't use IPv6 extension headers" ;-)

It is a way to say "Hey, you will be able to reuse bits of hardware and code rather than restart everything from scratch".
But you have a point. I could add "it doesn't use an IPv6 extension header" too.

> 
> The draft seems to suggest that if the payload is an IPv6 packet, one can use BIER to multicast it without encapsulating the payload at all.  Rather, all one has to do is overwrite the enduser's destination address with an "address" value that contains the BIER prefix, SI, and 64-bit BitString.  Do I understand that correctly? If so, this seems to create a number of issues that are not discussed.
> 
> Suppose, for example, that the original payload has an IPv6 multicast address in its destination address field.  If that is overwritten, then when the packet reaches a BFER, the BFER will have no way of knowing what the original destination address was.  But the multicast flow layer needs that information in order to further process the packet!
> 

I do not suggest to overwrite the destination address sent by the source. If you don't want to restore it later, it is possible. But this is obviously a bad idea if you need to restore it.
I was rather saying that the source can directly originate a packet which destination is a BIER IPv6 address.
Or if the source sends an IPv6 multicast packets that you need to be sent through the BIER domain, you can encapsulate it inside the IPv6 BIER packet. 

> If an enduser's UDP packet is carried as the payload of an IPv6 BIER packet, and the UDP packet has non-zero checksum, then, as the draft says, "the UDP checksum must be recomputed when the BIER bits are changed."  So the BIER forwarding plane now has a new function -- it has to look to see what the payload is, and whether the the payload has a checksum that needs to be adjusted.  Does that sound like a good idea?

I see at least two possibilities here:
- Update checksum at every BitString change. This is actually not so expensive given that the IP checksum is linear (i.e. support updates). Obviously this requires looking at layer 4.
- Have a convention that the packet is checksumed with BitString set to all-zeroes.

> 
> The draft says: "It is possible to configure a host with an address which corresponds to a BIER address with a single bit set.  From the host perspective, such address is not different from a regular IPv6 address."  Well, what will then stop the host from using that address as an IPv6 source address?

It is a feature, not a bug. A BIER IPv6 address with a single bit set is a valid unicast address.

> 
> Packets carrying such an IPv6 source address may get dropped for using a prefix that has not been delegated to the host's subnet.
> 
> If a host uses a "BIER address" as its source address in a given packet, and the packet doesn't get dropped, the packet may elicit responses that put the BIER address in their destination address fields.  This seems like a security issue, as it creates an attack vector that can create 64 responses to a single probe.

Are you talking about BCP38 ? Ingress filtering means that a host cannot send a packet with any source address. Of course, if the host is assigned a BIER address, adjacent routers will have to be configured to not drop the packets that are originated with this source address. It doesn't mean that you allow the host to use any BIER address though. The first-hop router (or any other routers) can be configured to do a source address filtering (just like BCP 38) to make sure that the source address is not a BIER address with more than one bit set.

> 
> Hopefully hosts will not treat the BIER prefix as a delegated prefix that they can use to compute their own "privacy addresses"; if any of those computed addresses make it to the source address field of an IPv6 packet, and the packet does not get dropped, chaos could ensue.

Why would you advertise an IPv6 BIER prefix with the M bit set to 0 (enabling SLAAC) in your RAs ?! I mean, there is no way SLAAC could be used for that purpose. You would use DHCPv6 (or some other equivalent) for address configuration anyway.

> 
> The draft suggests that the BIER prefix can be treated by BFRs outside a given domain as a routable unicast prefix, presumably leading towards one or more BFRs that serve as border routers for a given BIER domain.  (At one point, the draft even says that BIER packets are IPv6 unicast packets, which seems a bit misleading. ;-)) That's interesting, but putting this feature to use requires one to work out quite a few details that are not mentioned.  And I don't think it will work for non-BFRs within the BIER domain.  To pass BIER packet through non-BFRs within the destination BIER domain, one still needs additional encapsulation.

The document does not claim that you can have non-BFR routers within the BIER domain. It just claims that you can use regular unicast forwarding between the source and the BFIR.
What sort of details do you think are not mentioned ? It seems to me that a BIER packet could be very-well routed as unicast based on the destination BIER prefix up to the BIER domain.

> 
> The draft talks about "configuring" a BIER router with the rules needed to interpret a given IPv6 destination address field.  Does this mean that IGP/BGP extensions for signaling BFR-ids and other BFR parameters will not be used?  Why?  Or is it allowable for these parameters to be signaled as well as configured?  And if we are going to talk about inter-domain BIER, how is the inter-domain signaling done?

You can ! You can ! :-)
"Configured" means, "as far as the BIER Layer is concerned", whether this comes from an educated system administrator, a running process, or a monkey, I couldn't care less.
I will clarify in the next iteration.

> 
> It is not clear from the draft whether the value of i (number of bits in the SI field) and the set of BFR-ids is specific to a given prefix, or whether it is the same for all prefixes.  (Or for all prefixes corresponding to a given domain.  Or perhaps sub-domain?)

Well. This clearly is the kind of things that can be discussed, right ?
When I first thought about this approach, I made an experimental implementation for Linux. I ended up not caring about these. I didn't even implemented the concept of S.I., but rather decided that every BIER prefix would use a different 'bit index to next-hop' mapping table. So, I would say that the set of BRF-ids is specific to a given prefix. I would even say we can remove the concept of domain and S.I. But that is the sort of things that I would need BIER experts to discuss with.

> 
> Section 6 gives the "advantages" of the proposal, but it is not clear what the proposal is being compared to.  For instance, one of the advantages is claimed to be that the proposal doesn't use IPv6 extension headers.  Is the proposal being compared to another proposal that does use IPv6 extension headers?  If so, where is that proposal described?  (None of the proposals I've seen so far use IPv6 extension headers.)

Okay. What encap should I compare it to ?

> 
> Another "advantage" is that the proposal "may be used for transporting IP multicast packets, but also for sending IP payloads directly to multiple destinations".  Does this "advantage" distinguish it from some other proposal?  I'd have thought that any multicast encapsulation can cause any packet to sent to multiple destinations.

Only if the destination implements BIER. In this proposal, the destination does not need to decap anything. It directly receives a unicast packet.

> 
> Of course, maybe the first question is whether we really have a need for an encapsulation that is IPv6-specific.  The draft doesn't actually seem to address this question.
> 

Do you think we shouldn't have another encap. ?
Or are you saying we should have a different encap ?

---

From: Eric C Rosen <[email-masked]>
Date: Thu, 15 Sep 2016 11:16:30 -0400

> IMHO, what really matters is the ratio of packets sent over the number of reached destinations. From that regard, a case with 1024 long BitString and 1M destinations with random access will not be…

You should point out then that your encapsulation does not support the 
full functionality of the BIER architecture as specified by the WG, but 
only an arbitrarily chosen subset of it.

> It is not addressed by the document indeed. Just for the sake of making a short document.

This is another way in which your proposal fails to support the BIER 
architecture.

> It is a way to say "Hey, you will be able to reuse bits of hardware and code rather than restart everything from scratch".

I don't think there is any proposed encapsulation whose implementation 
cannot make use of some existing logic.  So this "advantage" does not 
distinguish your proposal from any other.

> But you have a point. I could add "it doesn't use an IPv6 extension header" too.

Well, there's lots of things it doesn't do.  This particular point is 
only interesting if you are comparing your proposal to another proposal 
that does use IPv6 extension headers.

Of course, if there were such a proposal, it would probably claim as an 
advantage "it doesn't modify the destination address field of a packet 
while the packet is in flight", thereby avoiding a whole set of 
unintended side-effects.

(I do think it is good to avoid IPv6 extension headers, but none of the 
encapsulations that have been proposed to the WG use IPv6 extension 
headers, so this doesn't seem like much of an issue.)

> I was rather saying that the source can directly originate a packet which destination is a BIER IPv6 address.

There is nothing to prevent a source from originating a packet with any 
kind of encapsulation whatsoever.

> Or if the source sends an IPv6 multicast packets that you need to be sent through the BIER domain, you can encapsulate it inside the IPv6 BIER packet.

Same as any other encapsulation.

> - Update checksum at every BitString change. This is actually not so expensive given that the IP checksum is linear (i.e. support updates). Obviously this requires looking at layer 4.

If your encapsulation requires that the forwarders look at layer 4, I'd 
say that is a pretty big disadvantage.

> - Have a convention that the packet is checksumed with BitString set to all-zeroes.

I don't think a BIER encapsulation should require that UDP be modified.

> It is a feature, not a bug. A BIER IPv6 address with a single bit set is a valid unicast address.

No it isn't.  See my comments on using a "BIER IPv6 address" in the 
source address field.

> Are you talking about BCP38 ? Ingress filtering means that a host cannot send a packet with any source address. Of course, if the host is assigned a BIER address, adjacent routers will have to be configured to not drop the packets that are originated with this source address.

The need to configure adjacent routers this way doesn't seem like much 
of an advantage.

> It doesn't mean that you allow the host to use any BIER address though. The first-hop router (or any other routers) can be configured to do a source address filtering (just like BCP 38) to make sure that the source address is not a BIER address with more than one bit set.

More special case configuration needed in order to use this encapsulation.

> Why would you advertise an IPv6 BIER prefix with the M bit set to 0 (enabling SLAAC) in your RAs ?! I mean, there is no way SLAAC could be used for that purpose. You would use DHCPv6 (or some other…

Ahh, the old "it will never happen" defense.  After all, bits never get 
set incorrectly, or in a way not anticipated by the authors of a 
specification ;-)

Actually, every network disaster is preceded by a developer saying "that 
will never happen" because "no one would ever do that".
> The document does not claim that you can have non-BFR routers within the BIER domain.

That feature is supported by the architecture, so I guess this is 
another feature of the architecture that is not supported by your 
proposed encapsulation.

> It just claims that you can use regular unicast forwarding between the source and the BFIR.
> What sort of details do you think are not mentioned ?

The description of the procedures that cause the FIBs to be set up 
appropriately in all the routers outside the BIER domain.

Also, don't forget to consider the case of hosts that are multi-homed to 
several BIER domains.

And the case of a host that is not directly connected to its BIER 
domain.  If the BIER prefix leads a packet (via unicast forwarding) from 
the host to a distant BFIR, how will that same prefix lead a packet (via 
unicast forwarding) from the distant BIER domain to the host?

> I will clarify in the next iteration.
You have a companion draft describing the ISIS/OSPF/BGP extensions for 
doing the necessary signaling?
> When I first thought about this approach, I made an experimental implementation for Linux. I ended up not caring about these. I didn't even implemented the concept of S.I., but rather decided that…
It seems like your "experimental implementation" does not comply with 
the BIER architecture.
> Okay. What encap should I compare it to ?

There's not much point comparing it to a strawman that has neither been 
proposed to the WG in another draft nor described in this draft.

> Only if the destination implements BIER. In this proposal, the destination does not need to decap anything. It directly receives a unicast packet.

In most BIER encapsulation proposals, the decapsulation is done by the 
BFER, not by the destination host.

> Do you think we shouldn't have another encap. ?

It's true that I don't see a need for another encapsulation.  But ...

> Or are you saying we should have a different encap ?
But if we do have another encapsulation, it shouldn't be IPv6-specific, 
and it shouldn't require hop-by-hop modification of the IP destination 
address field, and it shouldn't require the BFRs to inspect layer 4, and 
it shouldn't require changes to layer 4, and it shouldn't allow a 
multicast address to be put in the IP source address field, and it 
shouldn't assume that bits never get set incorrectly, and it shouldn't 
require changes in source address filtering procedures.  In addition, it 
should be possible to use it to support the entire BIER architecture, 
including those features that didn't make it into your experimental 
Linux implementation.

---

From: Pierre Pfister <[email-masked]>
Date: Thu, 15 Sep 2016 19:29:06 +0200

> Le 15 sept. 2016 à 17:16, Eric C Rosen <[email-masked]> a écrit :
> 
> 
> You should point out then that your encapsulation does not support the full functionality of the BIER architecture as specified by the WG, but only an arbitrarily chosen subset of it.

You may see an arbitrarily chosen subset of features where others may see applicability to different use-cases such as enterprise or home networks, and IP networks rather than MPLS networks. 
The purpose of a new encapsulation is not to provide a second solution addressing the exact same issue and use-cases but rather see the areas where the first encap is not applicable.

The BIER architecture document is a draft. The working group may realize than it does not fully handle certain use-case yet.

> This is another way in which your proposal fails to support the BIER architecture.

No. 
This is a way to say that the proposal is a first iteration which will be improved and include sub-domains in further iterations if people think it is necessary.

You removed part of my answer too: "I guess we could use the IPv6 prefix for that as well.".
Any useful technical comments on this proposal ?

> I don't think there is any proposed encapsulation whose implementation cannot make use of some existing logic.  So this "advantage" does not distinguish your proposal from any other.

Please read carefully. I am not saying that the proposal "can make use of some existing logic". I am saying it makes use of longest match on the destination address in order to determine whether the packet is a BIER packet, the Set ID, and possibly the sub-domain.
Longest match prefix is not just a random piece of existing logic. It is widely optimized on both hardware and software.
In addition, it means that packet classification can be achieved by the use of the already existing routing operation. Which will simplify development and deployment.
Hence, this is an advantage.

> 
> If your encapsulation requires that the forwarders look at layer 4, I'd say that is a pretty big disadvantage.
> 

Which means we shouldn't use encapsulation within UDP either.
But you also mention that we shouldn't use IPv6 Extension Headers.
So this leaves us only encapsulation over Ethernet.
Can you elaborate if you think this is a better alternative ?

> No it isn't.  See my comments on using a "BIER IPv6 address" in the source address field.

See my answer.

> The need to configure adjacent routers this way doesn't seem like much of an advantage.

Yes. Routers that are adjacent to hosts will have to run or relay DHCPv6.
Which is pretty much necessary when you deal with hosts.

> More special case configuration needed in order to use this encapsulation.

Just DHCPv6.

> That feature is supported by the architecture, so I guess this is another feature of the architecture that is not supported by your proposed encapsulation.

Are you referring to this sentence ?
"BIER would then have to be run in a
   sub-domain that is bound to this topology.  If unicast tunnels are
   used to bypass non-BFRs, either the tunnels have to be restricted to
   this topology, or the tunnel endpoints have to be BFRs that do not
   have any non-BIER-capable interfaces."

It seemed pretty obvious to me that an IP based solution could use tunneling to solve this issue.

> The description of the procedures that cause the FIBs to be set up appropriately in all the routers outside the BIER domain.

From the draft:
If the BIER IPv6
      Prefix is a globally unique IPv6 prefix, reachable from outside
      the BIER domain, it is possible to send a packet from outside the
      BIER domain to multiple destination within the BIER domain.

I don't think it is necessary to describe in more details the many ways a unicast prefix can be made reachable from
a private or public part of the internet.

> 
> Also, don't forget to consider the case of hosts that are multi-homed to several BIER domains.

I don't think bier WG is exactly chartered to solve host multi-homing. ;-)

> 
> And the case of a host that is not directly connected to its BIER domain.  If the BIER prefix leads a packet (via unicast forwarding) from the host to a distant BFIR, how will that same prefix lead a packet (via unicast forwarding) from the distant BIER domain to the host?

I don't claim it does.

> You have a companion draft describing the ISIS/OSPF/BGP extensions for doing the necessary signaling?

I don't think it should be any different from the documents related to the other encaps.
At least, I think those extensions should be agnostic.
If you see reasons or cases where they are not, I would be interested to know more.

> It seems like your "experimental implementation" does not comply with the BIER architecture.

It does. Because it implements the required features and behaves in a way that is in agreement with the BIER architecture draft.
I am just giving implementation feedback stating that, in this particular case, the implementation does not need to explicitly consider the notion of S.I. in order to be compliant.

> In most BIER encapsulation proposals, the decapsulation is done by the BFER, not by the destination host.

Nothing prevents a BFER to be the destination of a packet though.
So I don't exactly see your point.

> But if we do have another encapsulation, it shouldn't be IPv6-specific, and it shouldn't require hop-by-hop modification of the IP destination address field, and it shouldn't require the BFRs to inspect layer 4, and it shouldn't require changes to layer 4, and

> it shouldn't allow a multicast address to be put in the IP source address field
Where did you see this ?!

As stated in a previous comment. You say no Layer 4, no extension headers, no IP header. 
This just leaves us to Ethernet layer.

Which one is more specific ? Ethernet or IPv6 ?

> , and it shouldn't assume that bits never get set incorrectly,

Can I assume that the BIER BitString and the IP version number are set correctly ?

> and it shouldn't require changes in source address filtering procedures.  

It doesn't.

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Sep 2016 14:16:21 -0400

Sometimes an idea that seems clever at first just doesn't work out when 
the details are examined.  The idea of using addresses that are unicast 
addresses in one context but multicast addresses in another falls into 
this category.  Inevitably a host will put a multicast address into an 
IP source address field, because the host doesn't really have any way of 
knowing that it is doing something wrong. Inevitably a host will create 
temporary IPv6 addresses from the BIER prefix, because it doesn't know 
that it is doing something wrong. It's hard for routers to filter out 
packets carrying such addresses in the source address field, because the 
local routers also cannot necessarily tell that the address is really a 
multicast address. There are just too many unintended side-effects for 
this to be a viable proposal.

The only answer has been "this won't happen unless someone does 
something wrong", which is not an answer at all.

On 9/15/2016 1:29 PM, Pierre Pfister wrote:
>
>> Le 15 sept. 2016 à 17:16, Eric C Rosen <[email-masked] 
>> <mailto:[email-masked]>> a écrit :
>>
>>
>> You should point out then that your encapsulation does not support 
>> the full functionality of the BIER architecture as specified by the 
>> WG, but only an arbitrarily chosen subset of it.
>
> You may see an arbitrarily chosen subset of features where others may 
> see applicability to different use-cases such as enterprise or home 
> networks, and IP networks rather than MPLS networks.
> The purpose of a new encapsulation is not to provide a second solution 
> addressing the exact same issue and use-cases but rather see the areas 
> where the first encap is not applicable.

Any BIER encapsulation needs to be able to support the features that are 
in the architecture.

It may or may not be true that the architecture is missing features that 
it should have, but the encapsulations still need to support the 
features that it does have.

> This is a way to say that the proposal is a first iteration which will 
> be improved and include sub-domains in further iterations if people 
> think it is necessary.

Well, it's hard to comment on what might or might not be in further 
iterations.

> Any useful technical comments on this proposal ?

At that level of detail there isn't much to comment on.  I would like to 
see the proposed signaling that associates the prefix with a sub-domain 
and/or other BIER parameters.

>
>>
>>>> The draft says: "It makes use of a typical IP longest match in 
>>>> order to decide whether a packet is a BIER packet or not, which 
>>>> means hardware and software existing solutions may be used for that 
>>>> purpose."  But existing/deployed hardware and software solutions 
>>>> cannot do the BIER forwarding!  So I don't see what advantage is 
>>>> being claimed here.  Or is this just a way of saying "at least we 
>>>> don't use IPv6 extension headers" ;-)
>>> It is a way to say "Hey, you will be able to reuse bits of hardware 
>>> and code rather than restart everything from scratch".
>>
>> I don't think there is any proposed encapsulation whose 
>> implementation cannot make use of some existing logic.  So this 
>> "advantage" does not distinguish your proposal from any other.
>
> Please read carefully. I am not saying that the proposal "can make use 
> of some existing logic". I am saying it makes use of longest match on 
> the destination address in order to determine whether the packet is a 
> BIER packet, the Set ID, and possibly the sub-domain.
> Longest match prefix is not just a random piece of existing logic. It 
> is widely optimized on both hardware and software.
> In addition, it means that packet classification can be achieved by 
> the use of the already existing routing operation. Which will simplify 
> development and deployment.

  In some other proposals for a second encaps, a packet is identified as 
a BIER  packet by virtue of the value of the ethertype field.  I just 
don't see the advantage of using the "widely optimized" longest match 
procedures rather than doing a simple lookup of a fixed length field.

>
>>
>> If your encapsulation requires that the forwarders look at layer 4, 
>> I'd say that is a pretty big disadvantage.
>>

I don't see why that follows, but we're not discussing a UDP-based 
encapsulation right now.

> Can you elaborate if you think this is a better alternative ?

The ethernet encapsulation proposals certainly do not have the same set 
of problems that the IPv6 proposal has.

> Which is pretty much necessary when you deal with hosts.

That doesn't seem to be true; right now I am on a network where the CPE 
router is a dhcpv6 client of the ISP router, but the hosts behind the 
CPE router only receive router advertisement messages.

But that's all irrelevant anyway, as "DCHPv6 will prevent any addressing 
problems from occurring" is not a credible claim.

>
>>>> The draft suggests that the BIER prefix can be treated by BFRs 
>>>> outside a given domain as a routable unicast prefix, presumably 
>>>> leading towards one or more BFRs that serve as border routers for a 
>>>> given BIER domain.  (At one point, the draft even says that BIER 
>>>> packets are IPv6 unicast packets, which seems a bit misleading. 
>>>> ;-)) That's interesting, but putting this feature to use requires 
>>>> one to work out quite a few details that are not mentioned.  And I 
>>>> don't think it will work for non-BFRs within the BIER domain.  To 
>>>> pass BIER packet through non-BFRs within the destination BIER 
>>>> domain, one still needs additional encapsulation.

In your proposal, the IPv6 prefix can cause a packet to be 
unicast-routed to a BIER domain.  If the first router it encounters in 
the BIER domain is a non-BFR, I don't see how tunneling will help.  
You'll probably say that this situation is impossible, but since the 
draft provides no details about how the prefixes are distributed, I 
don't see any basis for saying that this couldn't happen.

> I don't think it is necessary to describe in more details the many 
> ways a unicast prefix can be made reachable from

I'm afraid the situation is a bit more complicated.  The prefix is meant 
to be interpreted as a unicast prefix in certain contexts, but as a BIER 
prefix in other contexts.  The signaling and routing procedures need to 
ensure that the prefix will be interpreted, by each router to which it 
is propagated, in the context that is appropriate for that router.   
There are a lot of non-obvious details needed to make this happen.

> I don't think bier WG is exactly chartered to solve host multi-homing. ;-)

I don't see the relevance of that claim.  Many devices are multi-homed 
and BIER cannot assume that there is no multi-homing. Of course, BIER 
doesn't have to solve multi-homing issues that are not related to BIER.

>> domain.  If the BIER prefix leads a packet (via unicast forwarding) 
>> from the host to a distant BFIR, how will that same prefix lead a 
>> packet (via unicast forwarding) from the distant BIER domain to the host?
>
> I don't claim it does.

If the prefix is part of a unicast address of a host, it has to lead 
packets toward the host.  If the prefix identifies a BIER domain, it has 
to lead packets toward the BIER domain.  If the host isn't directly 
connected to the BIER domain, I don't see how this will work.  So there 
must be some restrictions on how these addresses and prefixes are 
assigned.  Perhaps these restrictions are included among the details 
that are" too obvious" to mention.  Of course, a lot of things seem 
obvious until you try to work out the details.

> If you see reasons or cases where they are not, I would be interested 

I don't think any of the existing drafts on ISIS/OSPF/BGP signaling for 
BIER say how to associate IPv6 prefixes with BIER parameters. Please 
don't say that the details are too obvious to mention!  Has any 
consideration even been given to the signaling that needs to exist in 
order to support this encapsulation?

>
>>>
>>> When I first thought about this approach, I made an experimental 
>>> implementation for Linux. I ended up not caring about these. I 
>>> didn't even implemented the concept of S.I., but rather decided that 
>>> every BIER prefix would use a different 'bit index to next-hop' 
>>> mapping table. So, I would say that the set of BRF-ids is specific 
>>> to a given prefix. I would even say we can remove the concept of 
>>> domain and S.I. But that is the sort of things that I would need 
>>> BIER experts to discuss with.
> It does. Because it implements the required features and behaves in a 
> way that is in agreement with the BIER architecture draft.
> I am just giving implementation feedback stating that, in this 
> particular case, the implementation does not need to explicitly 
> consider the notion of S.I. in order to be compliant.

An implementation that doesn't support SI is not compliant with the 
architecture.

>
>>
>>>> Another "advantage" is that the proposal "may be used for 
>>>> transporting IP multicast packets, but also for sending IP payloads 
>>>> directly to multiple destinations".  Does this "advantage" 
>>>> distinguish it from some other proposal?  I'd have thought that any 
>>>> multicast encapsulation can cause any packet to sent to multiple 
>>>> destinations.
>>> Only if the destination implements BIER. In this proposal, the 
>>> destination does not need to decap anything. It directly receives a 
>>> unicast packet.
>>

The point is that, for all encapsulations, decapsulation is done by the 
BFER, not by the destination.  So the fact that the destination doesn't 
have to do decapsulation does not distinguish one encapsulation from 
another.

You might be claiming, as an advantage, that BFERs don't always have to 
decapsulate and that BFIRs don't always have to encapsulate. This is the 
old "nat vs. tunneling" rathole.  In the particular case of BIER,  there 
is a  set of problems caused by modifying the destination address field 
and by obscuring the distinction between multicast addresses and unicast 
addresses.  These seem to dominate any apparent advantage of not having 
to encapsulate.

>
>> But if we do have another encapsulation, it shouldn't be 
>> IPv6-specific, and it shouldn't require hop-by-hop modification of 
>> the IP destination address field, and it shouldn't require the BFRs 
>> to inspect layer 4, and it shouldn't require changes to layer 4, and
>

This is the inevitable consequence of making it impossible for hosts or 
even routers to tell the difference between multicast addresses and 
unicast addresses.

> Can I assume that the BIER BitString and the IP version number are set 

It is always necessary to understand the impact on the network of 
configurations that are imperfect.  Certainly one cannot assume that the 
router advertisement configuration files always have the flags set 
correctly.

>> and it shouldn't require changes in source address filtering procedures.

Of course it does, because certain prefixes that are advertised to the 
hosts are now going to be prohibited from appearing in the source 
address field.

---

From: IJsbrand Wijnands <[email-masked]>
Date: Tue, 20 Sep 2016 16:29:47 +0200

Hi Eric,

Many thanks for your comments. A few responses inline.

> Sometimes an idea that seems clever at first just doesn't work out when the details are examined.  The idea of using addresses that are unicast addresses in one context but multicast addresses in another falls into this category.  Inevitably a host will put a multicast address into an IP source address field, because the host

Maybe we can consider registering a new IPv6 address space for BIER. I think a lot of your concerns would be addressed. That way we don’t get in the way with already defined unicast behaviour and still leaves it open to treat the packet as unicast in some specific cases, like with tunnelling through non-BIER nodes.

> Any BIER encapsulation needs to be able to support the features that are in the architecture.  

And we should not be afraid to allow exceptions and/or updates to the architecture draft if necessary.

> An implementation that doesn't support SI is not compliant with the architecture.

By allocating part of the “prefix” for the SI/SD identifier we can make this work just fine IMO. Since the SI/SD here is manually defined and the BitString length is fixed, I think there is really only need for one of them. If we call that SI or SD (or something else) we can argue about :-)

Thx,

Ice.

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 21 Sep 2016 10:09:20 +0000

Hi Ice,

If I understand your draft correctly, it talks about embedding BIER within IPv6 rather than transporting BIER over IP. The beauty of this approach is that all information needed for BIER forwarding is now directly contained in the received BIER packet. As a result, there is no need to retrieve them from the MPLS-BIER label and distribute the MPLS-BIER label mappings. However, what's the major advantage over the generic BIER header as defined in https://tools.ietf.org/html/draft-xu-bier-encapsulation-05 from your point of view?

Best regards,
Xiaohu

-----邮件原件-----
发件人: BIER [mailto:[email-masked]] 代表 IJsbrand Wijnands
发送时间: 2016年9月20日 22:30
收件人: Eric C Rosen
抄送: [email-masked]; Pierre Pfister
主题: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Eric,

Thx,

Ice.

_______________________________________________
BIER mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bier

---

From: IJsbrand Wijnands <[email-masked]>
Date: Wed, 21 Sep 2016 12:27:42 +0200

Hi Xiaohu,

I would say that one of the big benefits is that an IPv6 BIER encoded packet can be translated/transformed into a set of IPv6 Unicast packets (just one bit set) at the egress of the network very easily. This is very beneficial when running BIER between servers and avoids the need for MLD and DR/DF election between the Server and the router.

Thx,

Ice.

> On 21 Sep 2016, at 12:09, Xuxiaohu <[email-masked]> wrote:
> 
> Hi Ice,
>

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 22 Sep 2016 01:48:33 +0000

Hi Ice,

Could you please illustrate how to translate/transform an IPv6-BIER packet into a set of IPv6 unicast packets by setting only one bit? BTW, what's the payload of the IPv6-BIER packet in your example? In other words, what's the value of the Next-Header field of the IPv6-BIER header?
 
Best regards,
Xiaohu

-----邮件原件-----
发件人: IJsbrand Wijnands [mailto:[email-masked]] 
发送时间: 2016年9月21日 18:28
收件人: Xuxiaohu
抄送: Eric C Rosen; [email-masked]; Pierre Pfister
主题: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Xiaohu,

Thx,

Ice.

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 22 Sep 2016 04:07:19 +0000

Hi Ice,

Firstly, I'm glad to see that your IPv6-BIER draft somehow echoes my previous statement about the similarity between the BIER header and the IPv6 header (see below):

https://www.ietf.org/mail-archive/web/bier/current/msg00101.html
https://www.ietf.org/proceedings/92/slides/slides-92-bier-1.pdf

That's a great progress!:) IMO.

Secondly, the major difference between the IPv6-BIER encapsulation and the generic BIER encapsulation (https://tools.ietf.org/html/draft-xu-bier-encapsulation-05) is that the former makes the BIER as a special use of IPv6 while the latter makes the BIER as another network layer in parallel with IPv6 but with many similarities. I agree that allowing legacy routers to forward IPv6-BIER packets as normal IPv6 packets is a major merit of the IPv6-BIER although this merit is only applicable to IPv6 but not IPv4. However, in addition to those drawbacks pointed by Eric in his previous email, the associated costs include but not limited to:

1) The Bitstring length is not flexible any more since the Bitstring is restricted within the 128-bit long destination IPv6 address.
2) The BIER header format is not future-proven any more due to the lack of a version field dedicated for BIER. Or do you want to some bits within the destination IPv6 address as a BIER-specific version? If so, it would further suppress the available space for the Bitstring field. Or do you want to consume more IPv6 address space to make room for BIER?
3) It introduces modifications to the current mature IPv6 forwarding procedure branch. 
4) Due to the reusage of the IPv6 header format, the encapsulation overhead is unnecessary increased especially in IPv4 networks (e.g., use the 128-bit long IPv6 source address instead of 16-bit BFIR-id to indicate the sender).

Best regards,
Xiaohu

-----邮件原件-----
发件人: BIER [mailto:[email-masked]] 代表 Xuxiaohu
发送时间: 2016年9月22日 9:49
收件人: IJsbrand Wijnands
抄送: [email-masked]; Pierre Pfister; Eric C Rosen
主题: [Bier] 答复: New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Ice,

Best regards,
Xiaohu

-----邮件原件-----
发件人: IJsbrand Wijnands [mailto:[email-masked]] 
发送时间: 2016年9月21日 18:28
收件人: Xuxiaohu
抄送: Eric C Rosen; [email-masked]; Pierre Pfister
主题: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Xiaohu,

Thx,

Ice.

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 22 Sep 2016 08:02:48 +0000

BTW, I found the only merit of the IPv6-BIER that I had ever thought of is questionable after serious consideration. 

How to traverse non-BFRs within a BIER domain is a basic requirement of the BIER architecture as Eric had pointed out in this email. Even assume non-BFRs could forward IPv6-BIER packet as normal IPv6 packet towards one of the nearest BFRs which had ever advertised an BIER-prefix into the BIER domain, if the BFR that needs to send an IPv6-BIER packet to an adjacent non-BFR had also advertised a BIER prefix into the BIER domain, what would happen? The BIER packet could not be sent out since the nearest next-hop towards the BIER prefix is itself.  Even that packet is sent out by using some special policy, that IPv6-BIER packet could be sent back due to the shortest path forwarding principle. 

As for the need "to send a packet from outside the BIER domain to multiple destination within the BIER domain..." that is listed as an advantage in the IPv6-BIER packet, I would like to know which use case as described in the BIER use case draft needs such usage.

Best regards,
Xiaohu  

-----邮件原件-----
发件人: Xuxiaohu 
发送时间: 2016年9月22日 12:07
收件人: Xuxiaohu; IJsbrand Wijnands
抄送: [email-masked]; Pierre Pfister; Eric C Rosen
主题: 答复: [Bier] 答复: New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Ice,

Best regards,
Xiaohu

-----邮件原件-----
发件人: BIER [mailto:[email-masked]] 代表 Xuxiaohu
发送时间: 2016年9月22日 9:49
收件人: IJsbrand Wijnands
抄送: [email-masked]; Pierre Pfister; Eric C Rosen
主题: [Bier] 答复: New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Ice,

Best regards,
Xiaohu

-----邮件原件-----
发件人: IJsbrand Wijnands [mailto:[email-masked]] 
发送时间: 2016年9月21日 18:28
收件人: Xuxiaohu
抄送: Eric C Rosen; [email-masked]; Pierre Pfister
主题: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Xiaohu,

Thx,

Ice.

---

From: IJsbrand Wijnands <[email-masked]>
Date: Thu, 22 Sep 2016 10:34:49 +0200

Hi Xiaohu,

> That's a great progress!:) IMO.

I’m happy for you, but I see that differently.

The IPv6 BIER Prefix is a "Bit Index Forwarding Table" (BIFT) Identifier, similar to how the BIER MPLS label is. The difference between the two is that the MPLS label is local significant, and the BIER prefix is global significant.

Thx,

Ice.

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 22 Sep 2016 09:06:05 +0000

Hi Ice,

In fact, you could look the combination of <BSL, Sub-domain, SI> within the generic BIER header as "Bit Index Forwarding Table" (BIFT) Identifier as well if you would like. BTW, that is globally significant as well:)

Best regards,
Xiaohu

-----邮件原件-----
发件人: BIER [mailto:[email-masked]] 代表 IJsbrand Wijnands
发送时间: 2016年9月22日 16:35
收件人: Xuxiaohu
抄送: [email-masked]; Pierre Pfister; Eric C Rosen
主题: Re: [Bier] New Version Notification for draft-pfister-bier-over-ipv6-00.txt

Hi Xiaohu,

Thx,

Ice.

---

From: IJsbrand Wijnands <[email-masked]>
Date: Thu, 22 Sep 2016 11:18:36 +0200

Hi Xiaohu,

> In fact, you could look the combination of <BSL, Sub-domain, SI> within the generic BIER header as "Bit Index Forwarding Table" (BIFT) Identifier as well if you would like. BTW, that is globally si…

Yes, agreed.

Thx,

Ice.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 22 Sep 2016 19:07:49 +0200

How do i get a unique global allocation ?

---

From: Eric C Rosen <[email-masked]>
Date: Thu, 22 Sep 2016 15:22:35 -0400

Toerless, nice example.  But let's look at it more closely.

>                                          |-- Rcv1
>                              --- BFER1 --|
>                             /            |-- Rcv2
>        Src ---- Rtr --- BFIR
>                             \            |-- Rcv3
>                              --- BFER2 --|
>                                          |-- Rcv4
> 					|
> 					|-- Rtr2  ---- Rcv5
>
> Src sends one IPv6 unicast packet indicating (Rcv1,Rcv3,Rcv4). It will have a prefix
> owned by BFIR.
Actually, in this example, I think Src is the BFIR, what you call BFIR 
is actually just a BFR at the border of a BIER domain.

What does "owned by BFIR" mean?  Is the prefix jointly "owned" by all 
the BFRs bordering the BIER domain?  What about non-BFRs bordering the 
BIER domain?  What keeps the packet from going to them instead?

If Src thinks that the prefix is on the local network, it won't send the 
packet to Rtr, it will instead try to figure out the unicast MAC DA 
corresponding to the IP destination address.  This DA doesn't exist, so 
the packet will get dropped.  But if Src doesn't think that the prefix 
is on the local network connecting it to Rtr, it may send the BIER 
packet out a different interface.  Where will the packet end up then?

What if Src is doing load balancing among the default routers on the 
LAN, or among its interfaces?  For a given flow, the "IPv6 destination 
address" will change from time to time, which means the load balancing 
hash for that flow will change from time to time. This is the 
"non-deterministic ECMP" problem discussed in the architecture.  But 
since Src doesn't actually support BIER, it cannot use the solution 
described in the architecture draft.

What if there are receivers on the same LAN as the Src?  How will those 
receivers get the BIER packets sent by the Src?  Since Src thinks the 
BIER packet is a unicast packet, it will encapsulate it in a unicast 
ethernet frame whose MAC DA is a MAC address of Rtr. Since Rtr is not a 
BFR, Rtr will remove the ethernet header and forward the IPv6-BIER 
packet towards (let's say, for the sake of argument) the node you call 
BFIR.  How will BFIR get any copies of that packet back to the receivers 
that are on Src's subnet?  Sending the BIER packet (with modified 
BitString) back to Rtr will cause a loop.

> There it will be replicated by BIER. Copies arriving
> at BFER1,BFER2 will be converted back to IP unicast packets, one copy to Rcv2 from
> BFER1, one from BFER2 to Rcv3, one from BFER2 to Rcv4.

Suppose Src had an IP multicast packet to send, and encapsulated it in 
an IPv6-BIER header.  In that case, a BFER needs to decapsulate the 
packet and multicast the payload to the LAN on which the receivers 
reside.  How does a BFER know whether to (a) decapsulate and multicast 
or (b) whether to replicate, overwrite the destination address, and 
unicast?   I hope the answer isn't that the BFER needs to inspect the 
payload and guess what the hosts are expecting.

BFER2 needs to know which BFR-ids represent hosts on any particular LAN 
to which it is attached.  A BFR-id is just a shorthand for a BFR-prefix, 
i.e., for an IP address.   If BFER2 wants to use unicast replication to 
transmit on the last hop, why not just use the actual IP addresses of 
the receivers?

I don't see how BFER2 can get a BIER packet to Rcv5.  Rtr2 will at best 
loop the packet back to BFER2, and at worst the packet will get 
forwarded (through a route not necessarily shown in the above picture) 
all the way back to "BFIR", which "owns the prefix".

> How about supporting remote receivers like Rcv5 ? Sources can be remote so
> it would make sense to allow bidirectional support. Might need some signaling
> to BFER2 to map a remote IP address to a bit.
>
Each BFR and non-BFR can have a host route for every possible receiver, 
and some magic can be used to associate a BFR-id with each host route.  
That would eliminate some of the routing and looping issues.    There 
might be a few scaling issues though.

BTW, if one wants the BFR-ids to represent receiver hosts, one is going 
to need a lot more BFR-ids than if the BFR-ids represent BFERs.  So one 
would certainly  want to use an encapsulation that allows more than a 
64-bit BitString.  One could try to compensate for the very short 
BitString by having lots of sub-domains.  Then for each application (or 
perhaps each flow) one could try to assign the BFR-ids densely.   But 
we've heard that sub-domains aren't necessary for this encapsulation.  
More importantly, if a short BitString leads to more sub-domains, we 
will see an increase in the operational complexity and signaling load of 
the scheme.

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 23 Sep 2016 03:16:11 +0000

The combination of <BSL, Sub-domain, SI> has been explicitly contained in the received BIER packet. Hence just reading them is enough:)

Best regards,
Xiaohu

> -----邮件原件-----
> 发件人: Toerless Eckert [mailto:[email-masked]]
> 发送时间: 2016年9月23日 1:08
> 收件人: IJsbrand Wijnands
> 抄送: Xuxiaohu; [email-masked]; Pierre Pfister; Eric C Rosen
> > > In fact, you could look the combination of <BSL, Sub-domain, SI> within the
> generic BIER header as "Bit Index Forwarding Table" (BIFT) Identifier as well if
> you would like. BTW, that is globally significant as well:)
> >
> > Yes, agreed.
> >
> > Thx,
> >
> > Ice.
> >