Subject: [bess] Few queries on draft-ietf-bess-evpn-inter-subnet-forwarding-14

From: "Dikshit, Saumya" <[email-masked]>
Date: Fri, 13 Aug 2021 18:14:20 +0000

Hello Authors of  draft-ietf-bess-evpn-inter-subnet-forwarding,

I have following queries and comments about this draft “draft-ietf-bess-evpn-inter-subnet-forwarding”.
Please help clarify.

>>>>Section https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-8.1

MUST be at least equal to corresponding SYNC MAC sequence number
      if one is present.
Can we formally define what a “SYNC MAC sequence number” ?

>>>>Section https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-8.3

“MAC Mx with a sequence number that is higher than or equal to
   sequence number assigned to a LOCAL route for MAC Mx:
   o  PE MUST trigger probe and deletion procedure for all LOCAL IPs
      associated with MAC Mx.
   o  PE MUST trigger deletion procedure for LOCAL MAC route for Mx.

”
As per rfc7423, if equal sequence number is received, then the one published with lower vtep-ip is retained, and the other one is withdrawn.
While this section talks about probing it again.
This should be called out in the Interop section as well, for the co-existence of old rule and newly defined

Quoting from  https://datatracker.ietf.org/doc/html/rfc7432#section-15:

“If two (or more) PEs advertise the same MAC

   address with the same sequence number but different Ethernet segment

   identifiers, a PE that receives these routes selects the route

   advertised by the PE with the lowest IP address as the best route”

>>>> Section https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-8.6

“   an inter-op scenario with a different implementation could arise,

   where a PE implementation non-compliant with this document or with

   RFC 7432<https://datatracker.ietf.org/doc/html/rfc7432> assigns and advertises independent sequence numbers to MAC

   and MAC+IP routes”
How do we expect this implementation to inter-op, as it may expect two different MAC-only and MAC-IP advertisement from remote peers as well.?
Can we paraphrase this ?

>>>> Section https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-8.8

“Following a host move from PE1 to PE2, the host's MAC is

      discovered at PE2 as a local MAC via a data frames received from

      the host.”

Do we need to call out the misconfiguration case, where a probe may lead to DUP responses,

one from the (local learning) access side and other one across the fabric (overlay tunnel).

>>>> Section https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-10.4.1

“unfreezing the

      route at the FROZEN location will result in the route being

      advertised with a higher sequence number.”
Why are we tying probing with “unfreezing” ? FROZEN will typically indicate dropping of flows. Probing can still go on in parallel ?
Can this be called out explicitly.

Thanks
Saumya.

From: BESS [mailto:[email-masked]] On Behalf Of Dikshit, Saumya
Sent: Friday, August 13, 2021 4:15 PM
To: John Scudder <[email-masked]>; The IESG <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [bess] John Scudder's No Objection on draft-ietf-bess-evpn-inter-subnet-forwarding-14: (with COMMENT)

I have query regarding the following text in the section " https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-4.3.1<https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-extended-mobility-05#section-4.3.1>" :
" [IP7, M1] is learnt as a new route at
   [PE3, PE4] and advertised to remote PEs with a sequence number of 0.
   As a result, L3 reachability to IP7 would be established across the
   overlay, however, MAC mobility procedure for MAC1 will not trigger as
   a result of this MAC-IP route advertisement"

If a host is moved with the same MAC, the following is still being following in current implementation(s):
- Either "MAC-only-route" or "MAC-IP-route" advertisement, the sequence number is bumped in both cases
- On receiving side,
      -  the sequence-number is picked up from "MAC-only-route" or "MAC-IP-route" and applied to MAC learnings
      - the bumped up sequence number leads a withdraw of "MAC-only" or "MAC-IP-route" from the inferior (earlier) publisher

Kindly help explain, if the text mentioned in “section 4.3.1” is creating some doubts regarding the way things operate with current standards.
Though I definitely believe that this literature does away with lot of existing ambiguities.
I think we need to paraphrase this section atleast.

Thanks
Saumya.

-----Original Message-----
From: BESS [mailto:[email-masked]] On Behalf Of John Scudder via Datatracker
Sent: Thursday, July 15, 2021 4:53 AM
To: The IESG <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [bess] John Scudder's No Objection on draft-ietf-bess-evpn-inter-subnet-forwarding-14: (with COMMENT)

John Scudder has entered the following ballot position for
draft-ietf-bess-evpn-inter-subnet-forwarding-14: No Objection

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html<https://www.ietf.org/iesg/statement/discuss-criteria.html>
for more information about DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-inter-subnet-forwarding/<https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-inter-subnet-forwarding/>

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks to the authors for their work in addressing my comments. Copying my
(resolved) discuss points here for posterity.

----

I found this document difficult to review. Some of this might be due to the
fact that I'm not an expert on EVPN, but I think some of the reason is that the
document could be structured better and expressed more clearly. The only reason
I'm not opposing progression of the document on the grounds that it's too
unclear to implement is that I've been told, and accept on faith, that
implementations *have* been successfully written starting from the spec, which
implies it's implementable -- I guess by people who are expert in EVPN already,
it wouldn't be implementable by me.

In any case, I do have some points I would like to discuss, that are more
actionable.

1. I agree with Robert Wilton's comment on -09:

```
One question I have is whether it is possible to have a deployment where some
devices support synchronous mode and others support asynchronous mode.  Am I
right in presuming that this is not supported and if so is this capability
signaled in any way? Or is the expectation that this would be controlled via
deployment choice of network device, or though configuration management? ```

This issue still exists in -14. I think it should be addressed in the document.
Similarly, I agree with Warren Kumari's comment, also on -09:

```
I would strongly recommend that the authors read the OpsDir review at:
https://datatracker.ietf.org/doc/review-ietf-bess-evpn-inter-subnet-forwarding-09-opsdir-lc-jaeggli-2020-07-06/<https://datatracker.ietf.org/doc/review-ietf-bess-evpn-inter-subnet-forwarding-09-opsdir-lc-jaeggli-2020-07-06/>
, especially the: "it would be helpful if section 4 would be more explicit for
non-implementors on when symetric or asymetric modules would be chosen, as it
stands the variation basically reads like the enumeration of the features of
various implementations." comment (which I fully agree with). ```

It seems both of these comments could -- and should! -- be addressed by adding
a few paragraphs talking about these topics. This could be done either in §4,
as Warren suggests, or in some other section (e.g. you could add an
"operational considerations" section).

2. Section 7.1

I’m guessing this question isn’t unique to this document, but since this is
where I encountered it, I’ll ask: it seems as though the described mobility
procedures are vulnerable to a condition where a particular (IP, MAC) appears
at two different NVEs at the same time. If this condition exists (either
innocently, or maliciously) what prevents the source and target NVEs from
continually attempting to claim the (IP, MAC) from one another, flooding the
network with updates all the while?

(This applies to 7.2 as well.)

Since this seems like a potential security issue, I'm including it in my
DISCUSS.

----

Below are a number of questions and comments that I hope might help improve the
document. I haven't chosen to make them blocking by including them in my
DISCUSS; nonetheless I would appreciate replies to them.

1. I agree with the comments by several of the other reviewers, that there are
just too many gratuitous acronyms in this document. They aren't the only thing
that makes it hard to read, but they certainly contribute. I'm disappointed to
see this hasn't been addressed between versions -09 and -14. It would have been
a small matter of search-and-replace to go through and expand most of the
acronyms.

2. Section 2

```
   R1: The solution must allow for both inter-subnet and intra-subnet
   traffic belonging to the same tenant to be locally routed and bridged
   respectively.  The solution must provide IP routing for inter-subnet
   traffic and Ethernet Bridging for intra-subnet traffic.  It should be
   noted that if an IP-VRF in a NVE is configured for IPv6 and that NVE
   receives IPv4 traffic on the corresponding VLAN, then the IPv4
   traffic is treated as L2 traffic and it is bridged.  Also vise versa,
   if an IP-VRF in a NVE is configured for IPv4 and that NVE receives
   IPv6 traffic on the corresponding VLAN, then the IPv6 traffic is
   treated as L2 traffic and it is bridged.

   R2: The solution must support bridging for non-IP traffic.
```

R1 is a little tortured, where you add all the caveats about “treated as L2
traffic”. Seems to me like it would fall out more naturally if you had simply
introduced the concepts of routable and non-routable traffic, where routable
traffic is that for which a suitable IP-VRF exists. That would also have the
pleasant effect of making R2 say “… must support bridging for non-routable
traffic” instead of “non-IP traffic”, which is technically incorrect (since per
R1 you might have non-routable IP traffic).

```
   R3: The solution must allow inter-subnet switching to be disabled on
   a per VLAN basis on PEs where the traffic needs to be backhauled to
   another node (i.e., for performing FW or DPI functionality).
```

What’s “switching”? The document is about routing vs. bridging, which do you
mean? I think you mean “routing”. IMO you should get rid of the word
“switching” and replace with something less ambiguous, e.g. “routing”. (Both
here and the one other place in the doc where you use “switching”.)

Also, I think you don’t mean “i.e.”, I think you mean “e.g.”. The meaning of
“i.e.” is “in other words”. The meaning of “e.g.” is “for example”. The best
way to avoid these problems, IMO, is to simply write out what you mean, so in
this case write “(for example, for performing FW or DPI functionality).” (And
oh by the way, you haven’t defined or expanded FW or DPI, please do so.)

3. Section 4

```
   o  references to ARP table in the context of asymmetric IRB is a
      logical view of a forwarding table that maintains an IP to MAC
      binding entry on a layer 3 interface for both IPv4 and IPv6.
      These entries are not subject to ARP or ND protocol.
```

This passage shines a spotlight on the fact that “ARP table” as it’s used in
this document is a misnomer, since it’s a table that is not (necessarily)
populated by ARP. I don’t propose that you change the nomenclature, since it’s
firmly established even though wrong — but it might be worth adding the first
sentence or one like it to your Terminology section.

4. Section 4

Figure 2 depicts BT2 being present on the ingress PE, but the text makes it
clear that in the symmetric mode that this figure depicts, BT2 doesn’t actually
need to be there. Wouldn’t it be clearer if you didn’t show it?

5. Section 4

I have a hard time parsing this text:

```
                                              Each BT on a PE is
   associated with a unique VLAN (e.g., with a BD)
```

So, 1 VLAN —> at least 1 BT (1:many)

```
                                                  where in turn it is
   associated with a single MAC-VRF
```

So, 1 MAC-VRF —> at least 1 BT (1:many)

```
                                    in the case of VLAN-Based mode or a
   number of BTs can be associated with a single MAC-VRF in the case of
   VLAN-Aware Bundle mode.
```

So, 1 MAC-VRF —> at least 1 BT (1:many)

Since this is stated as an exception I guess that means you meant the preceding
two (that I parsed as 1:many) are actually supposed to be 1:1? If so I think
this needs a rewrite (it probably does regardless, for clarity).

6. Section 4.1

When you write “Internet standard bit order“, do you mean “network byte order“?
Although even network byte order appears to be non-applicable, since the values
are shown with an explicit byte order.

I realize the definitions are merely pasted from RFC 5798 and that ship has
sailed, but unless you can explain what “(in hex, in Internet standard
bit-order)” is supposed to mean, I suggest removing it. (Alternately and less
desirably, make it explicit that you’re providing a direct quotation of RFC
5798.)

7. Section 5.1

You say the Encapsulation Extended Community and Router’s MAC Extended
Community have to be sent, but you say nothing about the required values. For
Router's MAC, §8.1 specifies the required value, I suggest a forward reference
to it. For Encapsulation, the closest I was able to find to a place where this
is specified was section 9.1.1, but that's only an example. There really needs
to be some place where it's spelled out. A bare minimum would be to cite RFC
9012 §4.1, but that just provides the syntax -- you really should say something
more about how to decide what value to send. For that matter, it could be what
valueS to send -- is it legal for a NVE to advertise multiple Encapsulation
Extended Communities? You don't say it isn't, and there are potential reasons
to do so.

8. Section 5.2

```
   o  Using MAC-VRF Route Target (and Ethernet Tag if different from
      zero), it identifies the corresponding MAC-VRF (and BT).  If the
      MAC- VRF (and BT) exists (e.g., it is locally configured) then it
```

You use “e.g.” so I presume there might be other reasons the MAC-VRF and BT
might exist even if not locally configured?

```
      imports the MAC address into it.  Otherwise, it does not import
      the MAC address.

   o  Using IP-VRF route target, it identifies the corresponding IP-VRF
      and imports the IP address into it.
```

You don’t provide any conditional language in this bullet about “if the IP-VRF
exists”. Why is that caveat required for MAC-VRF but not for IP-VRF?

9. Section 5.2

```
   The inclusion of MPLS label2 field in this route signals to the
   receiving PE that this route is for symmetric IRB mode and MPLS
   label2 needs to be installed in forwarding path to identify the
   corresponding IP-VRF.
```

I was unable to make head nor tail of this paragraph. I suppose §5.4 is where
the behavior is actually specified, so in a way it doesn’t matter (although
maybe a forward reference would help).

10. Section 5.2

```
   If the receiving PE receives this route with both the MAC-VRF and IP-
   VRF route targets and if the receiving PE does not support either
   asymmetric or symmetric IRB modes, then if it has the corresponding
   MAC-VRF, it only imports the MAC address.  Otherwise, if it doesn't
   have the corresponding MAC-VRF, it must not import this route.
```

If it doesn’t support either asymmetric or symmetric IRB modes, then doesn’t
that mean it doesn’t implement this specification at all? In that circumstance,
how do you expect your “must not” to be respected?

11. Section 5.3

```
                               If host B's (MAC, IP) has not yet been
   learnt either via a gratuitous ARP OR via a prior gleaning procedure,
   a new gleaning procedure MUST be triggered
```

Since you’ve used MUST here, you MUST provide a reference to where the “new
gleaning procedure” is specified.

Also, has not been learnt by whom? The procedure must be triggered where?

12. Section 5.3

The second paragraph, that begins "Consider a subnet A", is tremendously
confusing to a first-time reader (or at least to this first-time reader). I
realize you probably think you're being helpful by providing a worked example,
but as I read through it, it was the opposite of helpful. This is especially
true because §5 and its subsections is about "Symmetric IRB Procedures" -- and
the paragraph in question provides no procedures.

Some options to improve the situation --

- Remove the paragraph entirely.
- Preface the paragraph with "as an example to show why advertisement as RT-5
is required,"

13. Section 5.4

```
   o  global mode: VNI is set to the received label2 in the route which
      is domain-wide assigned.  This VNI value from received label2 MUST
      be the same as the locally configured VNI for the IP VRF as all
      PEs in the NVO MUST be configured with the same IP VRF VNI for
      this mode of operation.
```

What action is to be taken if this MUST is violated?

14. Section 6.1

```
   For asymmetric IRB mode, Router's MAC EC is not needed because
```

Please either expand “EC” or add it to your definitions section. (Also applies
to 5.1)

15. Section 6.2

```
   o  If only MAC-VRF route target is used, then the receiving PE uses
      the MAC-VRF route target to identify the corresponding IP-VRF --
      i.e., many MAC-VRF route targets map to the same IP-VRF for a
      given tenant.  In this case, MAC-VRF may be used by the receiving
      PE to identify the corresponding IP VRF
```

Do you mean “in this case, the MAC-VRF *route target* may be used…”?

16. Section 6.2

```
   If the receiving PE receives the MAC/IP Advertisement route with MPLS
   label2 field and it uses symmetric IRB mode
```

This entire section is entitled “asymmetric IRB procedures“. Why is there
specification language regarding symmetric procedures in it? (I’m pretty sure
this is not the only place this kind of problem appears.)

17. Section 7.3

```
   On the source NVE, an age-out timer (for the silent host that has
   moved) is used to trigger an ARP probe.  This age-out timer can be
   either ARP timer or MAC age-out timer and this is an implementation
   choice.  The ARP request gets sent both locally to all the attached
   TSes on that subnet as well as it gets sent to all the remote NVEs
   (including the target NVE) participating in that subnet.  The source
   NVE also withdraw the EVPN MAC/IP Advertisement route with only the
   MAC address (if it has previously advertised such a route).
```

Wouldn’t the source NVE only withdraw the route after a timeout had expired? As
you have written this paragraph, in case the silent TS has not moved, the
following would happen:

```
Time t: age-out timer fires, ARP probe is sent
Time t: NVE withdraws route advertisement
Time u > t: TS receives ARP probe, sends ARP reply
Time v > u: NVE receives ARP reply
Time v: NVE re-advertises route
```

Presumably this churn isn’t what you intended.

18. Section 9.2

How does the NVE learn what subnets are behind its attached TS?

19. Section 9.2

What about if TS4 wants to reach SN1? How does it know where to send the
packet? (I suppose the answer may be the same as for #18.)

_______________________________________________
BESS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/bess<https://www.ietf.org/mailman/listinfo/bess>