Subject: Bridging network or networked bridges

From: Muneyoshi Suzuki <[email-masked]>
Date: Tue, 05 Nov 2002 09:18:18 +0900

Since beginning of the last month, several discussions about VPLS were held
on the PPVPN WG generic requirements, L2VPN, and discovery design teams
mailing lists as well as personal communications. The teams understand
that there are many issues to be solved, and feel that the issues should
be discussed in the public. So, the teams don't reach any consensus and 
don't decide anything yet.

For further discussion, this memo summarize and sort out issues rose on 
the lists. Note that some of issues addressed in this mail may be beyond
the scope of the IETF, but may belong to the IEEE 802 committee. So, 
discussions from technical as well as administrative viewpoint for possible
directions are expected.

Thanks,

Muneyoshi Suzuki

---
1. Reference

Bridge is defined in IEEE 802.1D-1998, 802.1t-2001 (amendment to 802.1D),
and 802.1w-2001 (RSTP extension). 802.1D and 802.1t specify Bridge 
architecture, MAC frame relay operation, Bridge protocols (such as STP, 
GARP, and GMRP), and management protocol. And 802.1w specifies RSTP.

Note that the 802.1 WG is now discussing 802.1y (corrections to 802.1D,
802.1t and 802.1w). In this process, STP may be officially obsoleted
and replaced by RSTP.

Note that Bridge implementation of many products in the market is still 
based on 802.1D-1993 or earlier specification. Also note that there are 
many cheap Bridge products that only support MAC frame relay operation and 
don't implement Bridge and management protocols. If a user install this 
type product, loop free deployment is the user responsibility.

VLAN is defined in IEEE 802.1Q-1998, 802.1u-2001 (corrections to 802.1Q), 
802.1v-2001 (protocol/port-based VLAN extension). 802.1Q, 802.1u, and 802.1v
define extension of Bridge defined in 802.1D-1998. It specify extended 
Bridge architecture, VLAN tagging and MAC frame relay operation based on 
VLAN tag, and GVRP. Note that the VLAN defined in the IEEE 802.1 is NOT the
technology that emulates multiple logical LANs over a single physical LAN.

Note that the 802.1 WG is now discussing 802.1s (MSTP extension), and 
some VLAN-aware Bridge products support per-VLAN based STP, which is 
proprietary protocol. Also note that some VLAN-aware Bridge products 
in the market support the stacked VLAN mechanism which is intended 
used by SPs, enables transparent forwarding of customer VLAN tag in 
the SP network, but is not yet standardized.

2. Terminology used in this memo

1982-style Bridge: A Bridge that supports only MAC frame relay operation,
and the operation is based current or earlier 802.1D specification.

1998-style Bridge: A Bridge that conforms to IEEE 802.1D-1998 and 802.1t-
2001. It may support RSTP defined IEEE 802.1w-2001, but does not support 
VLAN capability defined in IEEE 802.1Q-1998.

VLAN-aware Bridge: A Bridge that conforms to IEEE 802.1D-1998, 802.1t-2001,
802.1Q-1998, 802.1u-2001, and 802.1v-2001. It may support RSTP defined IEEE 
802.1w-2001 or MSTP may be defined as IEEE 802.1s.

Bridge: 1982-style, 1998-style, or VLAN-aware Bridge.

Bridged LAN: A concatenation of individual LANs interconnected by Bridges.

LAN: LAN technology that provides the MAC service and is defined in 
a series of IEEE 802 LAN standards such as IEEE 802.3 Ethernet LAN.

Note that "LAN segment" means the medium connection, e.g., 1000BASE-T, 
between Medium Dependent Interfaces in a LAN. The use of this terminology 
may be inappropriate in the VPLS context, because it is not intended to 
provide medium dependent service.

3. Customer View

VPLS service is enabled by a logical Bridged LAN, which consists of 
physical or emulated LANs and Bridges. This Bridged LAN provides 
independent VPLS service instances per customer. From the viewpoint of 
VPLS service customers, a service instance emulated by VSIs in PEs for a 
customer is equivalent to:

  o A LAN
  o A single Bridge, that is: 
    - A 1982-style Bridge
    - A 1998-style Bridge
    - A VLAN-aware Bridge
  o A Bridged LAN composed by:
    - 1982-style Bridges (loop free topology must be ensured)
    - 1998-style Bridges
    - VLAN-aware Bridges
    - Bridges

Question: Expected scenario from customer's perspective.

Note that if a customer interconnects customer's VLAN-aware Bridges 
through a VPLS service instance, it may have to be equivalent to a LAN, 
a VLAN-aware Bridge, or a Bridged LAN composed by VLAN-aware Bridges.

One of issues at here is what should be emulated by a VSI in a PE or VSIs 
for a VPLS customer. A single LAN or a single Bridge may be emulated by 
VSIs in PEs belonging to an SP network. Also, a single VSI in a PE may 
emulate a single Bridge (remote-Bridge) and emulated Bridges may be 
interconnected by Ethernet pseudo wires.

The other issue is how the service instance handles the Bridge protocols 
such as STP/RSTP/MSTP and GARP/GMRP/GVRP transfered through BPDUs, as well
as, LCAP and MAC control protocol for PAUSE operation received from/sent to
the customer site. MAC frames used by these protocols are sent to 
multicast addresses. Emulated 1998-style and VLAN-aware Bridges must 
terminate most of these frames, then appropriate action based on the 
standard must be executed.

On the other hand, emulated LAN and 1982-style Bridge must transparently
forward most of these frames. This may decrease load in PEs. Note that 
emulated LAN, that learns customer's MAC addresses, and 1982-style Bridge 
must transparently forward customer STP/RSTP/MSTP BPDU frame, because these 
don't join the spanning tree. However, when a customer network topology 
is changed, the MAC address cache in VSIs for the customer should be 
flushed to reduce the time for topology change. So, a STP/RSTP/MSTP snooping 
mechanism may be required for these implementation.

4. SPs Requirements

VPLS service providers must identify order of the maximum number of the
customers (e.g., # of service instances, CEs, etc) to be supported. SPs 
may restrict maximum number of the MAC addresses supported by a single VPLS 
service (or a single customer site).

Question: Order of the maximum number of the customers assumed by SPs.
Thousands, tens of thousands, hundreds of thousands, or millions.....
And, the maximum number of the MAC addresses supported by a single VPLS 
service. Tens, hundreds, thousands, or tens of thousands......

Note that theses values may strongly impact on MAC forwarding table size 
in the VPLS.

5. Implementation Model

VPLS service is enabled by a logical Bridged LAN, which consists of 
physical or emulated LANs and Bridges. This Bridged LAN provides 
independent VPLS service instances per customer. A portion of the Bridged 
LAN is emulated by VSIs interconnected by Ethernet pseudo wires.

5.1 Support of VPLS Service

Port-based VLAN (defined in Annex D of IEEE 802.1D, 802.1u and 802.1v) 
enables per-customer based VLAN support, if a port corresponds to a single 
customer. Thus, VLAN-aware Bridges can be used for the support of VPLS 
service in the Bridged LAN.

However, the current VLAN standard support only 4094 VLANs in the Bridged 
LAN. Also it does not emulate a VLAN-aware Bridge for a customer, so the 
customer network cannot use the VLAN. These may be unacceptable restrictions
for VPLS service providers and customers.

Therefore, development of new layer 2 technology that supports SP class 
VLAN service is indispensable. Note that the technology to be developed may
be independent from layer 3 technology such as IP and MPLS. Stacked-VLAN 
aware Bridge, described in section 1, is one of candidates for solution.
Encapsulation Bridge which support MAC-in-MAC encapsulation is the other 
candidate. Stacked-VLAN aware Bridge may have to learn customer MAC 
addresses, while MAC-in-MAC may not have to learn these, so the latter 
may be able to reduce MAC address table size.

Note that there is a possibility that the same MAC addresses are assigned
to different Ethernet interfaces. VRRP assigns the same virtual MAC 
addresses to different interfaces. Per-VLAN based STP, described in 
section 1, also use a similar scheme. Thus, if a customer uses VLANs in 
the customer network, and if the same MAC addresses appear in different 
customer VLANs, current Stacked-VLAN aware Bridge implementation may not 
distinguish these MAC addresses, because it does not care customer VLAN 
identifiers. 

5.2 LAN/Bridge Emulation

PWE3 WG is discussing point-to-point Ethernet pseudo wire, which is
described in the following drafts.

draft-ietf-pwe3-control-protocol-00.txt
draft-ietf-pwe3-ethernet-encap-00.txt

PPVPN WG will use this pseudo wire as an emulation technology in the 
logical Bridged LAN that provide VPLS service. A part of the Bridged LAN 
is emulated by VSIs interconnected by Ethernet pseudo wires. That is 
equivalent to:

  o A LAN
  o A single Bridge
  o A Bridged LAN
  (or something like these)

The emulated part must ensure loop free MAC frame forwarding, however the
use of STP may be inappropriate in WAN environment. Therefore, the WG is
discussing topology restriction of pseudo wires that interconnect VSIs.

One of solutions is tree structure topology, here a VSI is a node and a
pseudo wire is a link, however details of this approach are not proposed
yet. Another solution is full mesh topology with split-horizon forwarding
scheme proposed in draft-lasserre-vkompella-ppvpn-vpls-02.txt.

The split-horizon forwarding scheme enables loop free MAC frame forwarding.
A VSI forwards a MAC frame received from a customer to others VSIs, but it 
never forward a frame received from a VSI to another VSI. It also learns
MAC addresses received from the customer network. Obviously, in this 
scheme, a VSI doesn't emulate a remote-Bridge defined in IEEE 802.1G.

Thus, one of issues at here is what is emulated by full mesh topology with
split-horizon forwarding scheme.

  o It is an emulated LAN, because it transparently forwards MAC frames.

  o It is an emulated Bridge, because it terminates MAC layer, then 
    forwards MAC frames to another interface, so it can interconnect 
    different Ethernet mediums, furthermore, it learns MAC addresses for 
    forwarding.

Note that if it is an emulated Bridge, at least, it may have to be a VLAN-
aware Bridge for support of VPLS service in the Bridged LAN.

The other issue is scalability. Obviously, full mesh topology is not scale,
if number of VSIs is increased. Support of broadcast and multicast is also
problematic, because it wastes bandwidth in the SP network and increases
jitter of MAC frame forwarding.

5.3 Routing for LAN/Bridge Emulation 

Topology restriction described in the previous section may not be 
acceptable for SPs, because SPs can not deploy PEs freely and full mesh 
topology may not scale.

However, if a VSI emulates a remote Bridge, currently only STP/RSTP can be
used for routing. RSTP much improved recovery time, however, we need more 
experiments of the use of RSTP in WAN environments.

The final issue is, if RSTP works in WAN environments, it only blocks 
links and constructs a tree or hierarchical tree topologies. It may not 
efficiently use link bandwidth. So, minimum OSPF or BGP-4 extension for 
MAC routing support may be required.

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 5 Nov 2002 22:46:52 -0500

Muneyoshi,
Thanks for sending this out. Some questions inline.
It may be useful to list the issues not in PPVPN, the issues in IEEE
realm, etc?

> standard must be executed.
Is this not in IEEE realm? (in the IEEE entity of the VSI)

> mechanism may be required for these implementation.
Are there some inconsistencies in the requirements?
A 1982-style bridge is expected to snoop RSTP/MSTP?

> aware Bridge, described in section 1, is one of candidates for solution.
Is the PW ID considered a candidate solution?

> may be able to reduce MAC address table size.
Is the latter addresses aggregatable, will protocols similar to "routing
protocols" be developed then?

> distinguish these MAC addresses, because it does not care customer VLAN
This is an implementation issue?

> scheme, a VSI doesn't emulate a remote-Bridge defined in IEEE 802.1G.
What are the problems of not emulating a remote-Bridge in this case?

---

From: Muneyoshi Suzuki <[email-masked]>
Date: Wed, 06 Nov 2002 14:43:17 +0900

Cheng-Yin,

Thanks. I summarized the issues but I don't have authority to select a
specific approach, it should be decided based on the WG consensus. I'm 
expecting WG discussion. So, I answer my personal feeling.

> Is this not in IEEE realm? (in the IEEE entity of the VSI)

Odiously the IEEE 802 is responsible for the protocol specification.
However, IMHO, emulation technology is the realm of the PPVPN WG.

> > flushed to reduce the time for topology change. So, a STP/RSTP/MSTPsnooping

I think the snooping mechanism is not indispensable but desired.
A 1982-style bridge may also be expected to support it.

> Is the PW ID considered a candidate solution?

This section does not discuss LAN/Bridge emulation technology such as PWs.

> protocols" be developed then?

We can say it is aggregation. But it seems to me an tunneling/layered/
hierarchical model rather than aggregation.

> This is an implementation issue?

There is no standard that specify a Stacked-VLAN aware Bridge.
So, this is an implementation issue.

> What are the problems of not emulating a remote-Bridge in this case?

I think we must identify what is emulated by it, because we must verify
conformity with existing LAN/Bridge implementations.

Thanks,

Muneyoshi Suzuki

---

From: Muneyoshi Suzuki <[email-masked]>
Date: Fri, 15 Nov 2002 17:48:22 +0900

This is updated version.

Thanks,

Muneyoshi Suzuki

--------------------
1. Reference

technology that emulates multiple logical LANs over a single LAN segment.

used by SPs, and enables transparent forwarding of customer VLAN tag in 

Bridge: A 1982-style, 1998-style, or VLAN-aware Bridge.

3. Customer View

  o A LAN

An issue at here is what should be emulated by a VSI in a PE or VSIs for 
a VPLS customer. A single LAN or a single Bridge may be emulated by 
VSIs in PEs belonging to an SP network. Also, a single VSI in a PE may 
emulate a single Bridge (remote-Bridge) and the emulated Bridges may be 
interconnected by Ethernet pseudo wires.

4. SPs Requirements

VPLS service providers must estimate order of the maximum number of
customers (e.g., # of service instances, CEs, etc) to be supported. SPs 
may restrict maximum number of MAC addresses supported by a single VPLS 
service (or a single customer site).

Questions: Order of the maximum number of customers assumed by SPs.
Thousands, tens of thousands, hundreds of thousands, or millions.....
And, the maximum number of MAC addresses supported by a single VPLS 
service. Tens, hundreds, thousands, or tens of thousands......

5. Implementation Model

Port-based VLAN (defined in Annex D of IEEE 802.1Q, 802.1u and 802.1v) 
enables per-customer based VLAN support, if a port corresponds to a single 
customer. In this case, a VPLS service instance for a customer may be
equivalent to a single 1998-style Bridge or a Bridged LAN composed by 
1998-style Bridges, except that the all customers share a single STP 
entity. Thus, VLAN-aware Bridge can be used for the support of VPLS 
service in the Bridged LAN.

However, the current VLAN standard supports only 4094 VLANs in a Bridged 
LAN. Also it does not emulate a VLAN-aware Bridge for a customer, so the 
customer network cannot use the VLAN. These may be unacceptable restrictions
for VPLS service providers and customers.

Encapsulation Bridge which supports MAC-in-MAC encapsulation is the other 

Note that MAC addresses are not unique. VRRP assigns the same virtual MAC 

5.2 LAN/Bridge Emulation

draft-ietf-pwe3-control-protocol-01.txt
draft-ietf-pwe3-ethernet-encap-01.txt

PPVPN WG will use this pseudo wire as an emulation technology in the 
logical Bridged LAN that provide VPLS service. A portion of the Bridged LAN 
is emulated by VSIs interconnected by Ethernet pseudo wires. That is 
equivalent to:

  o A LAN
  o A single Bridge
  o A Bridged LAN

One of important issues is that the layer 3 technology is able to support
logical LANs/Bridges, however, emulated service may provide different 
communication quality, such as throughput, delay time, error rate, and 
reliability, from physical LANs/Bridges. The VPLS service instance handles 
the Bridge protocols such as STP/RSTP/MSTP and GARP/GMRP/GVRP transfered 
through BPDUs, as well as, LCAP and MAC control protocol for PAUSE 
operation received from/sent to the customer site.

These protocols may assume LAN class communication quality, however, for 
the emulated portion, it is not easy to support LAN quality. Therefore, 
robustness of these protocols must be identified. If it is problematic to 
use these protocols in WAN environment, which may be low-throughput, long 
delay, high error rate, low-reliability, in addition, frames may be miss-
ordered or duplicated, MAC frames for these protocols must be forwarded 
in high-priority or through a special communication path.

The other issue is how the service instance handles these protocols.
MAC frames used by these protocols are sent to multicast addresses. Emulated
1998-style and VLAN-aware Bridges must terminate most of these frames, 
then appropriate action based on the standard must be executed. On the 
other hand, emulated LAN and 1982-style Bridge must transparently forward 
most of these frames. This may decrease load in PEs. 

Note that emulated LAN, that learns customer's MAC addresses, and 1982-style
Bridge must transparently forward customer STP/RSTP/MSTP BPDU frames, because
these don't join the spanning tree. However, when a customer network topology 
is changed, the MAC address cache in VSIs for the customer should be 
flushed to reduce the time for topology change. So, a STP/RSTP/MSTP snooping 
mechanism may be required for these implementation.

Also note that a loop may occur in a customer LAN whether the VPLS service
terminates customer STP/RSTP/MSTP BPDU frames or not. However, it is not
practically impossible to detect a customer loop in the SP network. If 
the SP network detect it, the SP is able to notify that the fact to the 
customer.

5.3 Split-horizon forwarding scheme

The emulated portion in the logical Bridged LAN must ensure loop free and 
in-sequence MAC frame forwarding, however the use of STP may be inappropriate
in WAN environment. Therefore, the WG is discussing topology restriction 
of pseudo wires that interconnect VSIs.

  o It is an emulated VLAN, because it transparently forwards MAC frames.

Note that this scheme does not terminate STP/RSTP/MSTP. If a pseudo wire 
fails, it may be restored using a layer 3 protection mechanism. In this
case, there is a possibility that the recovery time of the layer 3 mechanism
may affect failure detection of STP/RSTP/MSTP.

Also note that if a single pseudo wire in the mesh fails, this is logically 
equivalent to the situation that the particular two ports in a HUB/Bridge
cannot communicate each other but the remains are normal. So, it is 
important to confirm that the STP/RSTP/MSTP as well as the others Bridge 
protocols properly work in this kind of situation.

5.4 Routing for LAN/Bridge Emulation 

Topology restriction described in the previous section may be undesirable 
for SPs, because SPs can not deploy PEs freely and full mesh topology may 
not scale.

The final issue is, if RSTP works in WAN environments, it only blocks 
links and constructs a tree or hierarchical tree topologies. It may not 
efficiently use link bandwidth. So, minimum OSPF or BGP-4 extension for 
MAC routing support may be required. Note that this extended MAC routing 
protocol may be closed to layer 2, it does not need to interact with IP 
routing protocol.