Subject: [armd] problem statement and scope of ARMD

From: Liyizhou <[email-masked]>
Date: Thu, 28 Jul 2011 15:13:30 +0000

Hi Thomas and folks,

In the problem statement draft (draft-narten-armd-problem-statement<http://tools.ietf.org/id/draft-narten-armd-problem-statement-00.txt> ), it touches scalbility issue mostly. I think there are another set of problems regarding ARP/ND introduced by vm migration. (I use ARP to represent both ARP/ND in the following text)

 ARP was designed for address resolution, however in virtualized DC case, it is also used as an indication of completion of migration now. There are a number of vulnerabilities to use ARP as such indication (some details could be found in draft-liyz-armd-vm-migration-ps<http://tools.ietf.org/id/draft-liyz-armd-vm-migration-ps-01.txt>),

1. No ARP "forget". Blackhole issue. It is expected that flooded ARP message after migration would re-fresh all relevant MAC table entries to solve the blackhole issue. However it is not always true. (see bullet 2)

2. Unreliably delivery. With increasing scale of network, it is unavoidable that ARP message gets lost in congestion when it is used as indication of migration. It makes blackhole issue worse. At some location, frames are kept being sent to the blackhole until its MAC entry timeout. Simply sending *mutiple* ARPs may make it recover from message loss. However it causes scalibility issue worse.

3. Duplicate address detection (DAD). ARP was used for duplicate address detection (but there are different implementations on this). In migration case, there must be some suspension time of vm. Length of suspension time depends on how much changed memory contents need to be copied from old location to new location. During suspension period, normal DAD would not function effectively as VM is not able to respond to ARP message for DAD. Theoretically legally "stealing" of IP address by other host may happen.

I hope ARMD addresses both ARP's scalability issue and ARP's vulnerability issue when it is used as an indication of vm migration. I would like to see ARMD cover both in its scope.

Yizhou

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 28 Jul 2011 17:49:56 -0500

Hi, Yizhou.

On 7/28/11 10:13 AM, "Liyizhou" <[email-masked]> wrote:

> In the problem statement draft (draft-narten-armd-problem-statement), it
> touches scalbility issue mostly. I think there are another set of problems
> regarding ARP/ND introduced by vm migration. (I use ARP to represent both
> ARP/ND in the following text)

Yes, the current Problem Statement draft is intentionally narrow. It can be
expanded to include more scope - we need input, such as your input below, to
discover what the WG wants. This will be discussed during our WG meeting on
Friday, and I encourage you to provide input there in addition to your input
on the list.

>  ARP was designed for address resolution, however in virtualized DC case, it
> is also used as an indication of completion of migration now.
> ...

If ARP is used purely as an "indication of completion" then that may not be
related to address resolution, and may not be in-scope for ARMD. (e.g. if
the MAC address does not change during a migration, the gratuitous ARP is
not happening for the benefit of address resolution)

> 1. No ARP "forget". Blackhole issue. It is expected that flooded ARP message
> after migration would re-fresh all relevant MAC table entries to solve the
> blackhole issue. However it is not always true. (see bullet 2)

Bridge learning is not in-scope for ARMD, and therefore e.g. a relatively
small number of gratuitous ARP messages that are sent purely to update
switch MAC forwarding tables are not something we should focus on.  On the
other hand, if VM migrations happen so frequently that gratuitous ARP
messages overwhelm the network, then we should quantify that. I suspect that
migration causes a small number of ARP messages (~1 message) compared to
normal operation (many messages). Do you have a different view of this?

Regardless, at the previous IETF meeting, we suggested that an "ephemeral"
draft might be used to capture other related issues. Perhaps you would
consider writing a contribution to such a document, briefly explaining this
issue and/or providing pointers to references?

> 2. Unreliably delivery. With increasing scale of network, it is unavoidable
> that ARP message gets lost in congestion when it is used as indication of
> migration. It makes blackhole issue worse. At some location, frames are kept
> being sent to the blackhole until its MAC entry timeout. Simply sending
> *mutiple* ARPs may make it recover from message loss. However it causes
> scalibility issue worse.

This sounds like something that the WG should consider for the Problem
Statement. In essence, my understanding of the problem is that ARP lacks a
reliability mechanism, and fails (without graceful indication) during
network congestion. What I'd like to understand further is whether this is a
more significant problem than e.g. the fact that everything can fail during
congestion.

I encourage further list discussion of this point.

> 3. Duplicate address detection (DAD). ARP was used for duplicate address
> detection (but there are different implementations on this). In migration
> case, there must be some suspension time of vm. Length of suspension time
> depends on how much changed memory contents need to be copied from old
> location to new location. During suspension period, normal DAD would not
> function effectively as VM is not able to respond to ARP message for DAD.
> Theoretically legally "stealing" of IP address by other host may happen.

This is also interesting, and may be included in the Problem Statement if
there is WG interest to do so. The problem as I understand it, is that DAD
assumes that an address is reserved only if a host is active and able to
respond, and therefore any mechanism that temporarily disconnects a host
creates this potential conflict. In other words, DAD doesn't accommodate the
VM migration use-case unless the migration completes within a certain amount
of time. Do you agree with this characterization?

Do you believe that this problem applies to other situations, e.g. DHCP
leases expiring during a VM migration, etc?

> I hope ARMD addresses both ARP's scalability issue and ARP's vulnerability
> issue when it is used as an indication of vm migration. I would like to see
> ARMD cover both in its scope.

This seems like a reasonable request, but I'd like to see more discussion of
the issues - and I'd like to hear from other WG participants, whether they
agree these should be in the Problem Statement.

Cheers,
-Benson

---

From: Liyizhou <[email-masked]>
Date: Fri, 29 Jul 2011 13:33:15 +0000

Hi Benson,

Please see inlines with [yz] .

[yz] Sure.

[yz] It is also used for updating the interface number of ARP table entry though IP/MAC should keep unchanged in migration.

[yz] I agree with it.

[yz] I believe so.  Normal congestion control may use packet re-transmission on timeout or reducing packet transmission rate mechanisms. Those do not solve the blackhole issue. Gratuitous ARP does not have reliable re-transmission mechenism. Limiting transmission rate by traffic sessions would be helpful to alleviate overall congestion. It may consequently increase the possibility of next gratuitous ARP to be succesfully delivered. However blackhole is not removed until receiving succesful gratuitous ARP or ARP reply sent by migrated vm. Its duration varies a lot.

[yz] Yes.

Do you believe that this problem applies to other situations, e.g. DHCP
leases expiring during a VM migration, etc?
[yz] I do not know at the moment. Lease renew is expected to be requested by end host. It tries to renew the lease at 50% of lease time. If fails, try again (with broadcast) at 87.5% of lease. Not too sure what may happen if vm tries to renew the lease at suspension time. Will investigate further. 

Cheers,
-Benson