Subject: [Fwd: Re: First draft of ForCES protocol from the protocol design team]

From: Jamal Hadi Salim <[email-masked]>
Date: Sun, 27 Jun 2004 23:43:43 -0400

After some discussion with Joel we have reached the follwoing conclusion
which needs to be captured.

1) By virtue of the way Forces protocol is going, the CE
will be aware of all resources and would/should not issue
configs which it believes will fail because of lack of resources.
As an example it will not issue a route add when theres no space in
the table.
This is equivalent to having pieces of the the 2pc actually done on the
CE itself.
2) The value of a 2pc is still there in situations where
two objects have to be created / updated together because they refer to
each other. The example provided by Joel on the non-work conserving
scheduler (which by virtue of stealing say bandwidth would deny other
classes of said bandwidth) serves here. I think we probably need an
even stronger example; i cant think of one at the moment.
3) multi-manager issue which was discussed in the thread on the CE
scope needs the mechanism. But if my undertsanding is right we are
saying thats out of scope.

Did i miss something Joel?

cheers,
jamal

-----Forwarded Message-----

From: Jamal Hadi Salim <[email-masked]>
To: [email-masked]
Subject: Re: First draft of ForCES protocol from the protocol design team
Date: 27 Jun 2004 15:05:26 -0400

On Sun, 2004-06-27 at 11:41, Joel M. Halpern wrote:
> It might be possible to elaborate the forwarding table case.
> The difficulty is that even here, one can easily see a way around the ACID
> requirement.
>
> Suppose that the example were elaborated:
> Suppose that the CE wishes to install a new forwarding path for destination
> D through FE2.  Given that FE1 has a working forwarding path for
> destination D, the CE wants to ensure that the new path is not used by FE1
> unless the install is successful in FE2 as well.
>
> However, the reader may realize that this could have been solved without
> ACID, just by establishing the FE2 state before the FE1 state.

Assuming the path to destination D on packets coming into FE1 was
pointing to say FE20 port E - there is a possibility of an ambiguity.
I.e you have a window of time where packets coming in via FE1 will
continue going through FE20 port E while packets coming into FE2 to
destination D now go through the (new) correct path.
Unless such races are tolerable - which is a tradeoff if the ambiguities
exist in limited timescales or a CE policy intent.

> ACID tends to apply when there are interactions with other activities.
> Either with other control activiites, or effects on the data path that
> can not be cleanly applied in order.

nod.

> The most common cases I have seen are situations where dtwo objects have to
> be created / updated together because they refer to each other.

Refer to each other or would the case of one refer to other also be
a sufficient condition?

> Let me try another example.
> Suppose one were modifying some traffic treatment using a non-work
> conserving scheduler.
>
> You start by creating a new queue, and by creating a new classify entry,
> which still points to the existing treatment.
> Then one goes to create the non-work conserving handling.  As soon as the
> CE creates the scheduling entry, there is an effect on other traffic
> behavior even if there is nothing in the queue feeding this new entry.

Good example.
You picked an example that requires a resource to be
reserved/preallocated --> implication is that such a resource is denied
to other queues once reserved. At least thats what i think you are
getting to.

The first phase of the 2pc is infact a check for availability of a
resource. It asks the question whether a transaction can be achieved.
Show stoppers to a transaction could be lack of resources, conflicts,
etc. At the end of a the first phase, no resource is
preallocated/reserved AFAIK; of course how long such a guarantee is
kept without reservation is probably implementation specific.
Typically, assuming an intelligent CE, there would be no other
transaction requesting for the same resource from the CE until either
the transaction is aborted or timed out.

Back to the forwarding example: Could you say that adjusting tables
infact falls under the same category?

> At the same time, as soon as the CE adjusts the port selection of the
> classifier, traffic will be diverted to the queue.
> If either action occurs without the other, undesired results occur.
> Thus, one wants ACID at least with a transaction on an FE.
> And of course the two pieces could have been in different FEs.

True.

> I believe this is what was behind the selection of QoS / RSVP as an
> example.  But we need the elaboration to explain why it matters.
> (Admittedly, non-work-conserving schedulers are not common.  But they do
> occur.  And the real point is to show the potential for interaction that
> can not be safely serialized.)

By definition a transaction (which could be a series of operations) is
atomic.
I believe anything that is affecting state of the NEs distributed
computation needs to be a single transaction (and by definition atomic)
across the NE. Thats why i brought the forwarding example because the
next hop selection may happen on one FE and the L2 header rewrite may
happen on the other. Same concept applies to things that affect a state
within a FE such as the work conserving example you gave.

So maybe a good way is to fit what goes together is by describing
transaction which could be made of a series of operations (refer to
header semantics and the config message).

Thoughts?

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 28 Jun 2004 10:26:08 -0400

I might rephrase this slightly, although I think the point is correct.
It is not that the CE knows how much resource there is.  It might not
know.  But it does know what requests it has outstanding, and what things
it might decide to roll back.  Basically, if the interaction between
multiple requests are all from the same CE, and if partial establishment
does not affect traffic, then the CE can maange the states itslef,
including any interaction between multiple requests it chooses to issue
simultaneously.

However, I do believe that there are cases, like the example of non-work
conserving queues (which I removed for brevity) which can have effects and
therefore motivate inclusion of ACID capabilities / logic.

Yours,
Joel