Subject: MCAP ownership transfer

From: "Hattig, Myron" <[email-masked]>
Date: Mon, 5 Apr 1999 12:53:02 -0700

Joseph,

Here is more on the MCAP issue you brought up. Your
scenario was:

> With MCAP transfer of ownership, it' appears (based purely on thought
> experiments) to be possible to get into situations when there
> is no owner of
> a
> previously allocated channel, or where a channel is
> de-allocated when it
> shouldn't.
> I hope someone in this WG can show that what I describe is
> actually a non
> issue.
>
> Following is one scenario:
>        Node N1 wants to send on ip group address G1.    It
> sends a solicit.
>        Node N2 (which is almost through winding down
> ownership of channel
> C1) sends
>              advertise (G1-> channel C1, expiry in 10 secs)
>        N1 sends advertisement (G1->C1, exp in 60 secs),
> wanting transfer of
> ownership of C1.
>        The above advertisement from N1 is lost.
>        N2 goes ahead and de-allocates C1, but N1 thinks that
> it is now the
> owner of C1.
>        A single lost packet has caused this to happen.

The problem is not quite as bad as you have depicted, but we
may want to discuss if the solution is sufficient.

Advertisments are sent at least every 10 seconds (page 24,
sec 10.5 para 3). The trigger for the new owner to start
sending advertisements is when a timer associated with a
mapping gets below 30 seconds. For example, if the leaving
owner sends advertisements at 9 second intervals with the
last occurring with an expire time of 34, the leaving
owner has additional advertisement opportunities at 25, 16, and
7 seconds.

Now, if the owner-want-to-be sends advertisements every 10
seconds and sent its last advertisement at 31 seconds (relative
to the expire time generated by the leaving owner), it
has advertisement opportunities at 21, 11, and 1 seconds
prior to the mapping expiring. Thus the owner-want-to-be
may send three advertise packets before the mapping expires.

In order for the problem you described to occur, the leaving
owner would have to loose all three advertisements from the
owner-want-to-be. Alternately the problem can occur if the
owner want-to-be looses the 34 expire time advertisement from
the leaving owner, and the leaving owner looses the two
advertisements by the owner-want-to-be. In both cases, three
packets must be lost.

1394 is a reasonably reliable protocol, thus the conclusion
of past discussions is that it is unlikely the exact three packets
will be lost that cause the channel to be deallocate while still
being used. Sound reasonable?

-myron

---

From: Joseph Joy <[email-masked]>
Date: Tue, 6 Apr 1999 11:18:30 -0700

|From: Hattig, Myron [mailto:[email-masked]]
        |Sent: Monday, April 05, 1999 12:53 PM
        |...
        |In order for the problem you described to occur, the leaving
        |owner would have to loose all three advertisements from the
        |owner-want-to-be.
        |...

I realize that in the general case more than one packet
needs to be lost in order for a channel to be
de-allocated when it shouldn't. The specific case I
describe is when the owner-to-be N1 STARTS trying to
send to address G1 at the point when current-owner N2 is
about to send it's LAST advertisement.

I believe we will actually see this situation
when multiple parties are routinely initiating/discontinuing
multicast transmissions. I do not know the probability of
loss of an unacknowledged async stream packet. It
would be fine to ignore this case if the result was self
healing, but in this case the result is that nodes could
continue transmission on a channel which has been
de-allocated (and could perhaps be reallocated for some
entirely different use).

I do not see any simple fixes that would make transfer
of channel ownership bomb-proof. Following are the outlines
of two proposed solutions (both require non-trivial changes):
1. Replace the existing distributed allocation/transfer
   of ownership mechanism by a centralized mechanism. In
   this scheme, the node that is elected as BCM also becomes
   the "ip/1394 channel allocation server" -- responsible for
   actually allocating and de-allocating channels. Details of
   the packet formats for requesting and announcing channel
   allocations, determining when channels age out, etc., need
   to be worked out.
2. Replace the existing transfer of ownership mechanism by
   a directed mechanism -- the current owner (N) sends a
   "transfer-of-ownership" packet directly to the
   owner-to-be N'. If this   "send" -- actually a
   block write transaction -- succeeds,
   N can assume transfer has happened and not
   de-allocate the channel, otherwise N de-allocates the
   channel. Details such as packet formats for requesting
   and granting transfer of ownership, determining the
   address offset of the grantee, etc., need to be worked out.

Assuming there are no simple fixes that would make transfer of
channel ownership bomb-proof, there appears to be three
courses of action:
1. Keep MCAP as is (perhaps add a note as to the possibility of
   channel ownership problems.)
2. Fix the problems in the ip/1394 draft.
3. Remove the transfer-of-ownership portions from the ip/1394 draft.

Regards,

Joseph.

-----Original Message-----
From: Hattig, Myron [mailto:[email-masked]]
Sent: Monday, April 05, 1999 12:53 PM
To: Joseph Joy; [email-masked]
Subject: MCAP ownership transfer

Joseph,

-myron

---

From: "Hattig, Myron" <[email-masked]>
Date: Tue, 6 Apr 1999 14:11:04 -0700

> de-allocated (and could perhaps be reallocated for some

Agreed, the number of packets to create the problem can be
one if the window of failure is smaller.

I don't think anyone has done sufficient testing to claim
to know how robust async streams are. Also, none of us
know how often the problem you're describing will happen. Sigh.
So lets see how hard the problem is to fix ...

>    allocations, determining when channels age out, etc., need

This is probably a bad idea because BCM devices are not required
to be IP/1394 capable, thus this requirement would require non-
IP/1394 devices to be MCAP capable. The alternative is to elect
a MCAP server. We talked about this along time ago and agreed to
avoid another "manager/server election" protocol. I still want
to avoid another election protocol because it will likely
be as complex as MCAP itself.

>    address offset of the grantee, etc., need to be worked out.

This doesn't work becuase if the leaving owner misses the msg from
the owner-want-to-be, then the leaving owner has no idea who to send
this "transfer-of-ownership" to. Also the transaction would have to
be at the MCAP layer, not the 1394 layer. The MCAP layer request
would require a single 1394 write/response (or read/response)
transaction. The MCAP layer response would require another 1394
write/response transaction.

In general we need to add a state to the leaving owner to indicate
the leaving owner has indeed left and the channel was indeed deallocated.
Owner-want-to-be(s) that missed the opportunity to acquire owernship must
then allocate a channel from the IRM then use that channel in the
mapping.

Here is a simple mechanism to do this:
3. When the leaving node deallocates the channel it contines to advertise
the mapping with an expire time of 0 for 30 seconds. The 0 expire time
indicates the channel was deallocated. We may even change the channel
value in the mapping to something outside the range of 0-63
to reinforce the fact that the channel is no longer allocated. The
want-to-be owner(s) simply starts the channel allocation/advertise
process anew.

>    channel ownership problems.)

I doubt you would let us get away with this solution ;-), but it is
very kind of you to propose it. ;-)

> 2. Fix the problems in the ip/1394 draft.

See above comments.

> 3. Remove the transfer-of-ownership portions from the ip/1394 draft.

This makes the solution too limited in my opinion, but given we don't
know to what extent MCAP will be used, my opinion is an uninformed opinion.

Good discussion!

-myron

---

From: Joseph Joy <[email-masked]>
Date: Thu, 8 Apr 1999 18:12:23 -0700

Myron's suggestion of sending special advertisements for a
deallocated channel ("3. When the leaving ..." in the thread
below) looks good! I suggest one modification: the
channel should not be not actually deallocated while the special
advertisements with expiry time 0 are being sent --
it should only be deallocated AFTER the 30 second "quarantine"
time is over. This serves 2 purposes:
  (a) other nodes have time to stop transfer on this channel
      and (more importantly)
  (b) this channel will not be allocated by some other node
      wanting to do multicast sends on the same or different
      group address until AFTER the quarantine period is over.

Following are some clarifications on the responsibilities of
nodes participating in MCAP that are not obvious (to me at least)
from the spec. However, these responsibilities must be met in
order to help ensure robust behavior. Perhaps these should
go into the draft in some form.

1. Once a node sends an advertise for some channel C, it must
be prepared to eventually either hand of the channel or retire
it. It must be prepared to do this EVEN IF it doesn't know for
sure that it is the owner. It typically will NOT know that it
is the owner when it initiates the attempt to transfer
ownership of a channel to itself. It can't "lose interest" and
abandon channel C for any reason, once it has sent the
advertisement for channel C. Some reasons why a node may
be tempted to "lose interest" just after it sent an advertisement
for channel C:
  a) It may subsequently observe an advertisement for the same
      group on a different channel, but sent by a node with
      a larger NodeID.
  b) It may subsequently decide it doesn't want to transmit
     on the group(s) mapped to channel C.
To restate the key point:  the node should not abandon the channel
EVEN IF it does not know for sure that it is truly the owner of
channel.

2. Once a node N1 receives an advertisement with expire time 0 (or
   however else the going-to-be-deallocated-for-sure channel is
   advertised), it must honor this advertisement, regardless of the
   relative value of the node ID of the node originating the
   advertisement. By "honor this advertisement", I mean that N1
   should assume that the channel is invalid effective immediately,
   and that N1 is not the owner (regardless of any prior state it
   may have built-up about this channel).

Regards,

Joseph.
-----Original Message-----
From: Hattig, Myron [mailto:[email-masked]]
Sent: Tuesday, April 06, 1999 2:11 PM
To: Joseph Joy; [email-masked]
Subject: RE: MCAP ownership transfer

...

...

---

From: "Hattig, Myron" <[email-masked]>
Date: Fri, 9 Apr 1999 11:17:29 -0700

> From: Joseph Joy [mailto:[email-masked]]
> Sent: Thursday, April 08, 1999 6:12 PM
> To: [email-masked]
> Subject: Re: MCAP ownership transfer
>
>
> time is over.
> This serves 2 purposes:

Both good reasons, I agree.

I consider this issue closed now.

-myron