Subject: [Tsvwg] RE: iSCSI: No Framing

From: "Douglas Otis" <[email-masked]>
Date: Mon, 4 Feb 2002 12:55:47 -0800

David,

> > Those advocating a portion of the iSCSI application be in hardware
> > below the TCP transport should be willing to document information
> > exchanged between layers, and the states held within this layer.
> > You wish the IETF to endorse a scheme within a standards draft and
> > yet not be allowed to understand its implementation.
>
> That is a level of implementation detail that the IETF has historically
> not demanded the exposure of, and it would certainly be inappropriate to
> standardize.  To the extent that there are open source implementations of
> this form, the information will be disclosed in due course, other
> implementers will make their own decisions about what to make available.

I am not requesting a detailed API, but rather essential information
exchanged between this layer beneath TCP supporting framing and the
application.  An essential aspect of any application interchange is the
information exchanged with network related layers.  Keeping this information
proprietary does not allow a review.  I suspect the reason for keeping this
information private is to prevent an understanding of the scope of change
being advocated or to ensure proprietary solutions.

> > This scheme so modifies the application attempting FIM utilization, it
> > would be impractical to describe this as not altering TCP.  TCP is not
> > just a wire specification.  Changing the receive packet into a memory
> > array needs additional clarification beyond a claim this does not
> > change TCP.
>
> That is unfortunately a waste of list bandwidth.  With my WG chair hat
> firmly on, it is my conclusion that Doug Otis has failed to provide
> sufficient technical support for his argument that FIM alters TCP, and
> hence I hereby reject the procedural request that FIM be removed from the
> iSCSI draft as being a modification of TCP that would be outside the IPS
> WG's charter. Discussion of the merits of FIM is fine, and FIM may yet be
> removed from the iSCSI draft for good technical reasons (or may not) but
> debating whether FIM alters TCP is a waste of list bandwidth at this
point.

Any scheme that introduces a new network layer that directly or indirectly
interfaces with the application would be a modification of any transport.  I
do not feel I am obligated to offer technical details of this scheme, but
rather those that advocate its use should feel such obligations.

> I also state that it is the rough consensus of the IPS WG that TCP will be
> used for the first version of iSCSI, and this is stated over Doug Otis's
> continuing objections.  Doug would be better advised to work on using SCTP
> for iSCSI rather than complaining about the fact that others are not doing
> so.

I have had no objections to the use of TCP.  I have steadfastly rejected
alteration of TCP to accommodate various schemes designed to introduce
framing within or below TCP however, especially those that introduce
application specific layers below the transport.  My principle objection is
that such a scheme is possible using SCTP without alteration of network
layering.  Your continued endorsement of such a scheme is troubling.  If one
is allowed to create a new layer below and above the transport, then any
alteration of the transport becomes possible.  Granting this abuse in
principle makes any attempt by the IETF to regulate a transport meaningless.
Here you are advocating that this new layer does not require any details
offered and yet you have concluded it does not alter TCP in the absence of
public information.

> These decisions are appealable in accordance with RFC 2026, but should not
> be further discussed on this list.

My objections to this framing scheme remains unchanged.  Your endorsement of
an undocumented scheme is in violation of several principles expressed
within RFC 2026.  One being clear and concise documentation, and the other
openness and fairness.  The IPS wg has made an incorrect technical choice
through the addition of this new layer below TCP which places the integrity
of the iSCSI draft in jeopardy.  I am willing to continue this through an
appeals process, but my hope would be that this aspect of the draft is
struck prior to that exercise.

Doug

> Thanks,
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
> [email-masked]         Cell: +1 (978) 394-7754
> ---------------------------------------------------
>

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 12:01:21 +0530

Hi all,
    The question might be more of  SCTP implementation specific....   According
to the SCTP rfc2960, the data sender should not send any data packets if it doesnt
have enough Send-Windown (Peer Rwnd).... Just take the below example....

The Peer's Rwnd (at the Init time ) is 1500 bytes.
The Sender receivies a data buffer of size 6000 from its upper layer to be sent
to the Peer.... The Sender will then fragment the packets to fragments A,B,C,D of
size 1500 bytes (because the local mtu size is 1500bytes)....
Now the Sender sends the frag A to the Peer... Upon receiving the frag A, the
Peer's Rwnd will be zero.... This is informed to the Sender by sending a SACK with
rwnd as 0. The Sender which would have queued the frags B,C,D... will then
send frag B as the probing data to the Peer... The peer again will respond with
a SACK of rwnd 0 .  Now the deadlock situation occurs,  as the
    * Sender cant send any more data packets as the Peer's rwnd is zero....
    * Peer cant increase its rwnd till it releases the data chunks which it had already
       received,  but to release that it needs to reassemble the frags (ABCD) and
      deliver it to the above layer, only then it can release the buffers..... Since its
      waiting for the frags C and D...  this will never release the buffers.... so
      will not increase the rwnd.....

This situation can be avoided if the Sender doesnt sent Data messages of size
greater than the Peer's rwnd.... But the Rfc doesnt say anything about this.....
Or there should be some timers, to get out of the Deadlock situation.....

Any suggestions........

Thanks
-arun

---

From: Ashraf <[email-masked]>
Date: Tue, 05 Feb 2002 12:22:24 +0530

Arun,
    This is what RFC has to say :
"If the data receiver  run out of buffer space while still  waiting for  for more fragments
to complete the re-assembly, of the message, it should dispatch part of its inbound message
through a partial deliver API, freeing some if its buffer space in order receive  rest of
the message"

guess this should take care of the deadlock  situation you are mentioning here.

Thanks,
Mohd. Ashraf

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 13:50:10 +0530

Hi Ashraf,
    Thanks for the rsp....
ya, I agree that partial-delivery will solve the problem... But imagine an errornous
stack ( or SCTP stack which didnt implement the partial delivery API)  is on the
receiving side.... Then with the below example scenario, the Sender stack will
get stuck.... ie.,  the association will never tear down (as RTX timer is not running)
so the Sender side will be hung (with no mistake on its side....) forever.....
I think this situation should be avoided......

Thanks
-arun

Ashraf wrote:

>   ------------------------------------------------------------------------------------------------------------------------
>                            Name: Wipro_Disclaimer.txt
>    Wipro_Disclaimer.txt    Type: Plain Text (text/plain)
>                        Encoding: 7bit

--
****************************************************************
V.Arun Prasad
HCL Technologies Ltd.
51, Jawaharlal Nehru Road,
Ekkattuthangal,
Guindy Industrial Estate,
Chennai - 600097.

Contact # : 9144 - 2334174
            9144 - 2334181
            extn : 233
****************************************************************

---

From: Ashraf <[email-masked]>
Date: Tue, 05 Feb 2002 16:54:15 +0530

Hi arun,
    I think  for the new data chunk ( which we are sending  to probe for change in window), we start RTT timer and  give
appropraite treatment, when it expires.

thanks,
ashraf

---

From: Randall Stewart <[email-masked]>
Date: Tue, 05 Feb 2002 05:33:41 -0600

Arun:

Some comments:

> I think this situation should be avoided......

I will address this comment below ...

> > guess this should take care of the deadlock  situation you are mentioning here.

The above is percisely what we designed the PD-API for. It is
a part I never really much liked :> but we saw it to be the
only way out of such a dead-lock.... Also teamed with this
is the receivers right to reneg. This way in a critical situation
it can drop some of the last chunks it is holding so it can
make room for the earlier pieces so it CAN use the PD-API...

Both are necessary to ensure no deadlock can occur..

Now as to the other issue.. what happens if your sender
is broken or an attacker.. a number of things come to mind
(as yet I have not had my coffee so there may be more :0)

1) TCP I would imagine can be in the same situation .. i.e.
   if it connects to a peer that is intentionally doing something
   bad.

2) It really becomes implementation dependant. I know of one
implementation
   that will only keep sending the same chunk for so long.. and then
   it will abort the associaition.

3) If the guy is really an attacker the problem is more in to the fact
   that you have entered into the association with him. I.e don't
   leave the front door open and expect a burgular not to enter...

4) If it is a broken implementation then it needs to get fixed and I
   would hope the one stuck would at least have enough logging going
   on to show that it is broken... (or even in the case 3 I think logs
   may help) so someone can fix it..

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 17:14:05 +0530

Hi Ashraf,
    ya RTT timer will be started, but upon its expiry, we come to the same problem..
because we cant send the retransmitted packets due to rwnd shortage......

Thanks
-arun

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 18:05:41 +0530

Hi Randall,
    Considering your 4th point about logging....
        I think the below scenario can be better handled
if the implementation of "Life_Time" (The user specifying for how long the particular
packet is to be tried to send) parameter in the SEND primitive  is present.
So once the timer (Life_time timer) expires, the user can be indicated that the
data cannot be sent (again for each each data request, there should be a timer
might be costly...)
This way atleast we may come to know that something wrong is going on in the
stack, and the user may choose to send a "Shutdown" primitive...
( I guess all the optional parameters also has some basic purpose.......)

Thanks
-arun

---

From: Randall Stewart <[email-masked]>
Date: Tue, 05 Feb 2002 07:07:44 -0600

Arun:

Interesting idea... but this will not help you. The lifetime
as specified can only effect packets that have NOT been
transmitted. 

As I said.. some implementations MAY take and implement an
overall number of times a chunk may be transmitted.. and if
you keep getting it retran'd and no ack.. you get an abort :>

The u-sctp draft had something more like what you are considering
below.. but that is not currently an active document in the IETF.

There are some of us still playing with it and you can find the
last updated document on the web site

http://www.sctp.org

if you are interested...

R

---

From: Ashraf <[email-masked]>
Date: Tue, 05 Feb 2002 19:17:42 +0530

Hi,
    My understanding is ( pls correct me if I am wromg),  each time when the timer expires, the path /assoc retrans counter will be

incremented and sender keeps on probing  for change in window size  by sending next fragment ( in ur caase fragment B ) and  ending

up in T3 timer expiry each time. Will the Assoc will be  terminated once this count reached the threshold value.?"

Thanks,
ashraf

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 19:57:55 +0530

Hi Randell,
    I think we misunderstood something.....  The scenario we are considering will
not send the data packet to the network at all.... When  the Peer- rwnd is zero,
one data packet is sent for probing,  then if the endpoint still receives the SACK
with rwnd as 0, then the stack will not send any packets to the network, instead
will Queue the data chunks in the local Queue and will be waiting for the
Peer-rwnd to increase (I think only one probing data chunk is allowed when
the Peer-rwnd is zero...) . So the solution u are mentioning ( the
MAX-ASSOC-RETRANS counter ) will not help because the counter will
be increamented only when the data chunk is sent to the network.....
Hope Iam right.......

Thanks
-arun

---

From: Arun Prasad <[email-masked]>
Date: Tue, 05 Feb 2002 20:06:39 +0530

hi Ashraf,

From Rfc:
However, regardless of the value of rwnd (including if it
      is 0), the data sender can always have one DATA chunk in flight to
      the receiver if allowed by cwnd

    My understanding from the above is, trying to send the Data chunk only once
for probing, and if that chunk is not acknowledged (or acknowledged with rwnd
as zero), then even when the Rtx timer expires for that Data chunk, the data chunk
will be queued in the local Q (as the rwnd is zero, and already one data chunk
has been sent for probing...) and will not be retransimtted.....
    I dont know whether my understanding is right......  Just getting a feeling that
Iam going wrong somewhere....( might be ur understanding is right.....)
Somebody please clarify.....

Thanks
-arun

---

From: Randall Stewart <[email-masked]>
Date: Tue, 05 Feb 2002 09:44:46 -0600

You always can probe with one chunk. However when the sack
comes back saying rwnd is still 0.. you are still in the
probe mode. So at the next expiration of the T3 you will
resend the probe chunk.... until the rwnd is raised or
you give up :-)

I don't see that any failure threshold will enter into
this picture...

R