Subject: Re: [storm] What if a SCSI Response PDU has GOOD status anda DataSegment?

From: "Knight, Frederick" <[email-masked]>
Date: Tue, 1 Dec 2009 09:02:02 -0500

I agree, Option 1 is preferred.  The direction at T10 so far, is that
nothing is done at the SCSI layer to enable this capability.  As you
noted for iSCSI, the other transports basically say the same thing -
they do not prohibit it.

The changes at T10 have so far, indicated no transport layer changes.

	Fred

-----Original Message-----
From: Mallikarjun Chadalapaka [mailto:[email-masked]] 
Sent: Monday, November 30, 2009 9:50 PM
To: 'Julian Satran'; [email-masked]
Cc: [email-masked]
Subject: Re: [storm] What if a SCSI Response PDU has GOOD status anda
DataSegment?

IMHO, I think the right forum to make a call on this is T10.  Let's
leave it
to T10 to define the proper behavior.  If T10 decides that "GOOD with
DataSegment" can simply be done without any new SCSI requirements, I
think
we'll have two options.

*Option 1*	
iSCSI should state the following in the main text and elaborate as an
implementer's advice.

Require that "GOOD with DataSegment" status be correctly handled by
*iSCSI*
implementations - i.e. although it doesn't imply for higher layers in
the
stack.  Next revision of consolidated iSCSI draft should simply make it
clear in that a DataSegment is allowed with any non-CHECK CONDITION
status
(RFC 3720 does not prohibit it either).

*Option 2*  
Define a new iSCSI negotiation key to explicitly allow "GOOD with
DataSegment" (or more generally, non-CHECK CONDITION) at the iSCSI
level.
   
If T10 OTOH defines new SCSI-isms to formally sanction this case (e.g.
mode
pages, commands etc.), iSCSI doesn't need to change anything although
still
an iSCSI implementer's note wouldn't hurt.

My preferred option is #1 - i.e. I like us not to add an iSCSI
negotiation
key for a SCSI feature.  It's something we have avoided doing so far by
design.

Thanks.

Mallikarjun

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Julian Satran
> Sent: Thursday, November 26, 2009 1:22 AM
> To: [email-masked]@emc.com
> Cc: [email-masked]
> Subject: Re: [storm] What if a SCSI Response PDU has GOOD status and a
> DataSegment?
> 
> David,
> 
> The key is a good idea ... but not a great one :-)
> As you certainly know there are many storage controllers (targets) on
> the market that support
> iSCSI, FCP, iSER and even several at the same time (for different
> sessions usually as we and T10 never seriously considered I_T nexi on
> different transports).
> 
> Their SCSI layer (in the ones I know about) is completely transport
> oblivious so the transport is not really the layer you want to
> introduce status+sense for statuses other than bad (unit check).
> 
> At the SCSI level it may make a lot of sense (no pun intended either).
> iSCSI will support it (without any key negotiation being required) as
> iSCSI does not prohibit it :-). I don't recall exactly what FCP had to
> say on this but I recall vaguely that there is no explicit
> interdiction. However some silicon vendors might be "surprised" if it
> happens.
> 
> Regards,
> Julo
> 
> 
> On 25/11/2009, at 19:57, <[email-masked]> <[email-masked]>
> wrote:
> 
> > [WG chair hat OFF]
> >
> > Here's a little more background.  There's an open issue in T10
> > about whether sending sense data with GOOD status can just be
> > done, vs. whether the target ought to know in advance that
> > the initiator is prepared for this.
> >
> > In keeping with the IETF advice to be "conservative in what
> > is sent and liberal in what is accepted", if T10 decides not
> > to address this at the SCSI level, then it'll be reasonable
> > to define an iSCSI negotiation key to indicate that the
> > initiator is prepared to accept (in the sense of "tolerate",
> > i.e., won't cause an error or worse, rather than any promise
> > to "process" or "understand") sense data for GOOD status.
> >
> > In addition (as Ralph suggests), it may make sense (pun intended)
> > to define this key to cover all status values, not just GOOD,
> > even though only GOOD is currently under discussion in T10.
> >
> > Thanks,
> > --David
> >
> >
> >> -----Original Message-----
> >> From: [email-masked] [mailto:[email-masked]]
> >> On Behalf Of Ralph Weber
> >> Sent: Tuesday, November 24, 2009 7:14 PM
> >> To: [email-masked]
> >> Subject: [storm] What if a SCSI Response PDU has GOOD status
> >> and a DataSegment?
> >>
> >> One of the questions that T10 has been wrestling with lately
> >> concerns a case where their equivalent of the iSCSI Response
> >> PDU contains a status other than CHECK CONDITION and sense
> >> data in a Data Segment.
> >>
> >> What will initiators do if handed one of these beasts?
> >>
> >> Note: GOOD is the current favorite doppelganger for CHECK
> >> CONDITION, but BUSY et al. should be considered too when
> >> thinking about what initiators might do.
> >>
> >> The current belief in T10 is that initiators most likely
> >> will ignore any Data Segment found in a Response PDU with
> >> GOOD status. This is based on the notion that initiators
> >> do not squander the extra cycles needed to validate entire
> >> format of a Response PDU.
> >>
> >> What do the implementers of iSCSI initiators think?
> >>
> >> All the best,
> >>
> >> .Ralph
> >>
> >> _______________________________________________
> >> storm mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/storm
> >>
> >>
> > _______________________________________________
> > storm mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/storm
> 
> _______________________________________________
> storm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/storm

_______________________________________________
storm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/storm

---

From: Mike Ko <[email-masked]>
Date: Fri, 04 Dec 2009 15:41:14 -0800

I have just submitted an updated iSER draft to the storm WG.  The following 
are extracted from section 14 on "Summary of Changes from RFC 5046":

1. Removed the requirement that a connection be opened in "normal"     TCP 
mode and transitioned to zero-copy mode.  The allows the spec to conform to 
existing implementation for both Infiniband and iWARP.  Changes were made in 
sections 2, 3.1.6, 4.2, 5.1, 5.1.1, 5.1.2, 5.1.3, 10.1.3.4, and 11.

2. Added a clause in section 6.2 to clarify that the initiator must not send 
more than InitiatorMaxRecvDataSegmentLength worth of data when a NOP-Out 
request is sent with a valid Initiator Task Tag.  Since 
InitiatorMaxRecvDataSegmentLength can be smaller than 
TargetMaxRecvDataSegmentLength, returning the original data in the NOP-Out 
request in this situation can overflow the receive buffer unless the length 
of the data sent with the NOP-Out request is less than 
InitiatorMaxRecvDataSegmentLength.

3. Added MaxAHSLength key in section 6.8 to set a limit on the AHS Length as 
proposed by Alex Nezhinsky.  This is useful when posting receive buffers in 
knowing what the maximum possible message length is in a PDU which contains 
AHS.

4. Added WriteAddressForSolicitedDataOnly key in section 6.9 to indicate how 
the memory region will be used as proposed by Alex Nezhinsky.  Alex argued 
that WSTAG (RKEY+VA for Infiniband) sent by the initiator with a WRITE SCSI 
command already accounts for the offset associated with the solicited data, 
and does not reference the entire data.  The rationale is that the initiator 
sometimes treats the memory regions intended for unsolicited and solicited 
data differently, as they are carried out using different Infiniband 
mechanisms (Send from the initiator, RDMA from the target) that imply 
different registration modes.  In contrast, the model implied by the iSER 
spec creators was that the memory occupied by data is treated as contiguous 
(or virtually contiguous, by means of scatter-gather mechanisms) and 
homogenous region.  Adding a new key will allow coping with current 
implementations.

 5. Added two 64-bit fields in iSER header in section 9.2 for the Read 
Virtual Address and the Write Virtual Address to conform to the Infiniband 
format as proposed by Bob Russell.  This allows one implementation such as 
the OFED stack to be used in both the Infiniband and the iWARP environment.

Proposed Changes To Be Considered

6. Alex Nezhinsky stated that iSER Hello and iSER HelloReply messages are 
not implemented in practice.  He argued that we should update the spec to 
remove the requirement to exchange the iSER Hello and iSER HelloReply 
messages.

Changes To Be Done

Update all diagrams to conform with the expanded iSER header format.

Mike

---

From: "Mallikarjun Chadalapaka" <[email-masked]>
Date: Fri, 4 Dec 2009 16:19:36 -0800

Hi Mike,

I hope you had a chance to see my note on this topic a few weeks ago.

> the OFED stack to be used in both the Infiniband and the iWARP
environment.

Are you proposing this protocol change to support just the OFED stack?

ZB-VA is supported with iWARP as a base feature, and I thought IB was going
to embrace it as an extension as of three years ago.  If that is the case,
IMO, we should reconsider making iSER protocol changes to support non-ZBVA
for benefiting one implementation - particularly when the change is neither
efficient nor free.  

Thanks.

Mallikarjun

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Mike
> Ko
> Sent: Friday, December 04, 2009 3:41 PM
> To: [email-masked]
> Subject: [storm] Updated iSER draft
>

---

From: Mike Ko <[email-masked]>
Date: Fri, 04 Dec 2009 16:41:49 -0800

Mallikarjun,

The question is whether our intention to update the iSER spec is to reflect 
"running code".  This hinges on whether there are any other commonly adopted 
software support for iSER other than the OFED stack.  Felix Marti responded 
in November that the Chelsio iWARP products support ZBVA, but no detail was 
given on how this is done.  Do you know of any "running code" in widespread 
use that supports ZBVA?

Unless there are some recent changes, AFAIK, there is no intention of 
embracing ZBVA in IB.  ZBVA is "tolerated" because it is in the iSER spec.

Mike

---

From: "Felix Marti" <[email-masked]>
Date: Fri, 4 Dec 2009 17:17:41 -0800

Hi Mike,

That is correct, the iWarp spec defines both VA based Target Offset and
Zero-Base Target Offset (aka ZBVA). I'm not sure if OFED currently makes
use of only one of them or both, but if there is a need for both, I
don't see why this couldn't be enabled.

felix

> Of Mike Ko
> Sent: Friday, December 04, 2009 4:42 PM
> To: Mallikarjun Chadalapaka; [email-masked]
> Subject: Re: [storm] Updated iSER draft
> 
> Mallikarjun,
> 
> in November that the Chelsio iWARP products support ZBVA, but no
detail
> was
> >> format as proposed by Bob Russell.  This allows one implementation
> such
> >> as
> > ZB-VA is supported with iWARP as a base feature, and I thought IB
was
> > going
> >> mode and transitioned to zero-copy mode.  The allows the spec to
> conform
> > to
> >> existing implementation for both Infiniband and iWARP.  Changes
were
> made
> > in
> >> 2. Added a clause in section 6.2 to clarify that the initiator must
> not
> > send
> >> request in this situation can overflow the receive buffer unless
the
> > length
> >> 3. Added MaxAHSLength key in section 6.8 to set a limit on the AHS
> Length
> > as
> >> proposed by Alex Nezhinsky.  This is useful when posting receive
> buffers
> > in
> >> 4. Added WriteAddressForSolicitedDataOnly key in section 6.9 to
> indicate
> > how
> >> that WSTAG (RKEY+VA for Infiniband) sent by the initiator with a
> WRITE
> > SCSI
> >> command already accounts for the offset associated with the
> solicited
> > data,
> >> different registration modes.  In contrast, the model implied by
the
> iSER
> >> format as proposed by Bob Russell.  This allows one implementation
> such
> >> as

---

From: "Mallikarjun Chadalapaka" <[email-masked]>
Date: Fri, 4 Dec 2009 17:29:21 -0800

Hi Mike,

> "running code".  

It should be, I was only trying to make sure other "running code"
implementations of iSER aren't impacted due to this change, particularly
when this change is not typical SCSI.  I am going to have to defer to those
working hands-on on "running code".

Thanks.

Mallikarjun

> From: Mike Ko [mailto:[email-masked]]
Of
conform
> > to
> >> existing implementation for both Infiniband and iWARP.  Changes were
made
Length
buffers
indicate