Subject: [storm] Draft minutes from Anaheim

From: <[email-masked]>
Date: Sat, 27 Mar 2010 02:59:13 -0400

Draft minutes are attached - please comment, correct, etc.

Also, in the absence of objection on this mailing list, decisions recorded in the minutes are considered to be the rough consensus of this WG, *except* that two issues were identified as sufficiently important to discuss separately on the list (see separate messages):
	- Text negotiation key for new iSCSI features (discussion in progress)
	- Features to remove from iSCSI (discussion to be started)

Many thanks to Craig Carlson for taking notes during the meeting.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: "Mark S. Edwards" <[email-masked]>
Date: Sat, 27 Mar 2010 13:55:30 +0000

Regarding the feature removal discussion, can I add SLP to the list ?

The RFC 3721 states

"iSCSI equipment that
       need discovery functions beyond SendTargets should at least
       implement SLP, and then consider iSNS when extended discovery
       management capabilities are required such as in larger storage
       networks.  It should be noted that since iSNS will support SLP,
       iSNS can be used to help manage the discovery information returned
       by SLP."

The implication is that targets and initiators should expect to find 
support for SLP before considering iSNS.

I remember our first iSCSI appliance and we spent ages trying to get 
SLP working because it the above wording effectively made it 
mandatory.  SLP turned out to be a complete bust and was effectively 
killed off when Microsoft refused to support it in their initiator 
and in their target logo tests.

The result is that today I doubt you could find a target or initiator 
out there supporting SLP.

For anybody that does still implement SLP we could change the wording 
for SLP a little to remove the implied hierarchy, or just admit that 
running code has created IETF consensus.

On another removal topic, I seem to recall that Mallikarjun also said 
that he was removing markers.  I don't particularly object to this 
but I do wonder if this will affect any HBA implementations ?

Regards,

Mark.

At 06:59 27/03/2010, [email-masked] wrote:
>Also, in the absence of objection on this mailing list, decisions 
>recorded in the minutes are considered to be the rough consensus of 
>this WG, *except* that two issues were identified as sufficiently 
>important to discuss separately on the list (see separate messages):
>_______________________________________________
>storm mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/storm

---

From: "Knight, Frederick" <[email-masked]>
Date: Sat, 27 Mar 2010 16:59:57 -0400

There are reports of working being done with (code being written for)
MARKERS, so it is premature to remove them.

	Fred Knight

-----Original Message-----
From: Mark S. Edwards [mailto:[email-masked]] 
Sent: Saturday, March 27, 2010 9:56 AM
To: [email-masked]
Subject: Re: [storm] Draft minutes from Anaheim

The RFC 3721 states

"iSCSI equipment that
       iSNS can be used to help manage the discovery information
returned
       by SLP."

Regards,

Mark.

---

From: "Mallikarjun Chadalapaka" <[email-masked]>
Date: Mon, 29 Mar 2010 17:32:36 -0700

Just to clarify...

> that he was removing markers.  

I had only said that it's one of the items I had heard prior requests on
(that it be removed).  Thanks for initiating the list discussion though! 

> but I do wonder if this will affect any HBA implementations ?

Good question, I don't know.  HBA vendors, especially iSCSI/iSER/RNIC
"roto-tilled" implementations, please chime in.

Mallikarjun

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Mark
> S. Edwards
> Sent: Saturday, March 27, 2010 6:56 AM
> To: [email-masked]

---

From: "Paul Koning" <[email-masked]>
Date: Mon, 29 Mar 2010 21:01:49 -0400

I sure would like markers to go away.  Rumors of their use are somewhat
interesting, but substantiated data would be more so.

	paul

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Mallikarjun Chadalapaka
> Sent: Monday, March 29, 2010 8:33 PM
> To: 'Mark S. Edwards'; [email-masked]
> Subject: [storm] Removing iSCSI Markers?
> 
> Just to clarify...
>

---

From: Asgeir Eiriksson <[email-masked]>
Date: Tue, 30 Mar 2010 06:23:52 -0700

Hello Paul,

The Chelsio RNIC do support the marker feature, but as far as I know the feature

has never been used in the field, and it isn't supported by all RNIC

implementations. 

I periodically ask our AE and developers about this feature and so far the answer

is that no one uses it, and no one is asking for it (4 years of data at this point).

Regards,

Asgeir Eiriksson

CTO

Chelsio Communications Inc.
 
> Date: Mon, 29 Mar 2010 21:01:49 -0400
> From: [email-masked]
> To: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [storm] Removing iSCSI Markers?
> 
 		 	   		  
_________________________________________________________________
Hotmail is redefining busy with tools for the New Busy. Get more from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID27925::T:WLMTAGL:ON:WL:en-US:WM_HMP:032010_2

---

From: "Paul Koning" <[email-masked]>
Date: Tue, 30 Mar 2010 10:22:35 -0400

Thanks Asgeir.

As I recall, the original idea behind markers is to make it possible to
build 10G HBAs that can run at wire speed, which was believed to be
impossible otherwise.

The subsequent record indicates that this was in fact not the case; 10G
HBAs are feasible and have been built without resorting to markers.
There is no other reason for using markers.  So if the one reason that
they were thought to be needed in fact turned out not to be real, the
obvious thing to do is to remove the unused complications from the spec.

I suppose one could argue that, placed in an appendix and "optional to
implement" they do no harm.  That's a fair point.  If there is still a
chance that they will turn out to be needed in the future we may want to
go that way.  I personally would bet against that chance.

                paul 

From: Asgeir Eiriksson [mailto:[email-masked]] 
Sent: Tuesday, March 30, 2010 9:24 AM
To: Paul Koning; [email-masked]; [email-masked];
[email-masked]
Subject: RE: [storm] Removing iSCSI Markers?

Hello Paul,
 
The Chelsio RNIC do support the marker feature, but as far as I know the
feature
has never been used in the field, and it isn't supported by all RNIC
implementations. 
 
I periodically ask our AE and developers about this feature and so far
the answer
is that no one uses it, and no one is asking for it (4 years of data at
this point).
 
Regards,
 
Asgeir Eiriksson
CTO
 
> > I had only said that it's one of the items I had heard prior
requests
> > on
> > > Regarding the feature removal discussion, can I add SLP to the
list
> ?

________________________________

Hotmail is redefining busy with tools for the New Busy. Get more from
your inbox. Sign up now.
<http://www.windowslive.com/campaign/thenewbusy?ocid=PID27925::T:WLMTAGL
:ON:WL:en-US:WM_HMP:032010_2>

---

From: "Mark S. Edwards" <[email-masked]>
Date: Tue, 30 Mar 2010 15:49:11 +0100

Paul,

That's pretty much my recollection, too.

One of those things that was thought to be a reasonable solution to a 
foreseeable change in the technology.  In the end, the technology 
found a different solution.

They were fascinating presentations, though.

Mark.

At 15:22 30/03/2010, Paul Koning wrote:
>Thanks Asgeir.
>
>As I recall, the original idea behind markers is to make it possible 
>to build 10G HBAs that can run at wire speed, which was believed to 
>be impossible otherwise.
>
>The subsequent record indicates that this was in fact not the case; 
>10G HBAs are feasible and have been built without resorting to 
>markers.  There is no other reason for using markers.  So if the one 
>reason that they were thought to be needed in fact turned out not to 
>be real, the obvious thing to do is to remove the unused 
>complications from the spec.
>
>I suppose one could argue that, placed in an appendix and "optional 
>to implement" they do no harm.  That's a fair point.  If there is 
>still a chance that they will turn out to be needed in the future we 
>may want to go that way.  I personally would bet against that chance.
>
>                 paul
>
>To: Paul Koning; [email-masked]; [email-masked]; [email-masked]
>----------
><http://www.windowslive.com/campaign/thenewbusy?ocid=PID27925::T:WLMTAGL:ON:WL:en-US:WM_HMP:032010_2>Sign 
>up now.

---

From: Stephen Bailey <[email-masked]>
Date: Tue, 30 Mar 2010 08:04:55 -0700

Despite the fact that I worked on the draft, I always thought markers
were silly and would be proven so over time.  Good to hear I was
actually right for once!

Steph

On Tue, Mar 30, 2010 at 7:49 AM, Mark S. Edwards <[email-masked]> wrote:
> foreseeable change in the technology.  In the end, the technology found a
> different solution.
>
> The subsequent record indicates that this was in fact not the case; 10G HBAs
> are feasible and have been built without resorting to markers.  There is no
> other reason for using markers.  So if the one reason that they were thought
> to be needed in fact turned out not to be real, the obvious thing to do is
> to remove the unused complications from the spec.
>
> I suppose one could argue that, placed in an appendix and “optional to
> implement” they do no harm.  That’s a fair point.  If there is still a
> chance that they will turn out to be needed in the future we may want to go
> that way.  I personally would bet against that chance.
>
>                 paul
>
> From: Asgeir Eiriksson [ mailto:[email-masked]]
>> > From: [email-masked] [ mailto:[email-masked]] On Behalf
>> > > From: [email-masked] [ mailto:[email-masked]] On
>> > Behalf Of
>> > Mark
>> > > S. Edwards
> Hotmail is redefining busy with tools for the New Busy. Get more from your
> inbox. Sign up now.
>

---

From: Asgeir Eiriksson <[email-masked]>
Date: Tue, 30 Mar 2010 08:28:05 -0700

Paul, Mark

The main selling point for markers is that they enable out or order placement

(on receive) while preserving in order completions and markers therefore have the

potential of decreasing buffering requirements in RNIC and to a less extent in

iSCSI HBA.

'Asgeir 

Date: Tue, 30 Mar 2010 15:49:11 +0100
To: [email-masked]
From: [email-masked]
Subject: Re: [storm] Removing iSCSI Markers?

Paul,

One of those things that was thought to be a reasonable solution to a foreseeable change in the technology.  In the end, the technology found a different solution.

Mark.

Thanks Asgeir.
 
As I recall, the original idea behind markers is to make it possible to build 10G HBAs that can run at wire speed, which was believed to be impossible otherwise.
 
The subsequent record indicates that this was in fact not the case; 10G HBAs are feasible and have been built without resorting to markers.  There is no other reason for using markers.  So if the one reason that they were thought to be needed in fact turned out not to be real, the obvious thing to do is to remove the unused complications from the spec.
 
I suppose one could argue that, placed in an appendix and “optional to implement” they do no harm.  That’s a fair point.  If there is still a chance that they will turn out to be needed in the future we may want to go that way.  I personally would bet against that chance.
 
                paul 

Hello Paul,

 
Regards,
 
Asgeir Eiriksson
CTO
 
> > > >Many thanks to Craig Carlson for taking notes during the meeting.

Hotmail is redefining busy with tools for the New Busy. Get more from your inbox. Sign up now. 		 	   		  
_________________________________________________________________
Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID27925::T:WLMTAGL:ON:WL:en-US:WM_HMP:032010_1

---

From: "Mark S. Edwards" <[email-masked]>
Date: Tue, 30 Mar 2010 17:22:17 +0100

Asgeir,

Yes, I do understand and remember all the arguments.  Indeed, I 
remember one of Randy's theoretical presentations positing that a 
marker aware 10GB offload NIC might only require as little as 2K 
onboard buffering RAM.

The point is, that at least for iSCSI the technology that arrived 
seems to marginalised the need for anyone to implement 
markers.  Given the fact that running code creates IETF consensus we 
have an opportunity at this time to remove unnecessary complications, 
markers are a candidate for being made optional or even being removed 
completely.

Personally I would be happy to see them removed.  My original note on 
this topic was to try to be a good citizen by asking anyone who might 
be affected, or know someone who would be affected, to speak up.

Mark.

At 16:28 30/03/2010, Asgeir Eiriksson wrote:
>One of those things that was thought to be a reasonable solution to 
>a foreseeable change in the technology.  In the end, the technology 
> <https://www.ietf.org/mailman/listinfo/storm>https://www.ietf.org/mailman/listinfo/storm
> > >
> <https://www.ietf.org/mailman/listinfo/storm>https://www.ietf.org/mailman/listinfo/storm
> <https://www.ietf.org/mailman/listinfo/storm>https://www.ietf.org/mailman/listinfo/storm
>
>----------
><http://www.windowslive.com/campaign/thenewbusy?ocid=PID27925::T:WLMTAGL:ON:WL:en-US:WM_HMP:032010_1>Learn 
>More.

---

From: "Mallikarjun Chadalapaka" <[email-masked]>
Date: Tue, 30 Mar 2010 09:42:07 -0700

I believe out of order placement continues to be critical for efficient
RNIC/DDP implementations, and markers play a role there.

Having said that, IMHO, that is not exactly the question we should tackle on
this thread.

We should focus on these two iSCSI-centric questions: 

1) Are there implementations out there that implement iSCSI Markers *as
defined by RFC 3720*? (Asgeir may have answered this question as "yes", but
he referenced an RNIC so I am not sure if he's referring to the MPA-version
of markers or iSCSI key-driven markers)

2) If "yes" to #1, if we drop iSCSI Markers in the new Consolidated draft,
would that cause problems to any "applications" - i.e. iSCSI and SCSI stacks
in either commercial O/S or proprietary embedded implementations?

If the answer to the second question is "No", we can go ahead and drop it
from the iSCSI Consolidated draft, independent of MPA/DDP/RDMAP.

Thanks.

Mallikarjun

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Mark S. Edwards
Sent: Tuesday, March 30, 2010 9:22 AM
To: [email-masked]
Subject: Re: [storm] Removing iSCSI Markers?

Asgeir,

Yes, I do understand and remember all the arguments.  Indeed, I remember one
of Randy's theoretical presentations positing that a marker aware 10GB
offload NIC might only require as little as 2K onboard buffering RAM.

The point is, that at least for iSCSI the technology that arrived seems to
marginalised the need for anyone to implement markers.  Given the fact that
running code creates IETF consensus we have an opportunity at this time to
remove unnecessary complications, markers are a candidate for being made
optional or even being removed completely.

Personally I would be happy to see them removed.  My original note on this
topic was to try to be a good citizen by asking anyone who might be
affected, or know someone who would be affected, to speak up.

Mark.

Paul, Mark
 
The main selling point for markers is that they enable out or order
placement
(on receive) while preserving in order completions and markers therefore
have the
potential of decreasing buffering requirements in RNIC and to a less extent
in
iSCSI HBA.
 
'Asgeir 
 
________________________________________
Date: Tue, 30 Mar 2010 15:49:11 +0100
To: [email-masked]
From: [email-masked]
Subject: Re: [storm] Removing iSCSI Markers?

Paul,

Mark.

 

                paul 

Hello Paul,

I periodically ask our AE and developers about this feature and so far the
answer
is that no one uses it, and no one is asking for it (4 years of data at this
point).
 
Regards,
 
Asgeir Eiriksson
CTO
 
> > > >Many thanks to Craig Carlson for taking notes during the meeting.
________________________________________
Hotmail is redefining busy with tools for the New Busy. Get more from your
inbox. Sign up now.

________________________________________
Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
Learn More.

---

From: William Stouder-Studenmund <[email-masked]>
Date: Tue, 30 Mar 2010 10:03:05 -0700

> As I recall, the original idea behind markers is to make it possible to build 10G HBAs that can run at wire speed, which was believed to be impossible otherwise.

I obviously came late to iSCSI when I started. That's _not_ what I thought markers were for. I thought they were to support within-connection error recover. I saw no other way to recover from a header digest error otherwise. Well, within the connection.

Back when it existed, the Wasabi target handled marker generation. I don't think we ever got around to adding the error recovery that would use it, but it was planned.

Take care,

Bill

---

From: Julian Satran <[email-masked]>
Date: Tue, 30 Mar 2010 20:08:09 +0300

Dear All,

Just to get the "recollections" straight:

Markers where put in place because (some of us) could not handle 
"fighting" for an general purpose RDMA solution and getting iSCSI done 
at the same time and preferred to handle what we though more urgent 
(iSCSI). IETF was not inclined at the time to seriously consider an RDMA 
solution - except for using it over SCTP (a brand new transport at the 
time) - and many of us felt that the very conservative storage industry 
will not accept SCTP (too new!).

So we stopped (for a while) trying to get the things right and have IETF 
look for a generalized RDMA sollution (generic for TCP) and put 
something ugly but usable in place. The RDMA stuff was done after iSCSI 
was closed (and BTW has its own marker type solution when run over TCP - 
a bit costlier than iSCSI but workable). iSER is the result of having 
RDMA over TCP (yes with those ugly markers in place!).

With RDMA over TCP in place I am not sure we want to keep the iSCSI 
markers unless their removal breaks somebodys implementation. We will 
have to live with markers on RDMA over TCP - though.

And the problem never was processing at wire speed - processing is cheap 
- but rather conserving memory bandwidth.

Julo

---

From: Asgeir Eiriksson <[email-masked]>
Date: Tue, 30 Mar 2010 10:11:41 -0700

Mallikarjun,

I was referring to the MPA-version of markers only, and the Chelsio iSCSI HBA do

not support iSCSI markers.

Regards,

'Asgeir
 
> From: [email-masked]
> To: [email-masked]; [email-masked]
> Date: Tue, 30 Mar 2010 09:42:07 -0700
 		 	   		  
_________________________________________________________________
Hotmail: Trusted email with powerful SPAM protection.
http://clk.atdmt.com/GBL/go/210850553/direct/01/

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 31 Mar 2010 19:44:29 -0700

> > from the iSCSI Consolidated draft, independent of MPA/DDP/RDMAP.

If there are deployed iSCSI HBA/CNICs that fully support markers then by all means we should
commend them for doing so and not leave them stranded.

But if there are none then keeping iSCSI markers really makes no sense. iSER provides a better
solution for out-of-order data placement. There's no point in backing two "future" options, it's
better to consolidate on one.

If your network needs to support out-of-order data placement for SCSI, you should be looking
for a NIC that supports iSER/iWARP.

--
Caitlin Bestler
[email-masked]
http://www.asomi.com/CaitlinBestlerResume.html