Subject: [v6ops] I-D Action:draft-ietf-v6ops-cpe-simple-security-15.txt

From: [email-masked]
Date: Tue, 12 Oct 2010 11:00:01 -0700 (PDT)

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IPv6 Operations Working Group of the IETF.

	Title           : Recommended Simple Security Capabilities in Customer Premises Equipment for Providing Residential IPv6 Internet Service
	Author(s)       : J. Woodyatt
	Filename        : draft-ietf-v6ops-cpe-simple-security-15.txt
	Pages           : 35
	Date            : 2010-10-12

This document identifies a set of recommendations for the makers of
devices describing how to provide for "simple security" capabilities
at the perimeter of local-area IPv6 networks in Internet-enabled
homes and small offices.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-v6ops-cpe-simple-security-15.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Fred Baker <[email-masked]>
Date: Tue, 12 Oct 2010 11:36:09 -0700

This is a quick consensus check. The CPE Simple Security document is about to complete IESG review, and has gone through four revisions in the process of IETF Last Call and IESG review. I'd like to the working group to quickly ratify (or not) the changes that have been made and the current text.

    ftp://ftpeng.cisco.com/fred/v6ops/draft-ietf-v6ops-cpe-simple-security-11-15.html
    http://www.ietf.org/internet-drafts/draft-ietf-v6ops-cpe-simple-security-15.txt
    http://datatracker.ietf.org/doc/draft-ietf-v6ops-cpe-simple-security

If you have any comments, please make them by the weekend. Silence will be presumed to be consent.

Begin forwarded message:

> From: [email-masked]
> Date: October 12, 2010 11:00:01 AM PDT
> To: [email-masked]
> Cc: [email-masked]
> Subject: [v6ops] I-D Action:draft-ietf-v6ops-cpe-simple-security-15.txt
> 
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: Mark Smith <[email-masked]>
Date: Thu, 14 Oct 2010 20:51:09 +1030

Hi Fred,

I'm really quite concerned about this new text,

REC-13: By DEFAULT, Internet gateways SHOULD, automatically download
   and install software updates for extending IPv6 simple security for
   support of future standard upper layer transports and extension
   headers.

in particular the automatic install, as I don't think the
consequences of automatically updating the firmware "behind the
customers back" have been fully considered. 

If there are any failures with this mechanism at all, the customer
will most likely be calling the ISP, not the product vendor, despite the
vendor being in control of and responsible for when this new firmware is
deployed. It will appear to the customer that the ISP's service has
failed. As an update is likely to be pushed out to many CPE at once,
that could result in floods of 100s, 1000s etc. of unexpected calls to
the ISP's helpdesk moments after the update occurs. It could also mean
that updates occur in the middle of when the ISP's services are most
used due to differing time zones between when the vendor pushes new
firmware out verses and when customers are most commonly using the
Internet services.  In this scenario the ISP should be in control of
when these updates are made available.

Following the model used by vendors of desktop OSes these days, it
would be better if when updates were pushed out, the customer was
alerted to an update being available, and the customer had to somehow
manually trigger the update, to avoid disrupting services. 

I think it'd probably be better to make only a passing comment about
keeping the firmware up to date, avoiding it being a specific SHOULD
recommendation, without details, in this document. The Broadband Forum,
with mechanisms such as TR-69, seem to be more thoroughly considering
these issues.

Regards,
Mark.

On Tue, 12 Oct 2010 11:36:09 -0700

---

From: Fred Baker <[email-masked]>
Date: Thu, 14 Oct 2010 08:34:53 -0700

If the comment is not in this draft, then the work being done in the Broadband Forum on this topic should be replicated at Cable Labs for that class of networks. Question to their respective representatives: are you sufficiently on the same page that you can suggest a statement to go here, or would you prefer separate approaches from the Fora?

Mark: How would you suggest wording the "passing comment" regarding updates? It sounds like you would prefer something along the lines of:

REC-13:
Residential Internet Gateways SHOULD provide a means for their owners to securely update their firmware, for the installation of security patches and other manufacturer-recommended changes. Owners are responsible to install the firmware, and encouraged to do so promptly, but the means of informing them of the availability of the update is beyond the scope of this document.

</chair>
I can see both sides of this. I configure my laptop (OS and applications) to automatically download patches and notify me of their availability, and I decide to install them. I like that on two counts: *I* maintain my patches, and I *maintain* my patches. The model makes sense with other software and firmware relevant to me as a consumer. On the other hand, patches get applied to my wife's and my daughter's laptops when I notice that they need to be applied (in my daughter's case, that means that I'm visiting her) and make it happen. If I were the manufacturer of the residential gateway in question, the only way I see to maintain the software would be to maintain an email list as people register their products and send them an email, as people can't be depended on to log into a manufacturer or ISP site periodically to get the news, and the residential gateway doesn't generally have a screen that the owner periodically looks at to announce the need for an update. We all know how emails tend to get lost in the noise...
<chair>

All: opinions?

---

From: james woodyatt <[email-masked]>
Date: Thu, 14 Oct 2010 12:46:04 -0700

> customers back" have been fully considered. 

This new text is the result of IESG ballot discussion on the -14 revision, submitted by Jari Arkko on 2010-10-07.  Here is the text I just pulled out of the I-D datatracker.

>> Also, the only mention of new protocols and unrecognized protocols and what to
>> do with them is here:
>> 
>>  Conversely, simple Internet gateways are not expected to prohibit the
>>  development of new applications.  In particular, packets with end-to-
>>  end network security and routing extension headers for mobility are
>>  expected to pass Internet gateways freely.
>> 
>> In other words, if you package your traffic in IPsec (or Mobile IPv6) it will
>> work, otherwise not. I think the document should be more explicit about this
>> limitation, and that new extension headers, new transport protocols, etc. would
>> not work through the simple gateway. I would suggest that the document say
>> something about recommending automatic software updates for any device that
>> supports this type of a functionality so that newly allocated protocol numbers
>> could become supported in an easy manner.

After reviewing the current revision, Mr. Arkko cleared his objection and sent another comment about this topic to the v6ops chairs, the IESG and me.

>> The justification before REQ-13 seems a bit out of balance. It is only arguing about new attacks, not the development of new protocols. In my mind that is an equally good reason to have software updates. Then the requirement itself only argues about new protocols, and does not refer to the new attacks at all. I would edit this slightly differently, just have the requirement stand on its own, and the preceding paragraph about the motivation should include both aspects.

It seems like Mr. Smith and Mr. Arkko are in disagreement about whether software updates should be downloaded and applied automatically, and I don't know how to proceed from here.

I'm not sure Mr. Arkko's original comment about "new extension header, new transport protocols, etc. would not work through [IPv6 simple security]" is strictly correct.  We can expect that new transport protocols will be tracked with generic state records comprising source address and next header value.  Assuming I-D.ietf-6man-exthdr proceeds to standard, we can also expect the same for all new standard extension headers.

I have no opinion one way or another.  My principle interest is to be finished with editing this draft sooner rather than later.

--
james woodyatt <[email-masked]>
member of technical staff, communications engineering

---

From: Fred Baker <[email-masked]>
Date: Thu, 14 Oct 2010 13:09:37 -0700

> It seems like Mr. Smith and Mr. Arkko are in disagreement about whether software updates should be downloaded and applied automatically, and I don't know how to proceed from here.

Jari, being on the IESG, is copied in this. I would suggest that he, Mark, and Barbara argue their relative cases. At the end of the day, the reason that I asked for a final discussion is that there have been several changes, and the document has to satisfy not only it's last reviewer, but the operational community that is recommending it.

the status as I see it is:

 - Jari wants the patches applied automatically, which assures patch updates but removes a level of control from the owner/operator of the device

 - Mark wants the patches pushed out but applied with the owner/operator's consent

 - Barbara wants the owner/operator to have an interface that can apply a patch, but leave anything else undefined; this basically preserves the status quo, which means that residential gateways are likely to generally *not* track patches and as a result have security vulnerabilities, which is also the status quo.

A middle ground, perhaps, would be to recommend that the manufacturer provide an option that the user can select; the present language's "by default" suggests this as well. The options would be for the device to periodically access a manufacturer web site and download updated firmware or for the manufacturer to somehow (blast email?) advise the registered owner to do so at their convenience. The issue for this draft would be "what is the default", as that will be the most common actual behavior.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 15 Oct 2010 09:22:43 +1300

>  - Jari wants the patches applied automatically, which assures patch updates but removes a level of control from the owner/operator of the device

Speaking as a user, I deeply hate systems that install patches by default
without asking me. I don't think the IETF should be taking a position on
that one way or the other. On the other, leaving patching up to the user's
initiative has been thoroughly proved to fail. So I think Mark's
position is best, but I'd still prefer us to be neutral: recommend that
the implementor/vendor provides a patch mechanism, but not saying anything
about automatic vs consent vs voluntary.

    Brian

---

From: Fred Baker <[email-masked]>
Date: Thu, 14 Oct 2010 13:29:28 -0700

> about automatic vs consent vs voluntary.

So you in essence support Barbara's proposed language.

---

From: james woodyatt <[email-masked]>
Date: Thu, 14 Oct 2010 13:30:50 -0700

On Oct 14, 2010, at 13:09 , Fred Baker wrote:
> 
> The issue for this draft would be "what is the default", as that will be the most common actual behavior.

As long as software updates require user interaction to apply, we will continue to have the status quo ante, i.e. most gateways will never be updated from their factory software load.  I don't see much practical benefit in making recommendations about staging of firmware updates prior to sending an attention signal to a human.

I suggest tweaking the recommendation to say that an "easily configured administrative option" to download and apply updates automatically SHOULD be offered.

---

From: Mark Smith <[email-masked]>
Date: Fri, 15 Oct 2010 07:25:49 +1030

Hi Fred,

On Thu, 14 Oct 2010 08:34:53 -0700

> Residential Internet Gateways SHOULD provide a means for their owners to securely update their firmware, for the installation of security patches and other manufacturer-recommended changes. Owners…

Yes, I'd be happier with that. 

> </chair>
> I can see both sides of this. I configure my laptop (OS and applications) to automatically download patches and notify me of their availability, and I decide to install them. I like that on two counts: *I* maintain my patches, and I *maintain* my patches. The model makes sense with other software and firmware relevant to me as a consumer. On the other hand, patches get applied to my wife's and my daughter's laptops when I notice that they need to be applied (in my daughter's case, that means that I'm visiting her) and make it happen. If I were the manufacturer of the residential gateway in question, the only way I see to maintain the software would be to maintain an email list as people register their products and send them an email, as people can't be depended on to log into a manufacturer or ISP site periodically to get the news, and the residential gateway doesn't generally have a screen that the owner periodically looks at to announce the need for an update. We all know
  how emails tend to get lost in the noise...
> <chair>

My issue isn't so much the automated downloading, it's the automated
installation. CPE typically needs to be rebooted to perform that task,
so automated installation means an unexpected service outage. So in the
middle of making a VoIP phone call to the boss (or to emergency
services!), accessing the corporate VPN, watching a streaming video
etc., there'd all of a sudden be an unexplained outage of typically a
few minutes, as well as typically would would be alarming flashing of
lights on the device. Very un-user friendly behaviour, and worse
because the ISP is likely to be the first to be blamed for it, and have
wear the consequences of dealing with unhappy customers. Unfortunately
I've recently had to deal with a situation where vendor faults on large
numbers of CPE devices were seen to be my (ISP) employer's
responsibility and it was very unpleasant to say the least.

As for mechanism for applying an update (which I do agree is out of
scope), one thought I had would be possibly an "Firmware Upgrade
Available" flashing LED on the CPE, and a specific button to be held
down for say 5 seconds to load it would be a fairly simple way to both
notify the end-user and allow them to control when it is applied.

Thanks,
Mark.

---

From: "STARK, BARBARA H (ATTLABS)" <[email-masked]>
Date: Thu, 14 Oct 2010 17:02:48 -0400

BBF has defined a model for doing firmware upgrades via TR-069. But
support for this mechanism is not mandated, and not all SPs choose to
use it (though some do). Other methods I've seen are (1) the device has
an "update" web page that can be manually invoked by the user, (2)
during initial device configuration the user is prompted to set up a
periodic time for the device to go out and look for updates (e.g., the
4th of every month at 2:41am local time), (3) similar to (1) but in
addition to that the device periodically checks for updates and makes it
known on the main device web page that an update is available and the
user needs to manually invoke it, etc. These methods are also usable by
retail CE routers, and none of them have the effect of causing too many
routers to upgrade at the same time.

I wouldn't recommend that CE routers use a mechanism that causes en
masse upgrades. But since I've never seen a CE router manufacturer do
something like that (and most don't have the capability, since CE
routers are almost never manageable through a "push" management model),
I don't really think we need language in the simple-security draft
recommending against that.

I think that if you take the first sentence of the text Fred suggested,
that should be sufficient:

REC-13:
Residential Internet Gateways SHOULD provide a means for their owners to
securely update their firmware, for the installation of security patches
and other manufacturer-recommended changes.

Barbara

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Fred Baker
>
> If the comment is not in this draft, then the work being done in the
> Broadband Forum on this topic should be replicated at Cable Labs for
> that class of networks. Question to their respective representatives:
> are you sufficiently on the same page that you can suggest a statement
> to go here, or would you prefer separate approaches from the Fora?
> 
> Mark: How would you suggest wording the "passing comment" regarding
> updates? It sounds like you would prefer something along the lines of:
> 
> REC-13:
> Residential Internet Gateways SHOULD provide a means for their owners
> to securely update their firmware, for the installation of security
> patches and other manufacturer-recommended changes. Owners are
> responsible to install the firmware, and encouraged to do so promptly,
> but the means of informing them of the availability of the update is
> beyond the scope of this document.
> 
> </chair>
> I can see both sides of this. I configure my laptop (OS and
> applications) to automatically download patches and notify me of their
> availability, and I decide to install them. I like that on two counts:
> *I* maintain my patches, and I *maintain* my patches. The model makes
> sense with other software and firmware relevant to me as a consumer.
On
> the other hand, patches get applied to my wife's and my daughter's
> laptops when I notice that they need to be applied (in my daughter's
> case, that means that I'm visiting her) and make it happen. If I were
> the manufacturer of the residential gateway in question, the only way
I
> see to maintain the software would be to maintain an email list as
> people register their products and send them an email, as people can't
> be depended on to log into a manufacturer or ISP site periodically to
> get the news, and the residential gateway doesn't generally have a
> screen that the owner periodically looks at to announce the need for
an
> update. We all know h
>  ow emails tend to get lost in the noise...
> <chair>
> 
> All: opinions?
> 
> > vendor being in control of and responsible for when this new
firmware
> is
> >> This is a quick consensus check. The CPE Simple Security document
is
> about to complete IESG review, and has gone through four revisions in
> the process of IETF Last Call and IESG review. I'd like to the working
> group to quickly ratify (or not) the changes that have been made and
> the current text.
> >>
> >>> 	Title           : Recommended Simple Security Capabilities in
> Customer Premises Equipment for Providing Residential IPv6 Internet
> Service
> >>> 	Author(s)       : J. Woodyatt

---

From: james woodyatt <[email-masked]>
Date: Thu, 14 Oct 2010 14:03:42 -0700

On Oct 14, 2010, at 13:55 , Mark Smith wrote:
> 
> ...so automated installation means an unexpected service outage...

Because that, as we all know, never happens.  (I'm about *this* close to recommending that IANA set aside 192.168.100.1 as a reserved IPv4 address for use only by DOCSIS management interfaces.)

Still, your point seems compelling to me.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 15 Oct 2010 10:22:47 +1300

I think we should keep our noses out of default patching
policies. These may vary a lot according to the context.
I would certainly expect DOCSIS boxes to be upgraded without
involving the user, for example, but the same won't be true
for the proverbial $50 box from the local electronics supermarket.
One size of policy won't fit all. So don't say who does the patching:

REC-13:
Residential Internet Gateways SHOULD provide a convenient means to
securely update their firmware, for the installation of security patches
and other manufacturer-recommended changes.

Regards
   Brian Carpenter

---

From: Joel Jaeggli <[email-masked]>
Date: Thu, 14 Oct 2010 19:34:53 -0700

On 10/14/10 1:09 PM, Fred Baker wrote:
>> It seems like Mr. Smith and Mr. Arkko are in disagreement about
>> whether software updates should be downloaded and applied
>> automatically, and I don't know how to proceed from here.
> 
> Jari, being on the IESG, is copied in this. I would suggest that he,
> Mark, and Barbara argue their relative cases. At the end of the day,
> the reason that I asked for a final discussion is that there have
> been several changes, and the document has to satisfy not only it's
> last reviewer, but the operational community that is recommending
> it.
> 
> - Jari wants the patches applied automatically, which assures patch
> updates but removes a level of control from the owner/operator of the
> device
> 
> - Barbara wants the owner/operator to have an interface that can
> apply a patch, but leave anything else undefined; this basically
> preserves the status quo, which means that residential gateways are
> likely to generally *not* track patches and as a result have security
> vulnerabilities, which is also the status quo.
> 
> A middle ground, perhaps, would be to recommend that the manufacturer
> provide an option that the user can select; the present language's
> "by default" suggests this as well. The options would be for the
> device to periodically access a manufacturer web site and download
> updated firmware or for the manufacturer to somehow (blast email?)
> advise the registered owner to do so at their convenience. The issue
> for this draft would be "what is the default", as that will be the
> most common actual behavior. 

This seems a lot alike an implementation detail, if a device does not
update itself automatically then the acceptance of updates by end users
is going to be terribly low, if it does, they might approach 100%. I can
observe in another business that voluntary software updates on mobile
phones (speaking my own employer) have a rather low uptake rate (like
under 10%) ota updates for carrier supported devices on the the other
hand may approach 100%.

so if you actually want the devices to be up-datable in order to address
problems in the field they really do need to do so autonomously, there
no way around that.

Still for a device procured by a consumer it should be between them and
their vendor as to whether this happens.

For the record I think the text for rec-13 is sufficiently strong that
people with a good reason to do so will ignore it and as a general case
recommendation it is fine.

> _______________________________________________ v6ops mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Fernando Gont <[email-masked]>
Date: Fri, 15 Oct 2010 03:34:43 -0300

>> vulnerabilities, which is also the status quo.

Not to add more noise to this issue, but.... have you guys considered
Evilgrade? (see
<https://www.defcon.org/images/defcon-18/dc-18-presentations/Amato-Kirschabum/DEFCON-18-Amato-Kirschabum-Evilgrade.pdf>
and others...)

Bottom-line: beware with what you recommend, because you might get it ;-)

Thanks,
-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Joel Jaeggli <[email-masked]>
Date: Fri, 15 Oct 2010 00:19:25 -0700

Joel's widget number 2

On Oct 14, 2010, at 23:34, Fernando Gont <[email-masked]> wrote:

>>> Yhy- Barbara wants the owner/operator to have an interface that can

There are abundant examples of how to build a trustworthy update system. The fact that a bad one can also be built is not reason to avoid using one.

---

From: Fernando Gont <[email-masked]>
Date: Fri, 15 Oct 2010 04:42:27 -0300

On 15/10/2010 04:19 a.m., Joel Jaeggli wrote:

>> Not to add more noise to this issue, but.... have you guys
>> considered Evilgrade? (see 
> There are abundant examples of how to build a trustworthy update
> system. The fact that a bad one can also be built is not reason to
> avoid using one.

If "automatic updates" are mentioned, a note about these issues should
probably be included, too. (a pointer, at the very least... not
necessarily to this specific tool/presentation).

Thanks,

---

From: Ronald Bonica <[email-masked]>
Date: Fri, 15 Oct 2010 14:56:45 -0400

Folks,

I think that the business model is important, here. 

Assume that I have purchased an "managed service" from my friendly, local ISP. They provide IPv6 connectivity and drop off a router in my kitchen. I plug my computer into the router and never think about it again. If the ISP discovers a bug in the router software, they update it automatically and I am none the wiser. (This is the model that Jari suggests. It is also my ISP's service model.)

Now, assume that I purchase an "unmanaged service". The ISP provides IPv6 connectivity and I go to Crazy Eddy's Electronics World to purchase an Acme-100 router, running ACME-OS Version 1.1. A few months later, the Acme corporation discovers a bug in their implementation of feature X. So they release ACME-OS Version 1.2. While version 1.2 corrects the bug in feature X, it introduces another bug in feature Y.

In this case, the user knows best whether he wants to upgrade to version 1.2. Users who rely on feature X will want to upgrade, while users who do not rely on feature X will not be so motivated. In this case, the model that Mark suggests makes most sense.

Acme corporation wants to sell their gear into both managed and unmanaged environments. So, we need to find some middle ground. I would suggest a configuration switch between the two modes proposed above. Since the vast majority of residential users have no idea what their routers are doing, I would suggest Jari's mode as default.

                                                                     Ron

> Of Joel Jaeggli
> Sent: Thursday, October 14, 2010 10:35 PM
> To: Fred Baker
> Cc: IPv6 Operations; [email-masked]
> Subject: Re: [v6ops] Fwd: I-D Action:draft-ietf-v6ops-cpe-simple-
> security-15.txt
>

---

From: "Yiu L. Lee" <[email-masked]>
Date: Fri, 15 Oct 2010 16:38:03 -0400

As a user, I also don't like automatic update w/o my consent.

On 10/14/10 4:22 PM, "Brian E Carpenter" <[email-masked]>
wrote:

>>> It seems like Mr. Smith and Mr. Arkko are in disagreement about whether
>>> software updates should be downloaded and applied automatically, and I don't
>>> know how to proceed from here.
>> 
>> Jari, being on the IESG, is copied in this. I would suggest that he, Mark,
>> and Barbara argue their relative cases. At the end of the day, the reason
>> that I asked for a final discussion is that there have been several changes,
>> and the document has to satisfy not only it's last reviewer, but the
>> operational community that is recommending it.
>> 
>>  - Jari wants the patches applied automatically, which assures patch updates
>> but removes a level of control from the owner/operator of the device
> 
>>  - Barbara wants the owner/operator to have an interface that can apply a
>> patch, but leave anything else undefined; this basically preserves the status
>> quo, which means that residential gateways are likely to generally *not*
>> track patches and as a result have security vulnerabilities, which is also
>> the status quo.
>> 
>> A middle ground, perhaps, would be to recommend that the manufacturer provide
>> an option that the user can select; the present language's "by default"
>> suggests this as well. The options would be for the device to periodically
>> access a manufacturer web site and download updated firmware or for the
>> manufacturer to somehow (blast email?) advise the registered owner to do so
>> at their convenience. The issue for this draft would be "what is the
>> default", as that will be the most common actual behavior.

---

From: Thomas Herbst <[email-masked]>
Date: Fri, 15 Oct 2010 13:41:20 -0700

Based on past work in a consumer networking company, my understanding is that there are places in Europe where 
automatic updates are not legal if the device is consumer owned.

---

From: David Conrad <[email-masked]>
Date: Fri, 15 Oct 2010 11:00:42 -1000

> As a user, I also don't like automatic update w/o my consent.

If you have a TiVo, does it bother you that it updates itself without consent?

I might observe that anyone on this mailing list is not the target audience of automatic update.  

I suspect it would be more useful to view this from a risk mitigation/"least harm" perspective.  The reality is that the vast majority of users are unlikely to even know how to log into their CPE, much less be able to log in, determine whether or not installing an update is a good idea, and install that update.  Given the types of attacks we're seeing today (e.g., Drive-by Pharming), it seems to me that there are more risks to the Internet as a whole to requiring manual intervention than not.

With that in mind, I'd prefer that there be a recommendation that automatic updates be enabled by default with a mechanism to allow users to disable those updates if they so desire.

Regards,
-drc

---

From: Joel Jaeggli <[email-masked]>
Date: Fri, 15 Oct 2010 14:19:50 -0700

> As a user, I also don't like automatic update w/o my consent.

however comcast automatically updates your STB's and the docsis modems
with some degree of frequency...

Who owns and or manages devices that constitute the cpe varies by device
type business model and so on.

Your virus and potentially IDS and firewall definitions probably update
themselves automatically and without your consent, you can in all
likelihood turn that off.

Saying generically,  "I don't like automatically applying updates" is
fine, stating that consumer oriented devices SHOULD NOT update
themselves without a users' consent dooms them to running only the
software that they came with by in large.

---

From: "Yiu L. Lee" <[email-masked]>
Date: Fri, 15 Oct 2010 17:25:45 -0400

I give this comment as a user.

If this is a managed device (STB and CM), I don't mind they will do the
update for me. I know that if bad things happened, I could ask for a new
one. However, if this is an unmanaged device, I would like to be informed.
Any update has risk, I just want to make sure the users will understand the
risk.

On 10/15/10 5:19 PM, "Joel Jaeggli" <[email-masked]> wrote:

>>>> updates
>>>> provide

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 15 Oct 2010 22:27:38 +0100

> about automatic vs consent vs voluntary.

Speaking as an end user, I don't have a major problem with CPEs updating
themselves automatically.  Other than that, I agree with Brian completely.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 15 Oct 2010 22:29:56 +0100

> Any update has risk, I just want to make sure the users will understand the

Most users generally don't understand and aren't interested in knowing
about the risk associated with automatic updates.  You are the exception.

Oh dear.  This is becoming a bikeshed issue.

Nick

---

From: "Yiu L. Lee" <[email-masked]>
Date: Fri, 15 Oct 2010 17:35:02 -0400

Hi Dave,

> If you have a TiVo, does it bother you that it updates itself without consent?
I don't own a TiVo, so I can't comment.

> I suspect it would be more useful to view this from a risk mitigation/"least
> harm" perspective.  The reality is that the vast majority of users are
> unlikely to even know how to log into their CPE, much less be able to log in,
> determine whether or not installing an update is a good idea, and install that
> update.  Given the types of attacks we're seeing today (e.g., Drive-by
> Pharming), it seems to me that there are more risks to the Internet as a whole
> to requiring manual intervention than not.
> 
Agreed

> With that in mind, I'd prefer that there be a recommendation that automatic
> updates be enabled by default with a mechanism to allow users to disable those
> updates if they so desire.
> 
I think this is the linux, Windows and MacOSX approach. I think this is
acceptable to turn on the automatic update by default and let users to
disable it if they decide to.

Regards,
Yiu

---

From: Mark Baugher <[email-masked]>
Date: Fri, 15 Oct 2010 14:39:38 -0700

Right, what we have is a policy knob where some vendors will choose to one thing, others may choose to do another, and some countries have laws dictating what must be done.  I would think that we can motivate a SHOULD based on what the WG/IESG thinks is the best practice.  But that will vary by vendor, customer agreement, local laws and other matters.

Mark

---

From: "Yiu L. Lee" <[email-masked]>
Date: Fri, 15 Oct 2010 17:42:45 -0400

Well, I learned it in a hard way. One time I updated the firmeware (use the
automatic update feature) of a CPE and it bricked. I can't recover it and of
course I can't get an exchange since the router was older than one year. If
the update procedure is clean every time (like Windows and MacOSX update), I
will not be so careful of the automatic update.

Anyhow, I also agree Brian's email, IETF should recommend patch mechanism
but stay neutral about voluntary vs consent.

---

From: Mark Smith <[email-masked]>
Date: Sat, 16 Oct 2010 08:18:11 +1030

Hi Ron,

On Fri, 15 Oct 2010 14:56:45 -0400

> Assume that I have purchased an "managed service" from my friendly, local ISP. They provide IPv6 connectivity and drop off a router in my kitchen. I plug my computer into the router and never think…

The fundamental problem is that this draft is aimed at vendors. With
this clause the vendor chooses when and where to provide firmware
updates to the CPE users. So it won't be possible for the ISP to provide a
managed CPE service, because they don't control and can't be in control
of when the firmware is released to the CPE customers. They may not
even be aware of a new firmware release until they get a flood of
support calls about the Internet breaking.

If the draft is going to persist with specifying that firmware is
automatically distributed and installed, then it is also going to have
to specify mechanisms as to how ISPs are going to put inline with this
process, between the vendor and the CPE, so they're going to be able to
prevent automatic upgrades occurring at times when it is inconvenient to
customers. Of course, that will only be successful if all CPE vendors
comply with this "ISP controlled" mechanism. That'll also create an
additonal burden on the ISPs because they'll end up being responsible
for software maintenance for their customer's CPE, even in markets
where customers can purchase CPE independently of ISPs.

Think about applying this model to other devices and I hope you'll see
the fundamental problem -

o  would you be happy that your Internet attached TV's firmware is
automatically updated and installed when the vendor makes firmware
available, right in the middle of a movie?

o  would you be happy that your (eventual) Internet attached ECU's
firmware is automatically updated and installed when the car vendor
makes firmware available, right at the moment when you're travelling at
100Kph/60Mph?

o  would you be happy that your Internet attached plane's firmware is
automatically updated and installed when the plane vendor makes
firmware available, right at the moment you're a passenger in it at it,
and it is in the middle of taking off?

In each of these cases, the operator of the device has to be in control
of when these firmware upgrades occur as they can make the best
operational choice as to when they should occur. Device vendors
usually aren't also operators. For ISP managed CPE, the ISP is
the operator. For privately purchased CPE, the customer has chosen to
take on the responsibility of being the device's operator and is
therefore responsible for it's maintenance - if they don't want to be an
operator, they need to be prepared to engage somebody else to
perform that function. 

I certainly understand the issue of CPE firmware not being upgraded
regularly. However I think that may be more of a human-computer
interface issue. It isn't easy enough for non-technical people to do -
methods such as a flashing LED, persistent beeper and a upgrade
firmware button might be far more successful. But automated updates that
are distributed and installed without notice or control by ISPs or the
end-CPE owners won't solve the problem without causing possibly bigger
ones.

Regards,
Mark.

---

From: David Conrad <[email-masked]>
Date: Fri, 15 Oct 2010 11:49:41 -1000

On Oct 15, 2010, at 11:35 AM, Yiu L. Lee wrote:
> I think this is the linux, Windows and MacOSX approach.

Just to be clear, I believe all of these require manual intervention. I'm arguing against this. If a device requires manual intervention, I'd argue that people won't bother and we'll end up with broken, vulnerable CPEs as we do today, just a whole lot more of them.

(Note that this assumes vendors will actually update their boxes instead of through inaction  encouraging people to go out and buy new ones)

> I think this is

Ideally, during installation/initial configuration the user could be presented with the option of disabling unattended automatic update (since I figure installation/initial configuration is typically the only time most users log into the box).

Regards,
-drc

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 15 Oct 2010 16:52:30 -0500

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Yiu L. Lee
Sent: Friday, October 15, 2010 5:35 PM
To: David Conrad
Cc: IPv6 Operations; [email-masked]
Subject: Re: [v6ops] Fwd: I-D
Action:draft-ietf-v6ops-cpe-simple-security-15.txt

>disable it if they decide to.

Users of machines that run Windows, Linux, or MaxOS are generally more
aware of the security updates because the machines tell them so - these
are PCs used by folks with a nice GUI etc.  The home gateway is a
consumer device that a customer is not logged on for several hours a day
like working on their PCs.  Thus auto upgrade default of On for the home
router is debatable.

Hemant

---

From: David Conrad <[email-masked]>
Date: Fri, 15 Oct 2010 12:16:26 -1000

Mark,

On Oct 15, 2010, at 11:48 AM, Mark Smith wrote:
> So it won't be possible for the ISP to provide a
> of when the firmware is released to the CPE customers.

I'd think that recommending having an option that disables unattended automatic update would address this concern.  Managed service providers, since they'd be controlling the CPE, could go in and disable unattended automatic updates (if they so desire).

> customers.

Not really.

> o  would you be happy that your Internet attached TV's firmware is

Sort of like my TiVo updating itself?

> o  would you be happy that your (eventual) Internet attached ECU's
[...]
> o  would you be happy that your Internet attached plane's firmware is

Presumably, vendors would figure out how to do upgrades with minimal risk if the decided to follow the IETF recommendations, e.g., the ECU can easily tell when the car isn't running (or even occupied) and a plane's systems can easily determine whether the engines are running (or whatever). Posing reductio ad absurdum scenarios probably isn't really helping the discussion.

> regularly.

According to recent data out of DNS-OARC, there are between 160,000 and 320,000 name servers on the Internet still running BIND version 4 -- a name server that has been deprecated by ISC now for almost a decade.

People generally do not interact with CPE devices outside of perhaps initial installation/configuration.  They might cycle the power once in a while, but otherwise it is a black box.  These unattended devices have proven to be a target for various forms of attacks. I believe we should learn from past (and current) experiences and make recommendations that address those experiences.  Assuming that folks are going to become more conscientious about installing updates hasn't really worked in the past, I'm unsure why it will work in the future.

Regards,
-drc

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 16 Oct 2010 11:40:21 +1300

> I'd think that recommending having an option that disables unattended automatic update would address this concern.  Managed service providers, since they'd be controlling the CPE, could go in and d…

I think the discussion has shown that we (the IETF) don't have consensus
and that legal and commercial requirements will vary enormously in the
real world. To me there's only one possible conclusion: the IETF can't
"legislate" on this. So again here is my suggestion:

REC-13:

    Brian

---

From: james woodyatt <[email-masked]>
Date: Sat, 16 Oct 2010 12:32:33 -0700

On Oct 15, 2010, at 14:29, Nick Hilliard wrote:
> 
> This is becoming a bikeshed issue.

"What color is the bikeshed at Hereford?"

Wait.  I didn't get that movie reference at all right, did I?

---

From: Nick Hilliard <[email-masked]>
Date: Sat, 16 Oct 2010 20:54:13 +0100

> Wait.  I didn't get that movie reference at all right, did I?

http://www.bikeshed.com/

Just in case there's any doubt, the only acceptable version of this web
site is http://mediumaquamarine.bikeshed.com/.  All other colours are
objectively wrong and choosing them would display very bad judgement.

In my humble opinion, that is.

Nick

---

From: Scott Brim <[email-masked]>
Date: Sat, 16 Oct 2010 16:50:18 -0400

> In my humble opinion, that is.

http://en.wikipedia.org/wiki/Parkinson%27s_Law_of_Triviality

Also, allegedly, Henry Kissinger said about Harvard faculty meetings
that "the fighting is so bitter because the stakes are so small."

---

From: Sam Silvester <[email-masked]>
Date: Mon, 18 Oct 2010 11:47:30 +1030

>> The issue for this draft would be "what is the default", as that will be the most common actual behavior.

As an operator, I'd like to avoid vendor patches to be automatically
installed for the reasons noted here (mass rebooting, helpdesk calls,
potential for nasty bugs to be "mass installed by default" etc).

I don't mind the idea of mechanisms being switchable to do automatic
install or an appropriate interface provided to the operator to allow
for this to be enabled for managed services.

> I suggest tweaking the recommendation to say that an "easily configured administrative option" to download and apply updates automatically SHOULD be offered.

Whilst I don't offer a suggestion on the exact wording, I think it
would be good to recommend that devices have a mechanism for automatic
download - at least that way people are 'encouraged' or at least aware
without having to know to hunt around on the vendor's website for
updates.

I do wonder about the UI in terms of notifying users that some action
is required on their part when a new update is available for install.
It's probably an implementation detail however.

---

From: Joel Jaeggli <[email-masked]>
Date: Sun, 17 Oct 2010 23:30:05 -0700

> potential for nasty bugs to be "mass installed by default" etc).

An "automatic" process has some instigating condition, I imagine in for
example the docsis context this would be instigated by the operator, in
a cellular OMA/DM/FOTA context it's certainly instigated by the operator...

It strikes me as unlikely that we'll be able produce language that is
appropriate for every context, The general sense that a mechanism by
which automatic software updates can be applied should be provided
leaves leeway for application specific methods to be applied.

---

From: Sam Silvester <[email-masked]>
Date: Tue, 19 Oct 2010 20:33:31 +1030

> a cellular OMA/DM/FOTA context it's certainly instigated by the operator...

I'm rather ok with the operator case (the company I work for has a mix
of products including some of which we manage CPE software upgrades
centrally). The one that I'm a little more concerned about is if the
CPE vendor themselves can instigate the upgrade - especially in the
case of residential CPE, there isn't much in the way of testing or
quality control often.

> leaves leeway for application specific methods to be applied.

I agree broadly, although I do wonder a little if something can or
should to be said toward the previously mentioned "vendor instigated"
upgrade scenario.

Sam

---

From: Fred Baker <[email-masked]>
Date: Tue, 19 Oct 2010 11:40:07 -0700

> and other manufacturer-recommended changes.

I have been following this discussion with a view to figuring out what recommendations SHOULD be made :-) A key concern I have here, as I said last week, is the difference between a residential gateway and a laptop; A laptop's owner at least knows whether an irritating icon is jumping up and down or has the opportunity to see a dialog box; I don't see that happening with the router in my equipment closet under the stairs.

It seems that we agree that there are at least two options that need support:
   - automated software update
   - manually initiated software update

One could go into other mechanisms such as having the user download the new image to a laptop and then TFTP it to a router. Some things are best left unsuggested, for fear that the vendor might say "hey, that's a great idea!".

In both cases, the update has to be initiated by the residential CPE, for scale and reliability, and if automated should be randomized in time for reasons discussed in RFC 3439. Having the ISP or the vendor manage a list of the IP addresses of deployed routers is (ahem) problematic.

In both cases, we need a way to specify the URL and certificate of the download site. I should imagine the vendor would preconfigure them, and an ISP running managed services could configure a different URL+certificate in routers it deploys. There is of course the question of what happens should the parameter memory get zapped; I would leave that to the vendor. There is also the question of how the device knows whether it has already downloaded the image; I for one would do that by having the URL change itself, so that the client system ca always get access to an older image as a recovery path and so it can tell whether the image it is requesting is in fact new.

In preceding recommendations, the format of the document is to state a recommendation and follow it with an explanatory note. 

Which brings me to this suggestion:

/*
 * suggestion
 */
REC-13:
Residential Internet Gateways SHOULD provide a convenient means to securely update their firmware, for the installation of security patches and other manufacturer-recommended changes.

Vendors can expect users and operators to have differing viewpoints on the maintenance of patches, with some preferring automated update and some preferring manual initiation, and those preferring automated update wanting to download from a vendor site or one managed by the network operator. To handle the disparity, vendors are well advised if they provide manual and automated options. In the automated case, they would do well to facilitate pre-configuration of the download URL and a means of validating the software image such as a certificate.
/*
 * end of suggestion
 */

Opinions?

---

From: james woodyatt <[email-masked]>
Date: Tue, 19 Oct 2010 12:05:26 -0700

> In preceding recommendations, the format of the document is to state a recommendation and follow it with an explanatory note. 

Actually, there's a lot of conceptual introduction followed by recommendation.  The current version of REC-13 is an example of this form.

> Vendors can expect users and operators to have differing viewpoints on the maintenance of patches, with some preferring automated update and some preferring manual initiation, and those preferring…

I like this color.  Can I paint the bikeshed now?

p.s. I'd be inclined to leave alone the current paragraph preceding REC-13, then replace REC-13 with all of your suggested text above.

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Oct 2010 21:16:59 +0200

Hi,

> Vendors can expect users and operators to have differing viewpoints on the maintenance of patches, with some preferring automated update and some preferring manual initiation, and those preferring…

Works for me.  Both from the ISP side and from the end user side.

Gert Doering
-- 
did you enable IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 19 Oct 2010 15:19:06 -0400

Me too

> Of Gert Doering
> Sent: Tuesday, October 19, 2010 3:17 PM
> To: Fred Baker
> Cc: Operations; IESG IESG; [email-masked]
> securely update their firmware, for the installation of security
> patches and other manufacturer-recommended changes.
> >
> > Vendors can expect users and operators to have differing viewpoints
> on the maintenance of patches, with some preferring automated update
> and some preferring manual initiation, and those preferring automated
> update wanting to download from a vendor site or one managed by the
> network operator. To handle the disparity, vendors are well advised if
> they provide manual and automated options. In the automated case, they
> would do well to facilitate pre-configuration of the download URL and a
> means of validating the software image such as a certificate.
> > /*
> >  * end of suggestion
> >  */
>

---

From: <[email-masked]>
Date: Tue, 19 Oct 2010 16:00:48 -0400

I like it. As Gert already said, it speaks to both views.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Fred Baker
Sent: Tuesday, October 19, 2010 2:40 PM
To: Operations; IESG IESG
Cc: [email-masked]
Subject: Re: [v6ops] Fwd: I-D Action:draft-ietf-v6ops-cpe-simple-security-15.txt

/*
 * suggestion
 */
REC-13:

Opinions?

---

From: "STARK, BARBARA H (ATTLABS)" <[email-masked]>
Date: Tue, 19 Oct 2010 16:11:15 -0400

It's fine.
Barbara

> Of [email-masked]
> Sent: Tuesday, October 19, 2010 4:01 PM
> To: [email-masked]; [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: Re: [v6ops]Fwd: I-D Action:draft-ietf-v6ops-cpe-simple-
> security-15.txt
> 
> >> I'd think that recommending having an option that disables
> unattended automatic update would address this concern.  Managed
> service providers, since they'd be controlling the CPE, could go in
and
> disable unattended automatic updates (if they so desire).
> >
> patches
> I have been following this discussion with a view to figuring out what
> recommendations SHOULD be made :-) A key concern I have here, as I
said
> last week, is the difference between a residential gateway and a
> laptop; A laptop's owner at least knows whether an irritating icon is
> jumping up and down or has the opportunity to see a dialog box; I
don't
> see that happening with the router in my equipment closet under the
> stairs.
> 
> One could go into other mechanisms such as having the user download
the
> new image to a laptop and then TFTP it to a router. Some things are
> best left unsuggested, for fear that the vendor might say "hey, that's
> a great idea!".
> 
> In both cases, the update has to be initiated by the residential CPE,
> for scale and reliability, and if automated should be randomized in
> time for reasons discussed in RFC 3439. Having the ISP or the vendor
> manage a list of the IP addresses of deployed routers is (ahem)
> problematic.
> 
> In both cases, we need a way to specify the URL and certificate of the
> download site. I should imagine the vendor would preconfigure them,
and
> an ISP running managed services could configure a different
> URL+certificate in routers it deploys. There is of course the question
> of what happens should the parameter memory get zapped; I would leave
> that to the vendor. There is also the question of how the device knows
> whether it has already downloaded the image; I for one would do that
by
> having the URL change itself, so that the client system ca always get
> access to an older image as a recovery path and so it can tell whether
> the image it is requesting is in fact new.
> 
> In preceding recommendations, the format of the document is to state a
> recommendation and follow it with an explanatory note.
> 
> Vendors can expect users and operators to have differing viewpoints on
> the maintenance of patches, with some preferring automated update and
> some preferring manual initiation, and those preferring automated

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Oct 2010 09:29:21 +1300

Works for me too.

Regards
   Brian Carpenter

---

From: Mark Smith <[email-masked]>
Date: Wed, 20 Oct 2010 07:23:04 +1030

Hi Fred,

I'm ok with it.

Regards,
Mark.

On Tue, 19 Oct 2010 11:40:07 -0700

---

From: "Frank Bulk" <[email-masked]>
Date: Tue, 19 Oct 2010 22:12:26 -0500

I second David's idea.  It has the proper balance that allows the general
user to keep a properly-updated CPE, while allowing power users, who might
be annoyed by service interruptions, to disable it.

All this discussion doesn't preclude a CPE vendor from implementing a system
that only patches and reboots during the wee hours of the morning or upon a
power-cycle.

Frank

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
David Conrad
Sent: Friday, October 15, 2010 4:01 PM
To: Yiu L. Lee
Cc: IPv6 Operations; [email-masked]
Subject: Re: [v6ops] Fwd: I-D
Action:draft-ietf-v6ops-cpe-simple-security-15.txt

> As a user, I also don't like automatic update w/o my consent.

If you have a TiVo, does it bother you that it updates itself without
consent?

I might observe that anyone on this mailing list is not the target audience
of automatic update.  

I suspect it would be more useful to view this from a risk mitigation/"least
harm" perspective.  The reality is that the vast majority of users are
unlikely to even know how to log into their CPE, much less be able to log
in, determine whether or not installing an update is a good idea, and
install that update.  Given the types of attacks we're seeing today (e.g.,
Drive-by Pharming), it seems to me that there are more risks to the Internet
as a whole to requiring manual intervention than not.

With that in mind, I'd prefer that there be a recommendation that automatic
updates be enabled by default with a mechanism to allow users to disable
those updates if they so desire.

Regards,
-drc

---

From: "Frank Bulk" <[email-masked]>
Date: Tue, 19 Oct 2010 22:20:06 -0500

I'm afraid if we use "SHOULD", CPE vendors will follow the path so far
followed, which is not building that option into their product.

I'd rather see a "MUST" to the automatic update functionality, but a
"SHOULD" in regards whether it's turned on by default or not.

Frank

-----Original Message-----
Fred Baker
Sent: Tuesday, October 19, 2010 1:40 PM

On Oct 15, 2010, at 3:40 PM, Brian E Carpenter wrote:

>> I'd think that recommending having an option that disables unattended
automatic update would address this concern.  Managed service providers,
since they'd be controlling the CPE, could go in and disable unattended
automatic updates (if they so desire).

I have been following this discussion with a view to figuring out what
recommendations SHOULD be made :-) A key concern I have here, as I said last
week, is the difference between a residential gateway and a laptop; A
laptop's owner at least knows whether an irritating icon is jumping up and
down or has the opportunity to see a dialog box; I don't see that happening
with the router in my equipment closet under the stairs.

It seems that we agree that there are at least two options that need
support:
   - automated software update
   - manually initiated software update

One could go into other mechanisms such as having the user download the new
image to a laptop and then TFTP it to a router. Some things are best left
unsuggested, for fear that the vendor might say "hey, that's a great idea!".

In both cases, the update has to be initiated by the residential CPE, for
scale and reliability, and if automated should be randomized in time for
reasons discussed in RFC 3439. Having the ISP or the vendor manage a list of
the IP addresses of deployed routers is (ahem) problematic.

In both cases, we need a way to specify the URL and certificate of the
download site. I should imagine the vendor would preconfigure them, and an
ISP running managed services could configure a different URL+certificate in
routers it deploys. There is of course the question of what happens should
the parameter memory get zapped; I would leave that to the vendor. There is
also the question of how the device knows whether it has already downloaded
the image; I for one would do that by having the URL change itself, so that
the client system ca always get access to an older image as a recovery path
and so it can tell whether the image it is requesting is in fact new.

/*
 * suggestion
 */
REC-13:
Residential Internet Gateways SHOULD provide a convenient means to securely
update their firmware, for the installation of security patches and other
manufacturer-recommended changes.

Vendors can expect users and operators to have differing viewpoints on the
maintenance of patches, with some preferring automated update and some
preferring manual initiation, and those preferring automated update wanting
to download from a vendor site or one managed by the network operator. To
handle the disparity, vendors are well advised if they provide manual and
automated options. In the automated case, they would do well to facilitate
pre-configuration of the download URL and a means of validating the software
image such as a certificate.
/*
 * end of suggestion
 */

Opinions?

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 20 Oct 2010 03:35:20 +0000

+1

---

From: Rémi Després <[email-masked]>
Date: Wed, 20 Oct 2010 09:10:53 +0200

> I like this color.  Can I paint the bikeshed now?

Yes IMHO
I find this a good compromise.  

> p.s. I'd be inclined to leave alone the current paragraph preceding REC-13, then replace REC-13 with all of your suggested text above.

My support holds with or without the sentence, or also with Fred's paragraph before REC-13 in place of the current sentence.

Regards,
RD

---

From: Sam Silvester <[email-masked]>
Date: Wed, 20 Oct 2010 22:25:53 +1030

> Vendors can expect users and operators to have differing viewpoints on the maintenance of patches, with some preferring automated update and some preferring manual initiation, and those preferring…

Covers both cases and explains the situation well. Looks good.

Sam