Subject: [pcp] PCP failure scenarios & PCP state synchronization procedure

From: <[email-masked]>
Date: Thu, 6 Jan 2011 15:19:24 +0100

Dear all,
 
We have submitted a new I-D available at http://tools.ietf.org/html/draft-boucadair-pcp-failure-00.

Abstract

   This document identifies and analyses several PCP failure scenarios.
   A procedure to retrieve the mapping(s) from the PCP Server is
   proposed.  This procedure relies upon the use of two new PCP OpCodes:
   GET and GETNEXT.
 
Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  PCP Failure Scenarios  . . . . . . . . . . . . . . . . . . . .  3
     2.1.  Change of the IP Address of The PCP Server . . . . . . . .  3
     2.2.  Application Crash  . . . . . . . . . . . . . . . . . . . .  3
     2.3.  PCP Client Crash . . . . . . . . . . . . . . . . . . . . .  4
     2.4.  Change of the Internal IP Address  . . . . . . . . . . . .  4
     2.5.  Change of the CPE WAN IP Address . . . . . . . . . . . . .  5
     2.6.  Restart or Failure of the PCP Server . . . . . . . . . . .  5
       2.6.1.  Basic Rule . . . . . . . . . . . . . . . . . . . . . .  5
       2.6.2.  Clear PCP Mappings . . . . . . . . . . . . . . . . . .  6
       2.6.3.  State Redundancy is Enabled  . . . . . . . . . . . . .  6
       2.6.4.  Cold-Standby without State Redundancy  . . . . . . . .  6
       2.6.5.  Anycast Redundancy Mode  . . . . . . . . . . . . . . .  6
   3.  PCP State Synchronization Overview . . . . . . . . . . . . . .  6
   4.  GET and GETNEXT OpCodes  . . . . . . . . . . . . . . . . . . .  7
     4.1.  Message Format . . . . . . . . . . . . . . . . . . . . . .  7
     4.2.  New Result Codes . . . . . . . . . . . . . . . . . . . . .  8
     4.3.  GET: Retrieving One Single Mapping . . . . . . . . . . . .  9
     4.4.  GETNEXT: Retrieve the Mapping Table  . . . . . . . . . . .  9
       4.4.1.  Filter . . . . . . . . . . . . . . . . . . . . . . . .  9
       4.4.2.  Mapping Table Order  . . . . . . . . . . . . . . . . . 10
       4.4.3.  Behavior . . . . . . . . . . . . . . . . . . . . . . . 10
       4.4.4.  Misc . . . . . . . . . . . . . . . . . . . . . . . . . 11
     4.5.  Flow Examples  . . . . . . . . . . . . . . . . . . . . . . 11

Comments and inputs are welcome.
 
Cheers,
Med
*********************************
This message and any attachments (the "message") are confidential and intended solely for the addressees. 
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration. 
France Telecom Group shall not be liable for the message if altered, changed or falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender.
********************************

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 11 Jan 2011 11:45:18 -0800

> We have submitted a new I-D available at
> http://tools.ietf.org/html/draft-boucadair-pcp-failure-00.
...

This is a review of draft-boucadair-pcp-failure-00.txt.

First, thanks for writing this document -- it clarifies a lot of 
email discussions.

Section 2.2, "Application Crash", says in part:

   To avoid this inconvenience induced by stale mappings, the PCP Client
   MAY clear the "old" mappings before issuing the refresh requests; but
   this would require the PCP Client to store the information about the
   "old" port number.  This may not be easy in some scenarios because
   the PCP Client may handle PCP requests on behalf of several
   applications and no means to identify the requesting application may
   be supported.

Nothing is said about a PCP client embedded into an application.  I presume
that could work okay, by the application writing its PCP-allocated ports to
permanent storage (a file) to try to re-use after it crashes and
re-initializes.
Yes?

I noticed this note at the end of that same section that says:

      [Ed.  This is in fact about the "application ID" idea.]

For that to work, there must be a mechanism for the OS-embedded PCP client 
to know which application that create a mapping.  Perhaps, for example,
an opaque string is passed from the application to the OS-embedded
PCP client.  Once that mechanism exists, can't we foist the problem
onto the OS-embedded PCP client?  This could be as straight forward
as having the client code do this, for example:

  pcp -> PCP.new("Cisco SIP Client");
  pcp.remove_stale_mappings();

Once we have such a mechanism to identify an application in the 
OS-embedded PCP client's API, I believe the problems identified
by Section 2.2 mostly go away, because an application restart
can be detected by the OS-embedded PCP client.  

Section 2.4, "Change of the Internal IP Address"

   When a new IP address is assigned to a host embedding a PCP Client,
   the PCP Client MUST install on the PCP Server all the mappings it
   manages, using the new assigned IP address as the internal IP
   address.

This needs some additional discussion.  Let's take an example where
I have a computer with both WiFi and wired Ethernet connections
and I plug and unplug the Ethernet connection, or go in and out
of range of WiFi.  When do we want PCP to do something about
that change of connectivity?   Does the answer depend on IPv4,
IPv6, or dual stack running on that interface if we're doing PIN44, 
PIN64, PIN46, PIN66?  

Section 2.4 also says:

  "   [Ed.  Note: Do we need to support means to clear stale mappings
      first?  This may have an impact if the quota is exceed due to the
      presence of stale mappings.]

   A PCP Client may be used to manage pinholes on behalf of a third
   party (i.e., the PCP Client and the third party are not co-located on
   the same host).  If a new internal IP address is assigned to that
   third party (e.g., webcam), the PCP Client SHOULD be instructed to
   delete the old mapping(s) and create new one(s) using the new
   assigned internal IP address."

(As you know,) PCP currently has a mechanism that supports doing 
that - a PCP client can clear mappings for another host within the 
subscriber's network.  I believe you're asking if we want to 
encourage doing that.  I am worried about supporting such deletes 
at all.  Let's take as an example that my IP address is 192.168.1.1 
while I'm at home and then I visit a friend's house, were I get
a different address.  With a naïve implementation, I would simply 
clear mappings for 192.168.1.1, which is my friend's webcam.  
Oops.  A smarter implementation would notice that my SSID is 
different from my friend's SSID (SSID is something an OS-
embedded PCP client could know) but SSIDs can collide (all
the Starbucks use the same SSID) and Ethernet doesn't have a 
handy SSID.

The second paragraph I quoted above says that a PCP client
that created a pinhole for another device needs to refresh
the pinhole if the other device's IP address changes.  In
a PC, I don't see how that could be implemented.  It could
be implemented in a CPE, if that CPE is also running DHCP
and interfaces DHCP leases with PCP.

We may want to consider disallowing PCP from deleting another
host's mappings, or pull that feature entirely.  I created
ticket #33 "3rd party mappings: create/destroy" to track this issue,
http://trac.tools.ietf.org/wg/pcp/trac/ticket/33

Section 2.5, " Change of the CPE WAN IP Address" should also
discuss the NAT44 case (a.k.a. "NAT444").  That case suffers
from a similar problem highlighted by the Ed. Note in Section
2.5, where the PCP client has no reasonable way to detect the
CPE's WAN address changed.

It should also be noted in Section 2.5 that, on today's Internet
without CGN and without PCP, subscriber WAN addresses change.
And if the subscriber installed their own NAT, the hosts behind
the NAT are likewise completely unaware of the CPE's WAN
address change.  NAT-PMP includes a mechanism to alert the
hosts of this change, which uses multicast.  We might consider
if PCP could benefit from a mechanism that allows alerting
hosts in some fashion [multicast or unicast].  UPnP IGD does
not include such a mechanism, to my knowledge.

   2.6.2. Clear PCP Mappings

   When a command line or a configuration change is enforced to clear
   all or a subset of PCP mappings maintained by the PCP Server, the PCP
   Server MUST reset its Epoch to zero value.

Does the PCP server need to maintain only one Epoch value, or an Epoch 
value for each subscriber?  If one Epoch value, and only one subscriber's
mappings are changed, it will cause all PCP clients to refresh their 
pinholes -- unnecessarily.  So this means the base specification needs
to encouarge per-subscriber Epoch values.

Then, to the main point of the draft:

The two cases where GET/GETNEXT are useful are when an application
crashes and forgets its mappings (Section 2.2) and when the PCP 
client crashes and forgets its mappings (Section 2.3).

If we require PCP clients (implemented by applications or embedded in
an OS) to store their mappings in permanent storage, do we need
GET/GETNEXT?

Are there PCP clients which cannot store their mappings in 
permanent storage?  If so, let's discuss them.

-d

---

From: <[email-masked]>
Date: Wed, 12 Jan 2011 09:06:29 +0100

Re-,

Please see inline.

Cheers,
Med 

-----Message d'origine-----
De : Dan Wing [mailto:[email-masked]] 
Envoyé : mardi 11 janvier 2011 20:45
À : BOUCADAIR Mohamed OLNC/NAD/TIP; [email-masked]
Objet : RE: [pcp] PCP failure scenarios & PCP state synchronization procedure

...

Med: Yes. The text mentions only the case which poses problem. I updated the text to refer explicitely to this case.

Med: Yes. Note that the local PCP mapping table should have an "application_id" attribute for this to work. The following reco should also be followed: 
"It is NOT RECOMMENDED for a PCP Client to issue a request to delete all the mappings associated with an internal IP address since other applications and PCP Client(s) may use the same internal IP address to instruct their pinholes in the PCP Server."

Med: Good point. IMHO, the address selection should be done according to the-be-completed section: http://tools.ietf.org/html/draft-ietf-pcp-base-02#section-6.1.3. Based on the output of that process, the PCP Client can be triggered accordingly. 

Section 2.4 also says:

encourage doing that.  

Med: The question is not this one but how to remove the stale mappings associated with the "old" used internal IP address.

I am worried about supporting such deletes 
Oops. 

Med: Me too I'm in favour of having means to avoid such behaviour, this is why I have proposed in the past the PCP Identifier (http://www.ietf.org/mail-archive/web/pcp/current/msg00055.html)

 A smarter implementation would notice that my SSID is 
different from my friend's SSID (SSID is something an OS-
embedded PCP client could know) but SSIDs can collide (all
the Starbucks use the same SSID) and Ethernet doesn't have a 
handy SSID.

Med: Yes, and especially for an IWF co-located with the local DHCP server. Otherwise, we can recommend the use of static internal IP addresses if PCP is used to configure third-party pinholes.

Med: Or use the "PCP identifier".

Med: I added a note about the NAT444 case.

Med: I suggest you add an issue into the tracker to record it.

Med: FWIW, the logic of setting the Epoch value when enforcing CLI is described in: http://www.ietf.org/mail-archive/web/pcp/current/msg00091.html 
I'm not sure Epoch is the more efficient per-subscriber hint to audit the mappings; COUNT of the mappings can be considered too.

Med: Yes: some CPEs embedding a PCP Client/IWF.

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 12 Jan 2011 09:02:12 -0800

...
> Med: The question is not this one but how to remove the stale mappings
> associated with the "old" used internal IP address.

You're referring to the problem of detecting that a host's IP address
changed?  I agree that is hard.  I say don't worry about it.  

Let's consider the same situation (host's IP address changes 
arbitrarily) without NAT and without PCP, and where a 3rd party
set up some sort of rendezvous for the device -- because the device
couldn't do that itself.  For example, a simple networked camera
or a simple NAS device, where the user edits a web page that says 
"click here to see my home movies" which points to http://1.2.3.4
(an IP address literal).  This works until the NAS's IP address 
changes.  We have no way to solve that without NAT and without 
PCP.  So, I propose we don't try to solve that with PCP.

> Med: Me too I'm in favour of having means to avoid such behaviour, this
> is why I have proposed in the past the PCP Identifier
> (http://www.ietf.org/mail-archive/web/pcp/current/msg00055.html)

That solves a slew of other problems, though.  And creates some new
problems.

To solve the problem of accidental deletes, here are two other
ideas:

  * a 3rd party mapping could cause an opaque identifier to be 
    included in the PIN response, and require subsequent PIN* 
    requests that reduce the lifetime to include that identifier.  
    A problem with that idea might be loss of the first response, 
    though. 
  * Another idea which doesn't change PCP is that the server,
    on noticing a 3rd party mapping, only allows that same IP
    address to reduce the lifetime of the mapping.

> Med: Yes, and especially for an IWF co-located with the local DHCP
> server. Otherwise, we can recommend the use of static internal IP
> addresses if PCP is used to configure third-party pinholes.

That's the only safe thing to recommend.  Today, changing the
IP address on a server is an involved process (reduce DNS TTL,
bring it up on the new address and keep the old address active,
change DNS, wait until TTL*some_safety_multiplier, decommission
old address).  None of that is described in an RFC, it's just
common practice (documented elsewhere).  I don't see a need to
get into deep levels of education about changing client IP 
addresses and making that work nicely with PCP.  Besides, client
IP addresses don't change that often or, if they are being changed,
it's by a malicious ISP that is trying to interfere with their
subscribers running a server -- exactly the sort of ISP that
won't be much interested in PCP.

...
> Med: I suggest you add an issue into the tracker to record it.

http://trac.tools.ietf.org/wg/pcp/trac/ticket/34, as an enhancement.

> Med: FWIW, the logic of setting the Epoch value when enforcing CLI is
> described in: http://www.ietf.org/mail-
> archive/web/pcp/current/msg00091.html
> I'm not sure Epoch is the more efficient per-subscriber hint to audit
> the mappings; COUNT of the mappings can be considered too.

A count of the number of mappings only works if:

  1. there is one PCP client per host, which requires the PCP 
     client be part of the OS rather than part of the application.
  2. there is some sort of application-id stored on the PCP server,
     and counts are returned only for matching application-id.

If there is an application-id maintained for every PCP mapping,
we might as well have separate Epoch for every subscriber for
this case, as separate Epoch is fewer bytes on the server than 
an application-id for every mapping.

> Med: Yes: some CPEs embedding a PCP Client/IWF.

Please say more.

-d

---

From: <[email-masked]>
Date: Wed, 19 Jan 2011 13:38:52 +0100

Hi Dan,

Please see inline.

Cheers,
Med 

-----Message d'origine-----
De : Dan Wing [mailto:[email-masked]] 
Envoyé : mercredi 12 janvier 2011 18:02
À : BOUCADAIR Mohamed OLNC/NAD/TIP; [email-masked]
Objet : RE: [pcp] PCP failure scenarios & PCP state synchronization procedure

...

Med: This is a sensitive point for the CGN: port quota need to be carefully taken into account; otherwise the service will be impacted because of stale mappings. 

Med: See my comment about the port quota considerations.

Med: imho, we need to analyse further the options we have on the table including the ideas you list below.

Med: Which information the PCP Server will use assess 3rd party mapping? Compare the internal IP address vs. source IP address? 

> addresses if PCP is used to configure third-party pinholes.

That's the only safe thing to recommend.

Med: This can be added to the I-D.

  Today, changing the
addresses and making that work nicely with PCP.  

Med: Ok.

Besides, client

Med: See your point but this section is more about the change of the internal IP address.

...

Med: Thanks.

Med: I agree.

Please say more.

Med: See http://www.ietf.org/mail-archive/web/pcp/current/msg00626.html ;-)

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 19 Jan 2011 09:32:27 -0800

> Med: This is a sensitive point for the CGN: port quota need to be
> carefully taken into account; otherwise the service will be impacted
> because of stale mappings.

Ok, I added the following to -03's security considerations:

   Because of the state created in a NAPT or firewall, it is anticipated
   that port forwarding (PIN OpCodes) will have a quota applied to each
   subscriber.  If the quota is small and the maximum lifetime is large,
   a faulty or disconnected PCP client could cause a denial of service
   for other PCP clients belonging to that same subscriber.  To prevent
   this problem, if a PCP server is configured for a small per-subscriber 
   quota (e.g., less than 15 ports) then it is RECOMMENDED it also be be
   configured for a short maximum lifetime (e.g., 5 minutes).

...
> Med: Which information the PCP Server will use assess 3rd party
> mapping? Compare the internal IP address vs. source IP address?

Correct.

> Med: See http://www.ietf.org/mail-archive/web/pcp/current/msg00626.html

If the interworking function has no stable storage, I don't see how
it can function as a UPnP IGD device at all.

-d

---

From: <[email-masked]>
Date: Thu, 20 Jan 2011 08:01:44 +0100

Re-,

Please see inline.

Cheers,
Med 

-----Message d'origine-----
De : Dan Wing [mailto:[email-masked]] 
Envoyé : mercredi 19 janvier 2011 18:32
À : BOUCADAIR Mohamed OLNC/NAD/TIP; [email-masked]
Objet : RE: [pcp] PCP failure scenarios & PCP state synchronization procedure

Med: This is one of the mitigation actions we can recommend; a more concrete one is to allow the PCP Client to audit the mappings it instantiated in the PCP Server.

...

Correct.

Med: Dan, as a reminder we have wrote this in http://tools.ietf.org/html/draft-bpw-pcp-upnp-igd-interworking-01; 

"5.4. Port Mapping Tables

   IGD-PCP Interworking Function MUST store locally all the mappings
   instantiated by internal UPnP Control Points in the PCP Server.  Port
   Forwarding mappings SHOULD be stored in a permanent storage.  If not,
   upon reset or reboot, the IGD-PCP Interworking Function MUST
   synchronise its states as specified in Section 5.10."

We converged to this text instead of mandating permanent storage which is not met by low-end CPEs.

Reinaldo raised at that time "This is an extremely difficult requirement. Today most (if not all) CPEs only store in permanent storage manual configured mappings."

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 20 Jan 2011 08:34:08 -0800

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]
> Sent: Wednesday, January 19, 2011 11:02 PM
> To: Dan Wing; [email-masked]
> Cc: 'Reinaldo Penno'
> Subject: RE: [pcp] PCP failure scenarios & PCP state synchronization
> procedure
...
> Med: Dan, as a reminder we have wrote this in
> http://tools.ietf.org/html/draft-bpw-pcp-upnp-igd-interworking-01;
> 
> Reinaldo raised at that time "This is an extremely difficult
> requirement. Today most (if not all) CPEs only store in permanent
> storage manual configured mappings."

Let's consider today's UPnP IGD devices -- which (of course) do
nothing with PCP.  Those devices, today, also do not store their
UPnP IGD-created mappings in non-volatile RAM.  Thus, when they
reboot, they forget about their UPnP IGD-created mappings.

I don't see that PCP needs to take the burden to improve 
UPnP IGD.

-d

---

From: Simon Perreault <[email-masked]>
Date: Thu, 20 Jan 2011 12:12:33 -0500

> I don't see that PCP needs to take the burden to improve 

Agreed. I think implementations of PCP should be free to choose whether
they want to store mappings in permanent storage or not.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: <[email-masked]>
Date: Tue, 1 Feb 2011 08:34:14 +0100

Dear Simon,

In fact, we have proposed in http://tools.ietf.org/html/draft-boucadair-pcp-failure-00 the GET and GETNEXT methods to sync the state between the PCP Client and PCP Server because we think not all CPEs will be able to store the mappings in a permanent storage.

State reliability is important imho to consider. This is not considered as it should be in the current base pcp spec.

Cheers,
Med 

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de Simon Perreault
Envoyé : jeudi 20 janvier 2011 18:13
À : [email-masked]
Objet : Re: [pcp] UPnP IGD interworking [was RE: PCP failure scenarios & PCP state synchronization procedure]

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca
_______________________________________________
pcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcp