Subject: [Anima] Is this how BRSKI/IPIP works?

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Jul 2017 14:19:23 +1200

Hi BRSKI authors,

Is the following correct?

Topology (ASCII art):
                   ___________
                  | REGISTRAR |
                  |___________|
                        |Ar
                        | 
                   ...........
                  (    ACP    )
                 (   routing   )
                  (   cloud   )
                   ...........
                        |
                        |Ax
                   _____|_____
                  |   PROXY   |
                  |___________|
                   |Lx1      |Lx2 
                   |         |
                   |         |
  -------LAN1---------      -------LAN2----------
      |                                     |
      |Lp                                   |Lp
  ____|____                              ___|_____ 
 | PLEDGE1 |                            | PLEDGE2 |
 |_________|                            |_________| 

Assumptions:

Pledges have link-local address Lp. By chance, they are equal. (Nothing in
the standards prevents them from being equal. Even pseudo-random numbers can
be equal, so this case must work.)

Proxy has link-local addresses Lx1, Lx2 and ACP address Ax. We can require
that Lx1 != Lx2.

Registrar has ACP address Ar.

Packets for a UDP example:

(somewhat simplified IPv6 packets!)

Pledge sends to proxy [Lp, Lx1, 17, UDP-PAYLOAD1]

Proxy sends to Registrar [Ax, Ar, 41, [Lp, Lx1, 17, UDP-PAYLOAD1]]

Registrar replies to proxy [Ar, Ax, 41, [Lx1, Lp, 17, UDP-PAYLOAD2]]

Proxy replies to pledge [Lx1, Lp, 17, UDP-PAYLOAD2]

Note that the registrar echoes back the addresses Lp and Lx but they mean
nothing to it. The registrar simply borrows the proxy's LL address Lx
for the purpose of replying.

Note that even the 2uple {Ax, Lp} might not uniquely identify the pledge.
Since the proxy will have at least two interfaces, the address Lp might
exist on multiple LANs. However, the proxy will have different link-local
addresses on the two LANs, so the 3uples {Ax, Lp, Lx1} {Ax, Lp, Lx2}
will be unique. Hence the registrar can distinguish the transactions.

So, what the registrar needs to tell the proxy is: I accept IP in IP on address Ar.
Nothing else - no port number, no link-local address.

What the proxy needs to tell the pledge is: I accept BRSKI/TCP
or BRSKI/UDP on address Lx. And if it chooses to use IPIP to contact
the registrar, it simply forwards the packets as-is in both directions,
encapsulating and decapsulating accordingly. The pledge knows nothing about
IPIP.

Regards
   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 6 Jul 2017 05:37:19 +0200

> Hi BRSKI authors,

Can i still answer ?

Inline. I only have an ACP author, WG chair and general bloviator hat though...

> Is the following correct?

No, i think we can not. AFAIK, it is common practice to put
your MAC address as the host part into link-local addresses
and only high-margin network equipment vendors
afford ranges of in my experience at most up to 8 MAC
addresses to a single device. And you do not want to 
make a protocol that changes any current possible
and likely practices. 

I have not found evidence of being able to have multiple link-local
addresses on an interface. But even if that was architecturally permitted,
it too would likely not something you could easily do through
a non-privileged app via UDP socket API (i fear). This concern may be
bogus because the registrar would already need to deal with IPinIP,
which is no fun for a non-privileged app without OS support.

> for the purpose of replying.

Not quite "means nothing to it".
The registar will have to build connection state and the connection key is

[ Remote_vIP=(Lp, Lxi, Ax), Local_IP=(Ar), Proto=17, RemotePort, LocalPort ]

Given how my claim is that multiple Lxi may be the same AND that as
you already said, Lp may be the same, we need another field to
distinguish those connections.

C.1 of BRSKI suggests to allocate more addresses, eg: multiple Axi.
But given how ACP addresses are backed by certificate, its not as 
easy to get more ACP addresses as it would be in less secured
transport substrates.

Solutions:

I would change the encap from IPinIP to some UDP header variation:
I want to be able to implement pledge, proxy and registar as
simple apps using just UDP sockets. Thats something i can implement
anywhere i want. 

Unless someone comes up with some pre-existing encap that makes
life easy, i would just make the proxy insert a link-local
pseudo header between the UDP header and the original pledges UDP payload.
pseudo header would need to contain (Lp). Then in addition,
the proxy would need to open a separate UDP socket for each local
interface it has. That would make all UDP packet from proxy use a separate
RemotPort on the registrar. The benefit of this approach is that i could
start separate ASA, one for each interface, and if one interfaces proxy
is attacked by pledges, it will not have an impact on other interfaces.
And i minimize unnecessary headers.

Relevant connection key is then:

[ Remote_vIP=(Lp, Ax), Local_IP=(Ar), Proto=17, RemotePort, LocalPort ]

Aka: Lxi is irrelevant and can be ignored by registrar.

Given how this is not a throughput relevant proxy function i do not
care avbout the fact that i must recalculate UDP checksums over the
packet (something that has bothered UDP tunneling solutions in the IETF
for years now).

Cheers
    Toerless

> _______________________________________________
> Anima mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/anima

-- 
---
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Jul 2017 16:34:05 +1200

> your MAC address as the host part into link-local addresses

It used to be, but the recommendation today is a pseudo-random
value (RFC7217). In any case it's a software choice.

> make a protocol that changes any current possible

We're talking about different physical interfaces. Normally they
will have different MAC addresses, if they still use the old-fashioned
method, or different pseudo-randoms if they follow RFC7217.
In fact, the only way they could have the same LL address is
by manual configuration. So I stand by what I said: we can require
them to be different, and in practice they will be different anyway,
on all conforming IPv6 stacks.

> addresses on an interface. 

No, but that is not the scenario. My diagram shows two different interfaces.

> But even if that was architecturally permitted,

Right, but it isn't used as an IP address on a real interface.

> distinguish those connections.

If that was a real problem, which I don't think it is, we would have to
include the relevant interface number as used inside the proxy's stack.
(There's a horrible trick used only in the FreeBSD stack to do this, by using
some of the spare bits in a link local address for this purpose.
https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ipv6.html#ipv6-scope-index )
 
> I would change the encap from IPinIP to some UDP header variation:

Yes, while I was studying this I wondered why not just use IP-in-UDP.
It still allows the proxy to be dumb.

I'd like to hear from Michael Richardson on all this. Again, my goal
is to understand enough that we can get the representation in the GRASP
objectives right.

    Brian

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 6 Jul 2017 09:09:38 +0200

> value (RFC7217). In any case it's a software choice.

brand new recommendations do not equate to be expected
standard practice in products. Would be very good to have
folks with practical insight into various products to 
provide more information.

> method, or different pseudo-randoms if they follow RFC7217.

If you have an L3 switch with 48 ports, likelyhood that it will
not have 48 different MAC addresses is very high.

> by manual configuration.

I do not believe that to be true from past product experience.

> So I stand by what I said: we can require

> No, but that is not the scenario. My diagram shows two different interfaces.

I just meant that a second link-local address could solve the
problem if multiple interfaces have the same link-local address.

We should move this discussion to v6ops to get feedback from folks
with more insight - once we get a better understanding why some
UDP encap would not be the more logical option. Just because i may
not have link-local address issues does not mean that IPinIP gives
me the degree of lightweight impleemntation options i would like to
have (at app level).

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 7 Jul 2017 08:32:03 +1200

> provide more information.

Indeed. But the general-purpose o/s stacks (I mean Linux,
MacOS and Windows) have been using pseudo-random interface
IDs for several years, since long before RFC7217. If you're
talking about L2/L3 switches, I have no idea.

> not have 48 different MAC addresses is very high.

Yes, because it is *also* an L2 switch, a.k.a. a bridge, so naturally
it appears as a single L2 device. So this depends on its internal systems
model. This is the kind of complexity you get with layer violations,
and an L2/L3 switch is the king of all layer violations.
 
> I do not believe that to be true from past product experience.

I think the confusion is that because of the L2 bridge function,
all those 48 interfaces behave as a single interface viewed from L3,
and therefore naturally have the same L3 address. So if the L3 code
in the switch itself also sees them as a single interface, we have
no problem. If the L3 code can see 48 separate interfaces, we do
have a problem.

(There is a complication in the switch caused by optimisation for
multicast, because MLD snooping has to look at both L2 and L3
headers. But BRSKI proxying only has to look at L3.)

(There is more complication potentially caused by VLANs.)

> problem if multiple interfaces have the same link-local address.

s/multiple interfaces/multiple L3 interfaces/

> me the degree of lightweight impleemntation options i would like to

Agreed. IMNSHO this is not cooked yet and should not be mentioned
in a standards-track draft unless we want a very long discussion
with the IESG.

    Brian

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 7 Jul 2017 01:14:00 +0200

Ok, sent mail to v6ops, not cc'ed to anima (though shall not group-cross-post IETF policy *sigh*).

More inline.

> talking about L2/L3 switches, I have no idea.

Yeah. Especially MacOS and Windows are widely deployed router OSs ;-))

> and an L2/L3 switch is the king of all layer violations.

Its got less to do with layer violation and L2 switches but rather with
the cost of a large number of MAC addresses. Those have to be paid for.
Primarily so that people do not exhaust them too quickly. Like when
you would uhmm... assign a separate MAC address to every bloody L3
network devices subnet ;-)

> headers. But BRSKI proxying only has to look at L3.)

Only stupid MLD snooping needs to look at L2, but most MLD snooping is stupid ;-)

> (There is more complication potentially caused by VLANs.)

Indeed, i forgot. By default, multiple L3 subnets on a single physcial
ethernet will very often get the same MAC address. And it's quite
common for a router to just have such a trunk L3 interface.

> s/multiple interfaces/multiple L3 interfaces/

"subnets" in IETF lingo ;-) or "read what i mean" ;-)

> in a standards-track draft unless we want a very long discussion

Yes

Toerless

-- 
---
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 8 Jul 2017 14:00:29 +1200

And for those who aren't on v6ops, the answer is clear:

Although for classical host o/s I'm correct, and the norm is
to have a different link-local address on each interface,
a) this is not required by the IPv6 standards, and
b) on L2/L3 switches from major manfacturers, it is common
that many interfaces have the same link-local addresses.

Which means that my diagram should be:

                   ___________
                  | REGISTRAR |
                  |___________|
                        |Ar
                        | 
                   ...........
                  (    ACP    )
                 (   routing   )
                  (   cloud   )
                   ...........
                        |
                        |Ax
                   _____|_____
                  |   PROXY   |
                  |___________|
                   |Lx       |Lx
                   |         |
                   |         |

That means that packets from PLEDGE1 and PLEDGE2 to the proxy
could have 100% identical addresses. So straightforward IP-in-IP
to the registrar will not work, because the registrar would
have no way to distinguish them, and the proxy would have
no way to distinguish the replies. Maybe the port numbers
offer a solution but they are in the embedded payload.

Michael R, help!

Regards
   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 8 Jul 2017 04:42:59 +0200

Just in case someone missed my first 10 opinions:
UDP encap. Also allows to build pledge,proxy.registrar as normal socket apps.

-- 
---
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Tue, 11 Jul 2017 20:44:31 -0400

> Topology (ASCII art):

Topology is essentially correct.
As you point out, RFC7217 is the recommendation going forward, so having
a a big IEEE OUI allocation isn't necessary anymore.

But, the problem is you have a single Ax for the device.
The ACP needs to allocate Ax1 for LAN1, and Ax2 for LAN2, etc. That's why I
wanted a /96 or so provided by the ACP to each device.

So it becomes:

> Proxy sends to Registrar [Ax1, Ar, 41,   [Lp, Lx1, 17, UDP-PAYLOAD1]]
> Registrar replies to proxy [Ar, Ax1, 41, [Lx1, Lp, 17, UDP-PAYLOAD2]]

Toerless says that a non-priveledged process (the proxy) can't easily
configure additional LL addresses.  That's true. It also can't configure the
addresses for the ACP.  The ACP needs to do that.

The Lx1 and Lx2 could be identical, although I'd not want to design my device
that way.  I looked at the CDP/LLDP spec, and both are unclear what the
source L2 address is.

    > So, what the registrar needs to tell the proxy is: I accept IP in IP on
    > address Ar.  Nothing else - no port number, no link-local address.

There is a small problems with this.  With a UDP transport, we simply
have to arrange for the registrar to accept traffic to any LxX IP address.
That's not stock POSIX, but it's not that hard.  LxX state can be handled
by the application.  With TCP the kernel has be rather flexible, being
able to keep duplicate Lp<->Lx1 connections seperate in the kernel, and
at the same time, permitting any LxX on the Registrar's side.

Instead, I have two suggestions, not entirely mutually exclusive:
  1) the Registrar says, "I accept IPIP on Address Ar, use Lr for connections"
  2) we make Lr = well known Link-Local anycast address

In my implementation, I dynamically set up an IPIP interface for each Lx
on each proxy that appears.  The kernel assigns a new ifindex to each of
these interfaces, and the normal LL-requires-ifindex rules apply to
distinguish things.  This requires a retransmission since the first time
there is a packet from a new Ax1/LAN1, the packet does not match any current
IPIP tunnel, and is dropped by the kernel.  A process watches for these
and configures them LRU.

    > What the proxy needs to tell the pledge is: I accept BRSKI/TCP or
    > BRSKI/UDP on address Lx. And if it chooses to use IPIP to contact the
    > registrar, it simply forwards the packets as-is in both directions,
    > encapsulating and decapsulating accordingly. The pledge knows nothing
    > about IPIP.

It needs to know it's IPIP, but I think you mean, it knows nothing about
what's inside the inner IP header.   There is some work (which I've yet to
complete) to arrange to forward LL-IP packets coming out of the IPIP
interface to the physical interface, since LL packets are normally never
forwarded.  Really, this is a kind of bridge (that's what I'll tell the
v6-police).

As for Toerless' notion that we should invent a new UDP-based encapsulation
rather than use the well defined IPIP encapsulation, I have really no comment.
I'm pretty sure that many will want to leverage existing v6-extension header
chasing hardware for the purpose of auditing, which is why I prefer not
to invent new on-the-wire formats just to so that some software engineer can
avoid having to learn a new API call.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 12 Jul 2017 14:02:44 +1200

On 12/07/2017 12:44, Michael Richardson wrote:
> The ACP needs to allocate Ax1 for LAN1, and Ax2 for LAN2, etc.

Yes, that would disambiguate it. But in general it isn't necessary for
a node to have multiple ACP addresses just because it has multiple
interfaces, is it?

> That's why I

That would be one way, but it will create noise in 6man.
In any case if the requirement is one address per proxy+interface
I'm sure it can be done somehow; IPv6 addresses are cheap.

>   1) the Registrar says, "I accept IPIP on Address Ar, use Lr for connections"

I don't understand where Lr would be used. The LL messages are
all between the pledge and the proxy, which have perfectly fine
LL addresses of their own.

> It needs to know it's IPIP,

Why? The IPIP encapsulation and decapsulation can happen inside the proxy
(and the registrar). Why does the pledge care?

> but I think you mean, it knows nothing about

IPv6-over-UDP isn't exactly a new invention. It's been used in Teredo,
TSP (RFC5572) and SixXs (https://tools.ietf.org/html/draft-massar-v6ops-ayiya-02).
More to the point, draft-ietf-intarea-gue is in progress.

All the same, if straight IPIP works, so much the better.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Wed, 12 Jul 2017 11:57:19 -0400

Brian E Carpenter <[email-masked]> wrote:
    >> Brian E Carpenter <[email-masked]> wrote: > Is the
    >> following correct?
    >>
    >> > Topology (ASCII art):
    >>
    >> Topology is essentially correct.  As you point out, RFC7217 is the
    >> recommendation going forward, so having a a big IEEE OUI allocation
    >> isn't necessary anymore.
    >>
    >> But, the problem is you have a single Ax for the device.  The ACP
    >> needs to allocate Ax1 for LAN1, and Ax2 for LAN2, etc.

    > a node to have multiple ACP addresses just because it has multiple

We need a way to distinguish the different links (and scoope of the LL
address) in a way that can be communicated easily and cheaply to the
Registrar.    I've asked for the "V"irtualization bit to be bigger, and I see
no purpose for the Zone-ID myself.

    >> That's why I wanted a /96 or so provided by the ACP to each device.

    > That would be one way, but it will create noise in 6man.  In any case
    > if the requirement is one address per proxy+interface I'm sure it can
    > be done somehow; IPv6 addresses are cheap.

It's not a subnet for the device to announce, it's a large bag of /128s.

    >> Instead, I have two suggestions, not entirely mutually exclusive: 1)
    >> the Registrar says, "I accept IPIP on Address Ar, use Lr for
    >> connections"

    > I don't understand where Lr would be used. The LL messages are all
    > between the pledge and the proxy, which have perfectly fine LL
    > addresses of their own.

Assume a TCP connection from the Pledge to the Registrar.
It looks like, as you wrote:

   Pledge sends to proxy [Lp, Lx1, 17, TCP-PAYLOAD1]

On the registrar, when it emerges from the IPIP tunnel, this is what the
registrar sees.   If we do nothing, the Registrar says, "Lx1"? What's that
it's not me. Drop.  So we have to do something. Options are:

1) Registrar accepts any Lx1 as local.
2) We pick a well-known Lx value (a LL anycast).
3) We have the Registrar tell the proxy an Lx value to use.

1) Registrar accepts any Lx1 as local.
   There is no precedent in v6 APIs to open such a socket, but this actually
   supported on many platforms.  It's used for nasty stuff like transparent
   application layer proxies, forced HTTP proxying, and the like.

2) We pick a well-known Lx value (a LL anycast).
   The pledge has to do something slightly funky where it forces a particular
   L2 address for the anycast LL, because if there are multiple proxies on
   the network, straight ND will get one at random, and the Pledge actually
   wants to be specific about which one it uses. (since it wants to try them all)

3) We have the Registrar tell the proxy an Lx value to use.
   I chose to put this option into the protocol, because we can always
   set Lx= Lanycast in the future, and perhaps we can set it to ::
   if we want case (1).

    >> > encapsulating and decapsulating accordingly. The pledge knows
    >> nothing > about IPIP.
    >>

    > Why? The IPIP encapsulation and decapsulation can happen inside the
    > proxy (and the registrar). Why does the pledge care?

Sorry. "Proxy" vs "Pledge"
"IT" above referred to Proxy, but in fact it says "pledge". Typo.

    > TSP (RFC5572) and SixXs
    > (https://tools.ietf.org/html/draft-massar-v6ops-ayiya-02).  More to the
    > point, draft-ietf-intarea-gue is in progress.

    > All the same, if straight IPIP works, so much the better.

I resorted to using Ar as an additional signaling mechanism to "store" the
ifindex where the request was coming from rather than try to find some bits
in some other protocol like one you mention above.
TSP is too complex for our needs.

Teredo has some space for additional stuff. It should be straight forward to
define it over IPv6 rather than v4. Seems a bit weird to me though.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 13 Jul 2017 08:55:02 +1200

OK, I'm getting there. More in line:

>    application layer proxies, forced HTTP proxying, and the like.

I think there's a more subtle way to look at it. When the registrar receives
a protocol 41 packet from a new ACP address, it conceptually synthesises
a new virtual interface and assigns Lx1 as its link local address. On
that interface, things would look normal. Thus RFC2473:

>>  The tunnel link-layer input and output can be implemented similar
>>  to the input and output of other link-layer protocols, for
>>  instance, associating an interface or pseudo-interface with the
>>  IPv6 tunnel.

I really think this is a clean approach. In any case, what happens to
a protocol 41 packet is always a bit strange and is not covered by
the traditional socket model.
>    wants to be specific about which one it uses. (since it wants to try them all)

How can it work as anycast if is more than one proxy on the LAN?
I'm not at all clear that anycast under fe80::/10 makes much sense.

>    set Lx= Lanycast in the future, and perhaps we can set it to ::

I like this least of all. What happens if there are multiple registrars?
And when a proxy node comes up as a pledge, it must give itself a LL address
on each interface before it even tries to perform its own BRSKI, and before
it looks for its own proxy and joins the ACP. Is it supposed to change its
own LL address after it discovers the registrar? Or is it supposed to
have two simultaneous LL addresses? Is that even possible?
 
> "IT" above referred to Proxy, but in fact it says "pledge". Typo.

OK!

> define it over IPv6 rather than v4. Seems a bit weird to me though.

Yes. My point was only that this isn't uncharted territory.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Thu, 13 Jul 2017 11:40:11 +0200

>> 1) Registrar accepts any Lx1 as local.  There is no precedent in v6
    >> APIs to open such a socket, but this actually supported on many
    >> platforms.  It's used for nasty stuff like transparent application
    >> layer proxies, forced HTTP proxying, and the like.

    > I think there's a more subtle way to look at it. When the registrar
    > receives a protocol 41 packet from a new ACP address, it conceptually
    > synthesises a new virtual interface and assigns Lx1 as its link local
    > address. On that interface, things would look normal. Thus RFC2473:

I can buy this.
It argues that the Proxy should send a gratuitous packet to the Registrar to
prime that virtual interface.  An ICMP echo request perhaps.

How can we document this well?

    >> 3) We have the Registrar tell the proxy an Lx value to use.  I chose
    >> to put this option into the protocol, because we can always set Lx=
    >> Lanycast in the future, and perhaps we can set it to :: if we want
    >> case (1).

    > I like this least of all. What happens if there are multiple
    > registrars?  And when a proxy node comes up as a pledge, it must give
    > itself a LL address on each interface before it even tries to perform
    > its own BRSKI, and before it looks for its own proxy and joins the

Yeah, you are right, this doesn't work if there are multiple registrars.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 14 Jul 2017 08:36:41 +1200

> prime that virtual interface.  An ICMP echo request perhaps.

Or a GRASP M_NOOP, designed for such purposes!
 
> How can we document this well?

I think it has to be spelled out almost at the pseudocode level. We had to
spell out the encap/decap behaviour for 6to4 in some detail, and that was
just about the only bit of 6to4 that never created trouble ;-). There
are various encap/decap specs of that kind, and the NAT64 stuff
also goes into horrible detail...

   Brian

---

From: Eliot Lear <[email-masked]>
Date: Thu, 13 Jul 2017 22:58:45 +0200

Hi Toerless,

> provide more information.
On this point, I think it's quite likely that we will see a good number
of devices fielded that will do a lousy job of PRNG, and so it would be
inadvisable for them to implement RFC7217, lest they test their DAD code
in ways not really intended.  I'm not thinking about iPhones here, but
energy harvesting devices like some light switches, and a bunch of,
well,... crap.

The question is whether you should design for these devices.  IMHO "no"
is a perfectly valid answer, but I'm still a bit skeptical about the
value of 7217 for these class of devices in any event.

Eliot

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 14 Jul 2017 10:41:35 +1200

> value of 7217 for these class of devices in any event.

That may be true, but for BRSKI as such, the only hard requirement is
an address that is unique on a given link, which is a requirement anyway.
IPIP is more of an issue for the node providing the proxy, which is
hopefully a bit upscale from a light switch.

    Brian

---

From: Eliot Lear <[email-masked]>
Date: Fri, 14 Jul 2017 08:13:27 +0200

Hi Brian,

> hopefully a bit upscale from a light switch.

I made my comment in the context of a possible interface collision in
your diagram.  Those had to do with the autonomic nodes, not the
proxies, as I understand things.  To avoid those sorts of collisions, it
seems like using the h/w address remains sensible.  A collision in those
circumstances would be extremely unlikely, whereas relying on poor PRNG
almost assures it of happening.  These devices are likely to have very
little entropy available to them.

Eliot

---

From: Michael Richardson <[email-masked]>
Date: Fri, 14 Jul 2017 15:09:18 +0200

> value of 7217 for these class of devices in any event.

1) Constrained devices are out of scope for ANIMA.
2) even if they were in scope, kinetic powered light switches are not
   good candidates for join proxies.  Light bulbs, however.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 15 Jul 2017 08:14:04 +1200

> 1) Constrained devices are out of scope for ANIMA.

True, but let's not use that as an excuse, because our stuff
might just get used more widely.

>    good candidates for join proxies.  Light bulbs, however.

But On 14/07/2017 18:13, Eliot Lear wrote:
...
> little entropy available to them.

And they may well be BRSKI pledges, just not using GRASP for discovery.
So Eliot's point seems valid (but not an issue for ANIMA alone).

    Brian

---

From: Michael Richardson <[email-masked]>
Date: Fri, 14 Jul 2017 23:44:42 +0200

Brian E Carpenter <[email-masked]> wrote:
    >> Brian E Carpenter <[email-masked]> wrote: > OK, I'm
    >> getting there. More in line:
    >> 
    >> >> APIs to open such a socket, but this actually supported on many >>
    >> platforms.  It's used for nasty stuff like transparent application >>
    >> > receives a protocol 41 packet from a new ACP address, it
    >> conceptually > synthesises a new virtual interface and assigns Lx1 as
    >> its link local > address. On that interface, things would look
    >> normal. Thus RFC2473:
    >> 
    >> I can buy this.  It argues that the Proxy should send a gratuitous
    >> packet to the Registrar to prime that virtual interface.  An ICMP echo
    >> request perhaps.

    > Or a GRASP M_NOOP, designed for such purposes!

I think that's also reasonable.  

    > I think it has to be spelled out almost at the pseudocode level. We had
    > to spell out the encap/decap behaviour for 6to4 in some detail, and
    > that was just about the only bit of 6to4 that never created trouble
    > ;-). There are various encap/decap specs of that kind, and the NAT64
    > stuff also goes into horrible detail...

okay.  Are you suggesting the 6to4 document should be looked at for style?

---

From: Michael Richardson <[email-masked]>
Date: Sat, 15 Jul 2017 01:01:47 +0200

Brian E Carpenter <[email-masked]> wrote:
    > But On 14/07/2017 18:13, Eliot Lear wrote: ...
    >> proxies, as I understand things.  To avoid those sorts of collisions,
    >> it seems like using the h/w address remains sensible.  A collision in
    >> those circumstances would be extremely unlikely, whereas relying on
    >> poor PRNG almost assures it of happening.  These devices are likely to
    >> have very little entropy available to them.

    > So Eliot's point seems valid (but not an issue for ANIMA alone).

7217 says:
RID = F(Prefix, Net_Iface, Network_ID, DAD_Counter, secret_key)

only the secret_key is really unique, and perhaps that's what you are
worrying about?

secret_key:
          A secret key that is not known by the attacker.  The secret
          key SHOULD be of at least 128 bits.  It MUST be initialized to
          a pseudo-random number (see [RFC4086] for randomness
          requirements for security) when the operating system is
          installed or when the IPv6 protocol stack is "bootstrapped"
          for the first time.

As the secret_key should be generated when the system is "installed"
or "first bootstrapped", I'm not sure the PRNG quality at runtime.
It seems to me like the secret_key should be set at manufacturer time
on the "bed-of-nails" or other JTAG point, at the same time when the
BRSKI IDevID and (perhaps) MASA anchors are loaded.   If those things
are in a TPM, then the secret_key could be there too.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 15 Jul 2017 14:08:27 +1200

> okay.  Are you suggesting the 6to4 document should be looked at for style?

Not especially. I'm more saying: if you can't write the equivalent of 
pseudocode, you can't expect other programmers to get it right.

Maybe a really good example of a painstaking encap description
is in IPsec: RFC4301 section 5.1.2 and its subsections. I don't think
we need to go that far, but we need to be 100% unambiguous.

All IMHO of course.

   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 16 Jul 2017 19:24:49 +0200

Sorry, cathing up late with the thread.

Thanks, Eliot. Thats good information. The MAC address based limited
link-local address space is a problem for devices running a proxy.
Do you have an idea about some class of devices that has the issue
that you describe and that could be proxies ?

I know about these crazy LED lightbulbs that actually build a mesh
network. Is that what you where alluding to ? 

But would those type of devices really be able to do all the
security stuff of ANIM/BRSKI ?

Cheers
    Toerless

---

From: Eliot Lear <[email-masked]>
Date: Sun, 16 Jul 2017 19:50:11 +0200

> that you describe and that could be proxies ?

Sure.  Just about any device that does a poor job of randomization or
have a low amount of entropy.  And that, I'm afraid, is a very large
swathe of stuff.  But again, I think the diagram Brian drew out
indicates the problem to be with autonomic node, not the border device,
and there the problem will be assuredly more pronounced.

> security stuff of ANIM/BRSKI ?

Good question.  I do think that lightbulbs are likely to do okay with
this stuff, but smaller devices will probably not, simply as a matter of
COGS.  There are different forms of sensor networks in which the devices
are highly constrained.  It may be possible to pre-store a certain
amount of entropy, which can ease some of this, but in those cases
developers will need to be economical.  The use of different forms of
interface addresses, including CGAs needs to take into account this
parameter.

Eliot

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 16 Jul 2017 20:27:49 +0200

I thought i had asked that question already but not sure, and not seen an answer:
- I have never seen that a device has more than one link-local addr on an interface.
  Is this permitted by IPv6 arck ? Can you configure this in eg: Linux. I thought i
  tried on linux/cisco-ios in the past and i do not quite remember, but i think it
  failed (only one address).

- Do we have another address assignment scheme other than a) MAC address based,
  or b) RFC7217   or c) non-randomn manual config. Could we get some scope
  relative address range for this purpose ?

More inline:

> at the same time, permitting any LxX on the Registrar's side.

How would this work, can you elaborate a bit ? 

Btw: Off the top of my head i think its a lot easier to forget kernel stuff and
use a registrar that maps received eg: UDP (or IPinIP) packets with encapsulated 
UDP/CoAP payloads than trying to persuade to build a virtual interface for each
association. When the payload becomes TCP/TLS/BRSKI, i do see more value in
trying to have the registrar app NOT have to deal with TCP.

> rather than use the well defined IPIP encapsulation, I have really no comment.

That easy to make you speech-less ? How bout writing a registrar on anything
else than Linux. Do you feel confident you can get what you need into any OS
kernel ?

Cheers
   Toerless

-- 
---
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Mon, 17 Jul 2017 08:34:01 +0200

Toerless Eckert <[email-masked]> wrote:
    > I thought i had asked that question already but not sure, and not seen
    > an answer: - I have never seen that a device has more than one
    > link-local addr on an interface.  Is this permitted by IPv6 arck ? Can
    > you configure this in eg: Linux. I thought i tried on linux/cisco-ios
    > in the past and i do not quite remember, but i think it failed (only
    > one address).

dooku-[~](2.3.0) mcr 10879 %sudo ip -6 addr add fe80::1234/64 dev wlan0

dooku-[~](2.3.0) mcr 10788 %ifconfig wlan0
wlan0     Link encap:Ethernet  HWaddr 08:11:96:01:81:e0
          inet addr:31.133.129.16  Bcast:31.133.143.255  Mask:255.255.240.0
          inet6 addr: fe80::1234/64 Scope:Link
          inet6 addr: fe80::a11:96ff:fe01:81e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

dooku-[~](2.3.0) mcr 10788 %ip -6 addr ls dev wlan0
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::1234/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::a11:96ff:fe01:81e0/64 scope link
       valid_lft forever preferred_lft forever

---

From: Eliot Lear <[email-masked]>
Date: Mon, 17 Jul 2017 08:54:46 +0200

On the other hand, maybe it's fundamental, but is relying on LL in this
architecture to go beyond LL boundaries the right thing to do?

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 17 Jul 2017 19:28:05 +1200

> architecture to go beyond LL boundaries the right thing to do?

What it would do is prolong the link, virtually, up to a dedicated virtual
interface in the registrar. So I think it's safe, if unconventional.

    Brian

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Jul 2017 09:38:59 +0200

Can you propose a stateless proxy model that would not pass the link-local
addresses on to the registrar and that uses Michaels beloved IPinIP encap ?

Alas i have fallen in love with UDP encap because i like to see more
networking software now be build like any othrer application on top of
UDP/TCP APIs and not mess around with OS kernel features, so all
my answers would be "the proxy is an app that either statefully proxies TCP
or stateless proxies UDP".

If you want to statelessly proxy TCP and on the registrar use some existing 
TCP stack, then i would begrudgingly agree with Michael, that i also need some
kerne level handling of the encap so that i get kernel level TCP.

I am still waiting for some better explanation from Michael about the
"Linux kernel and overlapping TCP" to fully understand his proposal.

So, here is one proposal for IPinIP using the current -07 draft ULA addressing:

 A) We assign one of the 8 (3 bit) "T"ype codes to "Subnet Identifier for Encap".

 B) The proxy allocates a separate "Zone" number for every subnet. Zone = Subnet.
    In result it now has for every subnet a separate ULA address for the IPinIP encap.

 C) The registrar announces its ability to support IPinIP BRSKI via GRASP

 D) Each ACP device need to use its ACP DeviceID also as the host-part of its
    link-local address.

 E) The proxy as part of its tunnel functionality also assigns itself the Registrar
    link local address on every subnet. At least logically. Whether it really needs
    to do this physcially is implementation specific.
 
 F) The proxy announces via GRASP BRSKI-IPIP with the Registrar Link-Local address
    (which it can do because according to E) it owns it on its subnets - GRASP
     DULL does not permit third-party announcements, so E) is to make it legal for
     the proxy to announce this in GRASP).
 
 G) Any packets sent by pledges to the Registrar link-local address are IPinIP
    encapsulated using the "Subnet Identifier for Encap" as the source IP address and
    the Registrar ULA as the destination address.
 
 H) The registrar can create a separate IPinIP tunnel per remote proxy, per-subnet-on-proxy.
    It does not need further addresses.

Some more details might be needed, eg:
 - If a proxy has more than 2^13 interfaces it needs to dynamically allocate
   subnet encap addresses.
 - A proxy might want to map different subnets to differen registrars for load balancing.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: "Eliot Lear (elear)" <[email-masked]>
Date: Mon, 17 Jul 2017 07:41:40 +0000

Sure. Use normal unicast addresses (ULA or other) if available. 

Eliot

> On Jul 17, 2017, at 9:39 AM, Toerless Eckert <[email-masked]> wrote:
>

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Jul 2017 10:02:13 +0200

Remember that the pledge can only have a link-local address during bootstrap,
so i would not know how to interpret your comment.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Mon, 17 Jul 2017 10:55:40 +0200

> architecture to go beyond LL boundaries the right thing to do?

We've already established a way around the concern that made me think
that we needed multiple LL for the proxy, and also that we needed multiple
Ar for the proxy connection.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Jul 2017 18:32:36 +0200

I think.... *righ*... eg: The only clean way with IPinIP is that every proxies
subnet can be distinguished on a registrar by the encapsulation header of IPinIP
and that only has source and destination ACP ULA addresses, yada yada: we need
either more ULA on the proxy and/or the registrar. 

I guess we do not want any on-demand actions on the proxy because that would be
a dynamic state establishment that we try to avoid, so the proxy would need
to set up all the state needed upfront which means it needs one IPinIP tunnel
per subinterface, just at the off-chance that some pledges have the same address.

So then we look at the total address waste and obviously we may have 1000 pledges
but 1 or few registrars, so it definitely would make most sense to get these addresses
on the registrar, announce via GRASP and let pleges build tunnels with them.

And the registrars would dynamically build these tunnels based on (Src,Dst) encap headers
received, and hopefully somebody comes up with a better way for linux to do this than
loosing the first packet which seems to be what Michael is running into.

And given how the tunnels ultimately are built only on a (per-proxy,interface)
basis there is also not a direct attack vector against dynamic creation based
on nasty pledges.

So, i am warming up more to this ;-)

Cheers
    Toerless

-- 
---
[email-masked]

---

From: "Max Pritikin (pritikin)" <[email-masked]>
Date: Mon, 24 Jul 2017 15:28:02 +0000

re: "this case must work”
There sure is a lot of complexity in this thread to ensure that link local addresses can be used outside of the local scope. 

Some simpler suggestions,

1) a stateful proxy. I know, I know. But how many devices are actually going to need to perform BRSKI at the same time? And if that many devices are being bootstrapped then can’t they wait for the proxy to work through them in order? 

2) Require pledges to complete RFC4862 ‘Creation of Global Addresses’. Pledges then perform GRASP and then communicate with the Registrar directly. (Can RFC4193 ‘Unique Local IPv6 Unicast Addresses’ be assigned? I think so…)

or 

3) If it *doesn’t work* what happens? Currently BRSKI starts over w/o stating anything about obtaining a new Lp. Would it help to restart the stack and look for a unique Lp at that point? All we’re looking for is non-collision during bootstrapping. I don’t like this approach as much as the above options. 

- max

> On Jul 5, 2017, at 8:19 PM, Brian E Carpenter <[email-masked]> wrote:
> 
> Hi BRSKI authors,
>