Subject: RE: [Fwd: RE: Router Alert based Monitoring]

From: "David Muldowney" <[email-masked]>
Date: Mon, 1 Mar 2010 17:25:12 -0000

Thanks for all your feedback which raises some of the concerns that we're
mostly aware of and we recognise that we need to address these.

Firstly, we should point out that at this mechanism is primarily for use by
network operators/administrators to collect information from their own
domain only and therefore it would be their responsibility to ensure that it
is used responsibility. For example, the arguments that apply for SNMP
Manager usage apply here also. Requests from outside the domain should be
not be permitted.

Secondly, the problem we're trying to address is as follows. From an
operators perspective, the market is seeing that revenues from providing
bandwidth are declining as operators are trying to adapt their business
models to grow revenue from premium services where "best-effort" delivery on
its own may not be good enough and service degradation is likely to result
in loss of revenue. Therefore, we see that an operator has a vested interest
to monitor the paths to its most valuable customers. Taking into account
projected growth in traffic, particular in video traffic (fixed/wireless
incl.4G). We see the need for operators to efficiently discover and collect
selective data from each node associated to the path quickly for rapid fault
identification and resolution. SNMP could be used to collect the same data,
however, associating managed object instance to a path is tricky at best
especially considering dynamic nature of paths even within the one
management domain. Again, taking into account traffic growth predictions we
see possible alternatives such as IPFIX and DPI as having potential
scalability challenges given large volumes of data collected and the
corresponding post-processing required.

This work is very much IPv6 focussed but may also be of interest to opswawg.

Regards,
David Muldowney

---------------------------- Original Message ----------------------------
Subject: RE: Router Alert based Monitoring
From:    "Frank Bulk" <[email-masked]>
Date:    Sat, February 27, 2010 5:48 pm
To:      "'Christopher Morrow'" <[email-masked]>
Cc:      [email-masked]
--------------------------------------------------------------------------

I'd rather not SNMP instrument the multiple in/out queues, status, load, etc
of each interface on each router in my network.  I'd rather have a tool that
when there is an issue, or systematically, lets me know what's going on at
each hop in the path of the traffic.  And since some paths are asymmetric
and many can change, router alert would provide a round-trip snapshot.

Perhaps there are other ways to deal with these things.

Frank

-----Original Message-----
From: Christopher Morrow [mailto:[email-masked]]
Sent: Saturday, February 27, 2010 11:33 AM
To: [email-masked]
Cc: Jason Schiller; Vishwas Manral; [email-masked]
Subject: Re: Router Alert based Monitoring

On Sat, Feb 27, 2010 at 12:03 PM, Frank Bulk <[email-masked]> wrote:
> Why not implement Router Alert with a "deny all" by default and require
the
> router engineer to configure ACLs for its use?  Doesn't that address the
> DoS/resource concern?

out of curiousity, what's the use case for this monitoring thing that
SNMP doesn't already get? How can I decide which thing is using
resources on the router/switch if I have routing-protocol-churn,
interface hellos, ssh/telnet management, snmp polling, normal ip error
handling.

Why do we need to add another monitoring thing here?

Inside a single ASN there's probably less of a concern for this,
provided you can drop this at the edges that aren't your
monitoring/management systems connections, since this could very well
span many ASN's between 'remote office #1' and 'remote office #2' the
folks doing the monitoring are monitoring something 'not theirs'. This
is a very bad thing...

As I said before... 'all things that depend upon/use router alert need
to die. we should never create another protocol/dependency upon this
bit.'

-chris

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Jason Schiller
> Sent: Friday, February 26, 2010 11:53 AM
> To: Vishwas Manral
> Cc: [email-masked]
> Subject: Re: Router Alert based Monitoring
>
> Alan,
>
> I agree with Chris and Vishwas.  Using router alert or other IP options
> create a DoS vector towards the network elements I operate.  If this was
> implemented I would certainly filter things outside my  network from using
> this monitoring.
>
> Internally, if we did use it, we would be very careful about restricting
> the number of addresses that could send such traffic, and limiting the
> amount of traffic they would generate.
>
> __Jason
>
>
> ==========================================================================
> Jason Schiller                                               (703)886.6648
> Senior Internet Network Engineer                         fax:(703)886.0512
> Public IP Global Network Engineering                       [email-masked]
> UUNET / Verizon                         [email-masked]
>
> The good news about having an email address that is twice as long is that
> it increases traffic on the Internet.
>
> On Fri, 26 Feb 2010, Vishwas Manral wrote:
>
> |Date: Fri, 26 Feb 2010 09:00:18 -0800
> |From: Vishwas Manral <[email-masked]>
> |To: [email-masked]
> |Cc: [email-masked]
> |Subject: Re: Router Alert based Monitoring
> |
> |Hi Alan,
> |
> |Based on working on protocols that use Router Alert (RSVP) esepcially for
> |IPv6 I have to say that Router Alert is not the best way forward.
> |
> |It has not been properly implemented in most OS network stacks. It also
> |causes security issues. Infact there was a proposal recently to reccomend
> |that Router Alert is not used in any new protocol, besides ones that
> already
> |do.
> |
> |Thanks,
> |Vishwas
> |On Fri, Feb 26, 2010 at 8:19 AM, Alan Davy <[email-masked]> wrote:
> |
> |> Hi All,
> |>
> |> Previously we circulated a proposal [Nov 4th 09] about defining a new
> IPv6
> |> Hop by Hop extension header whereby performance metrics can be
collected
> |> per node along a path. Based on feedback received from the list the
> |> recommendation was to consider utilising Router Alert as per RFC2113.
> |>
> |> After researching this mechanism we propose to define a protocol using
> |> Router Alert where a network operator can request a sequence of managed
> |> objects to be collected at each node along a specific path within an
> |> administrative domain augmenting existing protocols and practices so
that
> |> the managed objects can be quickly associated to a specific path for
> |> further diagnosis without the need for mining large volumes of data
using
> |> DPI, IPFIX, active or passive probes etc. This could be in response to
an
> |> alarm, customer initiated trouble ticket or periodic service assurance
> for
> |> premium services/customers. This mechanism would not be used to
calculate
> |> the latency on a path so fast processing is not requirement. One sample
> |> scenario of use would be to gather a report of the current queue
lengths
> |> at each interface that the packet traverses on the path.
> |>
> |> We would like to keep this as customizable as possible, so we are
> |> considering a solution whereby an operator can specify the managed
> objects
> |> by means of an Object Identifier (so any supported MIBs data could be
> |> collected). Each node would report both the OID instance and its value
as
> |> the packet traverses the path. The operator can then use this
information
> |> to
> |> perform more targeted diagnostics as required.
> |>
> |> The main technical challenge we forsee is the feasibility of
determining
> |> the
> |> OID instances and their value at each hop.
> |>
> |> We would like to know if such a monitoring protocol would be viewed as
> |> desirable and technically a reasonable approach.
> |>
> |> Best Regards,
> |>
> |> Alan Davy
> |>
> |> --------------------------------------------------------------------
> |> IETF IPv6 working group mailing list
> |> [email-masked]
> |> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> |> --------------------------------------------------------------------
> |>
> |
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 1 Mar 2010 13:18:12 -0500

> Manager usage apply here also. Requests from outside the domain should be

Great, what's the method proposed for a packet to get sent along this
path? router-alert means each hop will inspect (and potentially add
data/whatever)... So, if you start from inside the 'noc' sending a
packet to a remote destination host, collection along the path
includes all hops. Since the packet here isn't destined to a router
on-net you can't filter this with simple iACLs... Since it's not tied
to a service that requires validation of the endpoint you can spoof
the src. If the goal is to check the path along a known trajectory,
you'll really want to 'traceroute' from  some endpoint (user-host) to
another endpoint (user-host), if you don't there's no guarantee that
this test packet/stream follows the same path as the user traffic you
are attempting to model/measure.

This is a morass, and router-alert isn't making it any better...

> to monitor the paths to its most valuable customers. Taking into account

Sure, but this means monitoring some set of mpls (or other encap,
potentially in an alternate VRF/RIB even) paths, which means injecting
traffic onto that path from arbitrary points in the network. This
isn't something your standard deployed monitoring systems will be able
to do, this is something that edge router/switch devices will have to
do, and there will have to be some creation of packets at one side,
destined to real hosts on the far side (where the far side
router/switch doesn't actually deliver said packet).

If you don't model the packets (down to the ports/protocols/hosts
really) you aren't measuring anything of consequence to this problem
space.

This sounds like a lot more complexity and pain than current
operations methods/models... and I'm still going to get cycles stolen
from parts of my network by something I didn't plan for and am not
able to limit.

> scalability challenges given large volumes of data collected and the

is parsing some netflow really that hard?? seriously?? (it doesn't get
you end-to-end latency and loss, but plenty of sla systems do today)

Again, this seems like a hellaciously bad plan...  router-alert ==
bad, stealing cycles == bad.

-chris

---

From: "David Muldowney" <[email-masked]>
Date: Tue, 2 Mar 2010 16:23:31 -0000

> Sent: 01 March 2010 18:18
> To: David Muldowney
> Cc: [email-masked]
> Subject: Re: [Fwd: RE: Router Alert based Monitoring]
> 
> 
> data/whatever)... 

We are investigating an initial message from the noc to the source router in
the path. The payload will contain at least an destination router on the
path as well as the set of data that is to be collected. The source router
would then create a new message with the router alert option set and send
this along the path towards destination router. Each router in the path
would add the requested data to the payload and forward it. At the
destination router, a new message would be created and the collected result
reported back to the noc. That would be the basic operation and is stateless
unlike rsvp.

> So, if you start from inside the 'noc' sending a

We need to assess that even if a secure technical solution could be defined
to collect the data related to path in this way, then it must have a clearly
defined benefit to justify doing so. The goal is to facilitate gathering
info about the path for further diagnosis, rather than emulating the user
traffic itself.

> really) you aren't measuring anything of consequence to this problem

Yes we are aware that we can use probes in order to ensure user traffic
experience can be emulated. This is part of an overall solution and while
probes may confirm that there is a problem in the network, it won't tell us
why or where, so what we're investigating is a means to accomplish this. 

> from parts of my network by something I didn't plan for and am not

Yes I agree, so we must determine whether or not it would be worth it.

> you end-to-end latency and loss, but plenty of sla systems do today)

Again, this is part of the bigger solution. Its not that its not that its
not hard albeit limited in what you can determine, it’s the potential volume
of it that would be the issue, and the associated transfer to a central
location for aggregation to determine path-based metrics etc.

> bad, stealing cycles == bad.

Again I agree on both counts, but if we were to collect the same data from
the noc making SNMP requests at each node along the path would it be any
worse?  It would certainly result in more data  being transferred around the
network and there would be a lot more complexity at the noc as it would need
to determine the path in advance.

Thanks,
David
> (703)886.6648
> >> |> Administrative Requests:
> https://www.ietf.org/mailman/listinfo/ipv6