Subject: [OPSAWG] draft-janapath-intarea-traceflow-00.txt

From: Balaji Venkat Venkataswami <[email-masked]>
Date: Tue, 7 Feb 2012 15:11:52 +0530

Hi ron, Melinda,
In summary here is a list of things that the draft solves...

a) The aim is to trace the path through the network for a specific flow, through the member links for Multipathing (LAG at L2 and ECMP at L3). Hence may need a more generic processing at intermediate nodes, which is not currently part of ICMP mechanisms. The packet's payload contains the flow information which is then inspected and a packet is built as it would have ingressed the device and the said packet is sent to the hardware. The said packet is then trapped prior to actual forwarding and the forwarding result from the hardware analyzed. This is not done in ICMP and a more control planish approach is taken to send the response back to the Originator.

  The incoming packet has a payload which is the actual packet to be traced, and the results are for this payload, rather than the incoming packet itself, and this would need the receiver to parse the payload and then try to check the hardware for the embedded flow and calculate the egress port details.
b) It is possible in this draft to have the response sent back to a node different from the one that originated the request. This to some extent allows us to trace a specific direction without depending on the reverse path, though it now depends on the path to the management station to be up and working. Please see the Indirect method in the draft for more details.
c) Fan out as an option, where the result would be to indicate all the available paths to the destination and also to trace all the paths available to the destination This would be very much useful for an NMS collecting details as it would see the entire fan-out rather than just a limited picture which is what the 2 RFCs provide.
In general the requirement is similar to the reason why we tend to use hping over ping, it allows us to test all of the possible paths, this is essentially the traceroute compliment, once we've identified that there is a problem, we need to find it's source, today there's nothing that can do this when ECMP/LAG is added to the mix.
d) calculate more information about the paths and the status of the egress port
  While it is possible that all of these can be added as additional objects in ICMP, we feel that a new application would be a better choice
1) Routers which are unaware of these should be able to forward it as regular data packet. As we send the original packet in the payload only those devices that recognize Traceflow packets will evaluate the payload and send to the Originator or the Indirect Originator the set of details required. Also we catch devices in the intermediate stages which do not support this feature.
2) The processing that is required is significantly different from regular ICMP and hence a new application may be a better choice
3) The mechanisms to trap the packet to the CPU is not the standard set of ICMP error messages

Ron's Comments :

This draft appears to be an attempt to squeeze more information out of traceroute.  Could you tell me what information you are probing for, beyond that which can already be obtained using ICMP extensions defined in RFCs 4950 and 5837?

Why is a new approach required? If additional information is required, why not extend the ICMP response, as  RFCs 4950 and 5837 did?

<RESPONSE> With respect to the draft, it is a more natural reflection of the decisions made on the forwarding plane (after consultation of the hardware tables take place) that is sought to be highlighted. When a packet comes into a device, the normal thing we would do in the case of an ICMP response is to ignore the hardware decision to be taken and take a more control planish approach to the packet and find out which interface it was to go out on through mimicking the hardware which might not be essentially the correct result. With respect to getting both a LAG member on which the packet is to be sent on or a path in the set of links that constitute an ECMP set of paths this packet which is sought to be sent out is not thrust into the hardware to make out the decision taken (If at all the packet is to be forwarded or not) and the exact LAG member or ECMP member on which it is to be sent.
Another aspect is the fan-out option which is missing in the ICMP extensions. Considering that ICMP messages are filtered a fan-out may not be implementable in the current scheme of things if ICMP messages are turned off or are filtered with respect to their forwarding.
Aspects like the Utilization anomaly TLV and the extensions to the result TLV, ACL decisions as a result of certain policies etc... are not taken into account in the RFCs quoted. Also the original packet which is sought to be forwarded is carried in the payload and decisions taken as per that packet than the outer header packet which is sought to be taken into account in the regular TRACEROUTE case.
Also aspects like the Hash algorithm to be used and the fields taken into account into this hash algorithm are not carried back to the Originator of say a TRACEROUTE. As melinda suggested the path coupled properties are the highlight of this draft. ICMP extensions are not path coupled as the story goes. We agree that the ICMP extensions in the second RFC quoted (5837) do have information that necessarily coincides with the information w.r.t LAG and ECMP. But the actual hardware plane is not tested with respect to the TRACEROUTE. It is thus a more control plane mimic of the hardware which may not be accurate for all practical purposes.

Melinda's comments :

Speaking as an individual contributor and as someone who's worked on what are essentially path-coupled diagnostics, ICMP is probably not a great approach at all, given our frankly limited ability to secure both requests and responses, complete inability to route messages (it may be desirable to pin the reverse path to the forward path when sending diagnostics) and a whole host of other limitations.  Using ICMP for this sort of thing is a quick hack, but I don't think it's very robust as a general-purpose diagnostic transport.  Which, of course, raises the question of whether or not a general-purpose diagnostic transport is what's wanted in the first place.

<RESPONSE> We agree. As far as the general purpose diagnostic transport is concerned it is kind of out of scope of this document.

It seems that two sets of tools are required. The first set of tools offers a general view of the device. These tools give operators access to fibs, ribs, mibs and everything else you can think of. CLI, NETCONF and SNMP occupy this space. Those protocols already offer decent authentication capabilities.

> The seconds set of tools give operators an opportunity to directly observe forwarding behavior. Ping and traceroute fit the bill here. Because these tools offer relatively little information, they can be secured on an all-or-none bases (either you filter selected ICMP messages or you don't).

> Am I missing a third set of tools?

Maybe.  Traceroute is an interesting case because you're inferring router behavior or path characteristics from observed side-effects. When something goes wrong you really don't know why.  If there's an asymmetry in the path it's very possible for everything to be copacetic in one direction but not in the other, and it's sometimes extremely difficult to differentiate the two.

<RESPONSE> Copacetic means in excellent order. This kind of differentiation is currently NOT possible and would be out of scope on this document.

But aside from that we were interested in the ability to trigger diagnostics and provide some flexibility in what was returned.

> My question remains: What information is draft-janpapath trying to obtain that isn't accessible already?

It looks like they're interested in some of the same things we were. A packet goes into a network and disappears.  What happened?  Quite often it's the result of some sort of policy decision inside a router, or, heaven help us, firewall.  As ACLs get increasingly hairy (research results show that literally 99% of firewall rulesets contain errors) it would be excellent to have a tool that shows you what rules were triggered at which device in a network.  Better still is to have a reasonably general (but not unreasonably general) tool to trigger or collect diagnostics in a network.

<RESPONSE> Agree.

I don't love this draft (I think it's excessively blithe about transport) but I think it represents a real problem, and one for which we've seen interest from the community.

<RESPONSE> How do we get to a less blithiness on the transport ? Could you please elucidate this ?
thanks and regards,
Jana and team