Subject: Re: UDP fragment packets

From: Kamlesh Shah <[email-masked]>
Date: Sun, 06 May 2001 22:22:01 -0700

Connie,

 From what I understand, NFC does take this into account since it is able to identify the three packets recevied as part of the same "flow".

What aggregation scheme did you use ?

Remember, in Cisco's NetFlow implementation, unless you are doing router based aggrgation, version 8, it is the NFC which is doing a lot of processsing and "aggregation" of flows..

At 03:24 PM 5/4/2001 -0700, Logg, Connie A. wrote:
>I am wondering if there is any concensus on how UDP fragment packets could be counted for flow purposes.
>
>I am anaylzing data from Cisco's Netflow feature which is generated by one of our Catalyst 6509s 
>with a MSFC in it.
>
>The majority of the UDP data is accounted in records with a source port and destination port of 0.
>
>In fact, these packets that are counted in these records are fragments of AFS flows, and I think they should be
>counted as AFS packets by the Netflow generation code.
>
>Here is a detailed decode of a series of packets (transferring a 5700 byte datagram) (hope I have the terminology
>correct)..  I had posed the question to Cisco of how these packets are handled, and the response I first
>got back is that "we do not understand why anyone would want to count fragment packets".  Such is life.
>
>I am wondering if you folks have an opinion on how these types of packets should be counted?
>
>Thanks, COnnie
>
>Connie Logg - Network Analyst - 650-926-2879
>Stanford Linear Accelerator Center
>MS 97; 2575 SandHill Road; Menlo Park CA 94025
>"Happiness is found along the way, not at the end of the road"
>
>##################################
>This example involves the transfer of 5700 bytes of data.  Since the mtu is only 1500, this data must be broken up into multiple packets (fragments of the data stream). There are 4 packets in this flow.  The first UDP packet contains the port numbers and first part of the data, and the following 3 UDP packets contain the rest of the data.  I want to have these counted, because they represent a large portion of the data.  The statistics from analyzing the netflow data will be grossly wrong if they are not since we have a lot of large data transfers. The identification field is the key to counting the packets correctly.  For the results that I see when I compare a packet capture with the netflow records, it appears that the last 3 packets are not counted.
>
>The last 3 fragment packets of the file transfer are indeed complete packets. The term "fragment" comes from the
>IP spec.  That is why I called them fragments.  If you look at the packet decodes I sent you will see the "fragment"
>bits.  I pointed to them.
>
>Packet 1:  has ports in it for level 3 decode
>---------
>
>----- Level # 1 is ETHERNET Offset: 0 Size: 14
> 
> Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
> Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
> Type           : 2048 (DoD IP)                    08 00              ..
>----- Level # 2 is DoD IP Offset: 14 Size: 20
> 
> Version        : 4                                 4                 .
> Header Length  : 20                                5                 .
> Type Of Service: 0                                00                 .
>                : 000..... Routine
>                : ...0.... Normal Delay
>                : ....0... Normal Throughput
>                : .....0.. Normal Reliability
> Total Length   : 1500                             05 dc              ..
> Identification : 2736                             0a b0              <---critical info      
> Flags          : 3                                 3                 .
>                : 0.. Reserved
>                : .1. Don't Fragment
>                : ..1 More Fragments                                  <--- note more fragments to come
> Fragment Offset: 0                                60 00              `.
> Time To Live   : 255                              ff                 .
> Protocol       : 17 (DODUDP)                      11                 .
> Header Checksum: 44704                            ae a0              ..
> Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
> Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
>----- Level # 3 is DoD UDP Offset: 34 Size: 8                         < note has ports in it
> 
> Source Port    : 7000 (AFS3-FILESERVER)           1b 58              .X
> Dest Port      : 7001 (AFS3-CALLBACK)             1b 59              .Y 
> Length         : 5700                             16 44              .D
> Checksum       : 23042                            5a 02              Z.
>----- Level # 7 is UserData Offset: 42 Size: 1476
> 
>  00032:                               a5 f5 c1 83 67 08             ....g.
>  00048: 90 3c 00 00 00 07 00 00 00 01 00 00 00 08 01 02   .<..............
>....
>
>Packet 2: UDP data packet with NO port info...just data
>---------
>
>Frame 49 Size  1518 Absolute Time Apr 26 15:41:35.401 ASCII MODE
>-------------------------------------------------------------------------------
>----- Level # 1 is ETHERNET Offset: 0 Size: 14
> 
> Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
> Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
> Type           : 2048 (DoD IP)                    08 00              ..
>----- Level # 2 is DoD IP Offset: 14 Size: 20
> 
> Version        : 4                                 4                 .
> Header Length  : 20                                5                 .
> Type Of Service: 0                                00                 .
>                : 000..... Routine
>                : ...0.... Normal Delay
>                : ....0... Normal Throughput
>                : .....0.. Normal Reliability
> Total Length   : 1500                             05 dc              ..
> Identification : 2736                             0a b0                <- critical info
> Flags          : 3                                 3                 .
>                : 0.. Reserved
>                : .1. Don't Fragment
>                : ..1 More Fragments                                    <--more fragments to come
> Fragment Offset: 1480                             60 b9              `.
> Time To Live   : 255                              ff                 .
> Protocol       : 17 (DODUDP)                      11                 .
> Header Checksum: 44519                            ad e7              ..
> Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
> Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
>----- Level # 7 is UserData Offset: 34 Size: 1484
> 
>  00032:       65 64 75 20 4e 4f 5f 55 4e 49 58 49 44 20     edu NO_UNIXID 
>
>
>Packet 3: Another fragment...udp data packet only
>---------
>
>Frame 50 Size  1518 Absolute Time Apr 26 15:41:35.402 ASCII MODE
>-------------------------------------------------------------------------------
>----- Level # 1 is ETHERNET Offset: 0 Size: 14
> 
> Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
> Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
> Type           : 2048 (DoD IP)                    08 00              ..
>----- Level # 2 is DoD IP Offset: 14 Size: 20
> 
> Version        : 4                                 4                 .
> Header Length  : 20                                5                 .
> Type Of Service: 0                                00                 .
>                : 000..... Routine
>                : ...0.... Normal Delay
>                : ....0... Normal Throughput
>                : .....0.. Normal Reliability
> Total Length   : 1500                             05 dc              ..
> Identification : 2736                             0a b0                  <--critical info
> Flags          : 3                                 3                 .
>                : 0.. Reserved
>                : .1. Don't Fragment
>                : ..1 More Fragments                                      <--------more fragments to come
> Fragment Offset: 2960                             61 72              ar
> Time To Live   : 255                              ff                 .
> Protocol       : 17 (DODUDP)                      11                 .
> Header Checksum: 44334                            ad 2e              ..
> Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
> Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
>----- Level # 7 is UserData Offset: 34 Size: 1484
> 
>  00032:       73 61 2e 69 6e 66 6e 2e 69 74 20 43 4f 53     sa.infn.it COS
>  00048: 54 41 4e 54 49 20 43 6f 73 74 61 6e 74 69 6e 69   TANTI Costantini
>
>
>Packet 4: Last UDP data packet
>---------
>
>Frame 51 Size  1298 Absolute Time Apr 26 15:41:35.402 ASCII MODE
>-------------------------------------------------------------------------------
>----- Level # 1 is ETHERNET Offset: 0 Size: 14
> 
> Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
> Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
> Type           : 2048 (DoD IP)                    08 00              ..
>----- Level # 2 is DoD IP Offset: 14 Size: 20
> 
> Version        : 4                                 4                 .
> Header Length  : 20                                5                 .
> Type Of Service: 0                                00                 .
>
>                : 000..... Routine
>                : ...0.... Normal Delay
>                : ....0... Normal Throughput
>                : .....0.. Normal Reliability
> Total Length   : 1280                             05 00              ..
> Identification : 2736                             0a b0                 <- critical info
> Flags          : 2                                 2                 .
>                : 0.. Reserved
>                : .1. Don't Fragment
>                : ..0 Last Fragment                                      < last fragment
> Fragment Offset: 344                              42 2b              B+
> Time To Live   : 255                              ff                 .
> Protocol       : 17 (DODUDP)                      11                 .
> Header Checksum: 52561                            cd 51              .Q
> Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
> Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
>----- Level # 7 is UserData Offset: 34 Size: 1264
> 
>  00032:       70 69 73 61 2e 69 6e 66 6e 2e 69 74 20 42     pisa.infn.it B
>
>#####################################
>
>
>Connie Logg - Network Analyst - 650-926-2879
>Stanford Linear Accelerator Center
>MS 97; 2575 SandHill Road; Menlo Park CA 94025
>"Happiness is found along the way, not at the end of the road" 

((((((((((((())))))))))))
Kamlesh Shah  CCIE # 2803
Technical Marketing Engineer - MPLS, ISBU

---

From: "Logg, Connie A." <[email-masked]>
Date: Mon, 07 May 2001 14:44:47 -0700

I am using the CALLREC aggregation scheme.

One day I spent a day capturing packets (spanning to a switch probe) and correlating the traffic to a particular server (it was its port that was spanned) to
the netflow records that were cut for the traffic.  There were a series of 7 AFS transfers between two nodes.
There were 3 udp fragment packets for each transfer, and the netflow record showed 20 for port 0(one was taken in by the MSFC).
There were 6 for the AFS first packets (one was taken in by the msfc). This took all day (actually a day and a half)
to find a consistent transfer that I could study.

Thus came my conclusion that the fragments were counted as port 0.  I replicated this several times that day.

Now unless netflow looks at the "Identification" field, it (I think) will have no way to know that the 4 packets are actually part of the
same flow. 

I have been hoping that someone knowledgable would look at the code in the switch which collects the stats, and
generates the flow records. 

It sounds like I may be wrong on my concept of the NFC (there is no thorough detailed documentation that I have been able to find).  I did not think (still do not) that it really does much work
except to aggregate the netflow records into files which represent the aggregation scheme desired.  Thus I am 
using the aggregation scheme CALLREC to give me all the records that are cut into one file (per 10 minutes).

I then process these records into my own "schemes" by examining the protocol field and source and destination
port fields. Thus, when I separate the data in this way, I get a lot of UDP port0 traffic, most of which is to/from
our AFS servers. Of course, I could create filters and my own aggregation schemes with the NFC (and I tried), but I
ran into problems that TAC could not solve, so I decided to do it myself.  That way I know precisely what heuristics
I am applying when deciding how to classify a flow record.

I appreciate any info that you can provide me on understanding this.

Thanks, 

Connie

-----Original Message-----
From: Kamlesh Shah [mailto:[email-masked]]
Sent: Sunday, May 06, 2001 10:22 PM
To: Logg, Connie A.; [email-masked]'
Subject: Re: UDP fragment packets

Connie,

---

From: Kamlesh Shah <[email-masked]>
Date: Mon, 07 May 2001 21:27:12 -0700

Connie,

I am teaching a class tomorrow and wednesday..I will certainly look into the details of this, including code if needed soon after that.

Please standby for more..

Regards.

---

From: Kamlesh Shah <[email-masked]>
Date: Mon, 14 May 2001 22:02:06 -0400

Connie,

I did some more digging into this issue. Sorry for taking long to get back to you, I am preparing for a trip right after teaching a class..but that's a different story :-)

What I found is that your observations about NFC NOT being able to do flow co-relation is correct in that it does not know if  the next 3 packets received are part of the same flow or not, since it is doing its work purely on the aggregation scheme fields, keys.

I also found that this is the case since call record aggregation scheme is simply counting {SA, SP, DA, DP} for address and port combination, it does not know when the port changes to zero in the export packet.

This is really an issue with how IP fragmentation works, since as I realized after looking around that IP fragments do not carry the application level port information, and hence port 0, which NFC treats as a new flow and shows up else where in the NFC collected files.

Hope this helps, essentially your filtering idea would possibly help here. Let me know if you run into any other issues here.

Regards.