Subject: UDP fragment packets

From: "Logg, Connie A." <[email-masked]>
Date: Fri, 04 May 2001 15:24:48 -0700

I am wondering if there is any concensus on how UDP fragment packets could be counted for flow purposes.

I am anaylzing data from Cisco's Netflow feature which is generated by one of our Catalyst 6509s 
with a MSFC in it.

The majority of the UDP data is accounted in records with a source port and destination port of 0.

In fact, these packets that are counted in these records are fragments of AFS flows, and I think they should be
counted as AFS packets by the Netflow generation code.

Here is a detailed decode of a series of packets (transferring a 5700 byte datagram) (hope I have the terminology
correct)..  I had posed the question to Cisco of how these packets are handled, and the response I first
got back is that "we do not understand why anyone would want to count fragment packets".  Such is life.

I am wondering if you folks have an opinion on how these types of packets should be counted?

Thanks, COnnie

Connie Logg - Network Analyst - 650-926-2879
Stanford Linear Accelerator Center
MS 97; 2575 SandHill Road; Menlo Park CA 94025
"Happiness is found along the way, not at the end of the road"

##################################
This example involves the transfer of 5700 bytes of data.  Since the mtu is only 1500, this data must be broken up into multiple packets (fragments of the data stream). There are 4 packets in this flow.  The first UDP packet contains the port numbers and first part of the data, and the following 3 UDP packets contain the rest of the data.  I want to have these counted, because they represent a large portion of the data.  The statistics from analyzing the netflow data will be grossly wrong if they are not since we have a lot of large data transfers. The identification field is the key to counting the packets correctly.  For the results that I see when I compare a packet capture with the netflow records, it appears that the last 3 packets are not counted.

The last 3 fragment packets of the file transfer are indeed complete packets. The term "fragment" comes from the
IP spec.  That is why I called them fragments.  If you look at the packet decodes I sent you will see the "fragment"
bits.  I pointed to them.

Packet 1:  has ports in it for level 3 decode
---------

----- Level # 1 is ETHERNET Offset: 0 Size: 14
 
 Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
 Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
 Type           : 2048 (DoD IP)                    08 00              ..
----- Level # 2 is DoD IP Offset: 14 Size: 20
 
 Version        : 4                                 4                 .
 Header Length  : 20                                5                 .
 Type Of Service: 0                                00                 .
                : 000..... Routine
                : ...0.... Normal Delay
                : ....0... Normal Throughput
                : .....0.. Normal Reliability
 Total Length   : 1500                             05 dc              ..
 Identification : 2736                             0a b0              <---critical info      
 Flags          : 3                                 3                 .
                : 0.. Reserved
                : .1. Don't Fragment
                : ..1 More Fragments                                  <--- note more fragments to come
 Fragment Offset: 0                                60 00              `.
 Time To Live   : 255                              ff                 .
 Protocol       : 17 (DODUDP)                      11                 .
 Header Checksum: 44704                            ae a0              ..
 Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
 Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
----- Level # 3 is DoD UDP Offset: 34 Size: 8                         < note has ports in it
 
 Source Port    : 7000 (AFS3-FILESERVER)           1b 58              .X
 Dest Port      : 7001 (AFS3-CALLBACK)             1b 59              .Y 
 Length         : 5700                             16 44              .D
 Checksum       : 23042                            5a 02              Z.
----- Level # 7 is UserData Offset: 42 Size: 1476
 
  00032:                               a5 f5 c1 83 67 08             ....g.
  00048: 90 3c 00 00 00 07 00 00 00 01 00 00 00 08 01 02   .<..............
....

Packet 2: UDP data packet with NO port info...just data
---------

Frame 49 Size  1518 Absolute Time Apr 26 15:41:35.401 ASCII MODE
-------------------------------------------------------------------------------
----- Level # 1 is ETHERNET Offset: 0 Size: 14
 
 Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
 Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
 Type           : 2048 (DoD IP)                    08 00              ..
----- Level # 2 is DoD IP Offset: 14 Size: 20
 
 Version        : 4                                 4                 .
 Header Length  : 20                                5                 .
 Type Of Service: 0                                00                 .
                : 000..... Routine
                : ...0.... Normal Delay
                : ....0... Normal Throughput
                : .....0.. Normal Reliability
 Total Length   : 1500                             05 dc              ..
 Identification : 2736                             0a b0                <- critical info
 Flags          : 3                                 3                 .
                : 0.. Reserved
                : .1. Don't Fragment
                : ..1 More Fragments                                    <--more fragments to come
 Fragment Offset: 1480                             60 b9              `.
 Time To Live   : 255                              ff                 .
 Protocol       : 17 (DODUDP)                      11                 .
 Header Checksum: 44519                            ad e7              ..
 Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
 Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
----- Level # 7 is UserData Offset: 34 Size: 1484
 
  00032:       65 64 75 20 4e 4f 5f 55 4e 49 58 49 44 20     edu NO_UNIXID 

Packet 3: Another fragment...udp data packet only
---------

Frame 50 Size  1518 Absolute Time Apr 26 15:41:35.402 ASCII MODE
-------------------------------------------------------------------------------
----- Level # 1 is ETHERNET Offset: 0 Size: 14
 
 Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
 Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
 Type           : 2048 (DoD IP)                    08 00              ..
----- Level # 2 is DoD IP Offset: 14 Size: 20
 
 Version        : 4                                 4                 .
 Header Length  : 20                                5                 .
 Type Of Service: 0                                00                 .
                : 000..... Routine
                : ...0.... Normal Delay
                : ....0... Normal Throughput
                : .....0.. Normal Reliability
 Total Length   : 1500                             05 dc              ..
 Identification : 2736                             0a b0                  <--critical info
 Flags          : 3                                 3                 .
                : 0.. Reserved
                : .1. Don't Fragment
                : ..1 More Fragments                                      <--------more fragments to come
 Fragment Offset: 2960                             61 72              ar
 Time To Live   : 255                              ff                 .
 Protocol       : 17 (DODUDP)                      11                 .
 Header Checksum: 44334                            ad 2e              ..
 Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
 Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
----- Level # 7 is UserData Offset: 34 Size: 1484
 
  00032:       73 61 2e 69 6e 66 6e 2e 69 74 20 43 4f 53     sa.infn.it COS
  00048: 54 41 4e 54 49 20 43 6f 73 74 61 6e 74 69 6e 69   TANTI Costantini

Packet 4: Last UDP data packet
---------

Frame 51 Size  1298 Absolute Time Apr 26 15:41:35.402 ASCII MODE
-------------------------------------------------------------------------------
----- Level # 1 is ETHERNET Offset: 0 Size: 14
 
 Dest Address   : Cisco   -07-ac-10                00 00 0c 07 ac 10  ......
 Source Address : Sun     -9a-d8-b3                08 00 20 9a d8 b3  .. ...
 Type           : 2048 (DoD IP)                    08 00              ..
----- Level # 2 is DoD IP Offset: 14 Size: 20
 
 Version        : 4                                 4                 .
 Header Length  : 20                                5                 .
 Type Of Service: 0                                00                 .

                : 000..... Routine
                : ...0.... Normal Delay
                : ....0... Normal Throughput
                : .....0.. Normal Reliability
 Total Length   : 1280                             05 00              ..
 Identification : 2736                             0a b0                 <- critical info
 Flags          : 2                                 2                 .
                : 0.. Reserved
                : .1. Don't Fragment
                : ..0 Last Fragment                                      < last fragment
 Fragment Offset: 344                              42 2b              B+
 Time To Live   : 255                              ff                 .
 Protocol       : 17 (DODUDP)                      11                 .
 Header Checksum: 52561                            cd 51              .Q
 Source Address : [134.79.16.109] AFS09.SLAC.Stanfo86 4f 10 6d        .O.m
 Dest Address   : [128.138.133.121] blanca.Colorado80 8a 85 79        ...y
----- Level # 7 is UserData Offset: 34 Size: 1264
 
  00032:       70 69 73 61 2e 69 6e 66 6e 2e 69 74 20 42     pisa.infn.it B

#####################################

Connie Logg - Network Analyst - 650-926-2879
Stanford Linear Accelerator Center
MS 97; 2575 SandHill Road; Menlo Park CA 94025
"Happiness is found along the way, not at the end of the road"

---

From: "Carter Bullard" <[email-masked]>
Date: Fri, 4 May 2001 21:11:19 -0400

Hey Connie,
   Argus-2.0.1 accounts for fragments by updating the parent
flow, i.e. the flow that is indexed by the first packet in
a collection of fragments.  If the first packet is not observed,
then there is no information that will allow you to index back
to the parent flow, and so Argus generates a separate flow
for the stream of fragments, that is defined by:
   src IP addr
   dst IP addr
   proto
   ip_id

This is reported as a special fragmented packet flow.
These are generally 2-4 packet flows, and so its rather
inefficient when there are a lot of them.  Aggregation can
merge these fragmented packet flows together by ignoring
the ip_id tag, allowing you to minimize the number of flows
reported when there are a lot of fragments where the first
packet is not seen.  The aggregated flow should retain
the indication that the flow is a partially assembled
fragment flow.

   Because Cisco has no state for tracking fragments, 
they are just aggregating with the information available
in the immediate packet, src, dst addrs and proto.

Hope this helps,

Carter

Carter Bullard
QoSient, LLC
300 E. 56th Street, Suite 18K
New York, New York  10022

[email-masked]
Phone +1 212 588-9133
Fax   +1 212 588-9134
http://qosient.com

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Logg, Connie A.
> Sent: Friday, May 04, 2001 6:25 PM
> To: [email-masked]'
> Subject: UDP fragment packets
> 
> 
> I am wondering if there is any concensus on how UDP fragment 
> packets could be counted for flow purposes.
> 
> In fact, these packets that are counted in these records are 
> fragments of AFS flows, and I think they should be
> Here is a detailed decode of a series of packets 
> (transferring a 5700 byte datagram) (hope I have the terminology
> correct)..  I had posed the question to Cisco of how these 
> packets are handled, and the response I first
> got back is that "we do not understand why anyone would want 
> to count fragment packets".  Such is life.
> 
> This example involves the transfer of 5700 bytes of data.  
> Since the mtu is only 1500, this data must be broken up into 
> multiple packets (fragments of the data stream). There are 4 
> packets in this flow.  The first UDP packet contains the port 
> numbers and first part of the data, and the following 3 UDP 
> packets contain the rest of the data.  I want to have these 
> counted, because they represent a large portion of the data.  
> The statistics from analyzing the netflow data will be 
> grossly wrong if they are not since we have a lot of large 
> data transfers. The identification field is the key to 
> counting the packets correctly.  For the results that I see 
> when I compare a packet capture with the netflow records, it 
> appears that the last 3 packets are not counted.
> 
> The last 3 fragment packets of the file transfer are indeed 
> complete packets. The term "fragment" comes from the
> IP spec.  That is why I called them fragments.  If you look 
> at the packet decodes I sent you will see the "fragment"
> bits.  I pointed to them.
> 
>  Version        : 4                                 4         
>         .
>  Header Length  : 20                                5         
>         .
>  Type Of Service: 0                                00         
>         .
>  Flags          : 3                                 3         
>         .
>  Time To Live   : 255                              ff         
>         .
>  Length         : 5700                             16 44      
>         .D
>  Checksum       : 23042                            5a 02      
>         Z.
>  Version        : 4                                 4         
>         .
>  Header Length  : 20                                5         
>         .
>  Type Of Service: 0                                00         
>         .
>  Flags          : 3                                 3         
>         .
>  Time To Live   : 255                              ff         
>         .
>  Version        : 4                                 4         
>         .
>  Header Length  : 20                                5         
>         .
>  Type Of Service: 0                                00         
>         .
>  Flags          : 3                                 3         
>         .
>  Time To Live   : 255                              ff         
>         .
>  Version        : 4                                 4         
>         .
>  Header Length  : 20                                5         
>         .
>  Type Of Service: 0                                00         
>         .
>  Flags          : 2                                 2         
>         .
>  Time To Live   : 255                              ff         
>         .