Subject: Re: Computing Traffic Matrix from netflow data

From: Jennifer Rexford <[email-masked]>
Date: Mon, 15 Nov 2004 10:38:45 -0500 (EST)

Jing,

> I'm starting to do up scripts which could be used to compute traffic
> matrix (node-to-node). But, I found it's a little bit troublesome as
> routing path may change with time.

Many studies of traffic matrices have assumed that routing changes
don't occur, or don't have much impact.  These studies typically
take a daily dump of the BGP tables to determine the BGP next-hop
for each destination prefix at each ingress router.  In some recent
work, we've found that ignoring BGP routing changes can sometimes
cause you to miss significant traffic shifts.  See

  http://www.cse.ucsd.edu/~teixeira/tm.pdf

for details.  For this study, we collected a feed of iBGP update
messages from each of the ingress routers of interest, so we
could track the changes in the BGP next-hops continuously.

> Also, using (source_ip, dest_ip) is hard to compute node level
> metrix because routing table size is really large and amount of raw
> data is huge( my situation, more than 120GB).

If you just want to compute the router-to-router traffic matrix,
you could

  - collect Netflow measurements on the incoming traffic
  - aggregate traffic with the same destination prefix, using
       the BGP prefix information
  - aggregate traffic with the same BGP next-hop, using the
       BGP next-hop information

Hope that helps...

-- Jen

---

From: Jing Shen <[email-masked]>
Date: Tue, 16 Nov 2004 19:48:52 +0800 (CST)

Hi ,

Thanks for your help.

> could track the changes in the BGP next-hops

To my situation, there is three E-BGP border routers
which connect to the same AS with multiple links. In
order to balance load on those links, BGP
advertisement is modified manually ( for traffic in),
and some part of network traffic is hard routed to
special BGP border router. So, OSPF just control part
of intra-AS traffic  while BGP behavior is modified
too.

In fact, I don't think this is a good manner to follow
in network optimization. But, before we get to know
load requirement, esp. the trends of load variation,
it's not possible to start network optimization.
That's the reason why I do with netflow.

Work on similar problem or information on how those
ISPs do with such problem is valuable.

>   - aggregate traffic with the same BGP next-hop,

If I do not make a mistake, above method is good for
deriving traffic matrix for a backbone AS. But my
situation is to optimize a "stub" network connecting
to one AS. The network has its own ASN, it runs BGP
with upper AS when it runs OSPF inside.  Is there any
work on deriving traffic matrix for OSPF area 0? 
Any work on OSPF optimzation for traffic engineering?

thanks again. 

regards

Jing 

'spam control '

_________________________________________________________
Do You Yahoo!?
150万曲MP3疯狂搜，带您闯入音乐殿堂
http://cn.rd.yahoo.com/mail_cn/tag/yisou/music/*http://music.yisou.com/
美女明星应有尽有，搜遍美图、艳图和酷图
http://cn.rd.yahoo.com/mail_cn/tag/yisou/image/*http://image.yisou.com
1G就是1000兆，雅虎电邮自助扩容！
http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/event/mail_1g/

---

From: Supratik Bhattacharyya <[email-masked]>
Date: Thu, 18 Nov 2004 14:17:23 -0800

Jing,

We did some work on assessing the impact of BGP dynamics on intra-domain
traffic, and we found that there was very little impact. You can find
our Sigmetrics 2004 paper at:

http://ipmon.sprint.com/pubs_trs/pubs/supratik/sigmetrics2004.pdf

For papers on IGP link weight optimization, building traffic matrices
from netflow data and related topics, you can visit:

http://ipmon.sprint.com/pubs_trs/webpages/ATLPUB.php

best,

Supratik

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Jing Shen
Sent: Tuesday, November 16, 2004 3:49 AM
To: Jennifer Rexford; Traffic Engineering List
Cc: [email-masked]
Subject: Re: Computing Traffic Matrix from netflow data

Hi ,

Thanks for your help.

thanks again. 

regards

Jing 

'spam control '

_________________________________________________________
Do You Yahoo!?
150万曲MP3疯狂搜，带您闯入音乐殿堂
http://cn.rd.yahoo.com/mail_cn/tag/yisou/music/*http://music.yisou.com/
美女明星应有尽有，搜遍美图、艳图和酷图
http://cn.rd.yahoo.com/mail_cn/tag/yisou/image/*http://image.yisou.com
1G就是1000兆，雅虎电邮自助扩容！
http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/event/ma
il_1g/