Subject: Re: Request for review of DNS related draft

From: Edward Warnicke <[email-masked]>
Date: Sat, 1 Mar 2003 12:52:15 -0500 (EST)

I'll provide two examples.

1)	In most nations there is a regulatory requirement to
	that a service providers provide a wiretap of a
	customer when presented with a warrant.  When performing
	such a tap customer premise equipment is not considered
	trusted, even if it is controlled by the SP.  Therefore
	the device in the network responsible for carrying
	out the wiretap must get cooperation from the
	first hop router in the SP's network.  See
	http://www.packetcable.com/downloads/specs/pkt-sp-esp-I01-991229.pdf
	for example.

2)	In the traditional PSTN networks ( according to my telco
	guys, I'm *not* a telco guy myself ) there are a group
	of maintenance functions which involve monitoring the
	actual voice streams of a call for small periods of
	time to check voice quality ( I'm told in the US this
	is legally permissible for between 20-30 seconds of
	a call ).  Analogous maintenance actions for a VoIP
	( say MGCP driven Cable VoIP ) SP would require
	cooperation of the first hop gateway in the service
	providers network.

Ed

On Fri, 28 Feb 2003, Dean Anderson wrote:

> This seems the wrong way to go about MGCP management activity. What makes
> you think the first hop router will have anything to do with VOIP or MGCP?
>
> In the VOIP networks that I've worked with (a number of large voip
> installations since 1998), the first hop routers were never VOIP-aware at
> all.
>
> 		--Dean
>
> On Fri, 28 Feb 2003, Edward Warnicke wrote:
>
> > Not everyone is privy to the routing tables.
> >
> > For example.  Imagine an MGCP call agent which wishes to
> > perform a maintenance activity that requires the cooperation
> > of the first hop gateway for an MGCP gateway.  I'm aware
> > of no MGCP call agent which participates in the IGP for
> > the networks containing those endpoints.  Additionally
> > almost all IGPs will allow for some route summarization
> > which would prevent me from actually knowing who the
> > first hop gateway is in some circumstances.
> >
> > So in this example I would either have to provision the MGCP
> > call agent with information about where the first hop gateway
> > for each endpoint was, or allow it to use the method of the
> > draft to resolve it.
> >
> > Since many of the environments where MGCP is seeing deployment
> > have a very large number of endpoints, and an unusually large
> > churn of first hop gateway changes ( think splitting a cable
> > node ), as well as a tendency toward hierarchical distibution
> > of control of the network, DNS resolution of this information
> > seemed to make sense.
> >
> > The reason RFC 1101 doesn't get used for these sorts of
> > purposes is that it doesn't support features in common use
> > in these networks ( variable length subnet masks ).
> >
> > Ed
> >
> > On Fri, 28 Feb 2003, Robert Elz wrote:
> >
> > >     Date:        Thu, 27 Feb 2003 16:01:03 -0500
> > >     From:        Ed Warnicke <[email-masked]>
> > >     Message-ID:  <[email-masked]>
> > >
> > >   | But DHCP will not allow a device which
> > >   | is not the endpoint ( or the DHCP server ) to discover the network which
> > >   | contains
> > >   | an IP address and the first hop gateway(s) which service that network.
> > >
> > > Rather than how, perhaps the question should be why would anyone care?
> > >
> > > That's largely why 1101 never got used - the nodes for which it might
> > > have provided information that could didn't already know it via other
> > > means, never had much of a reason to care.
> > >
> > > Why would my nodes care what the network that contains some random IP
> > > address might happen to be (or why would I ever care more than the
> > > routing tables will tell me) ?
> > >
> > > kre
> > >
> > >
> >
> >
> >
> > #----------------------------------------------------------------------
> > # To unsubscribe, send a message to <[email-masked]>.
> >
>
> #----------------------------------------------------------------------
> # To unsubscribe, send a message to <[email-masked]>.
>

#----------------------------------------------------------------------
# To unsubscribe, send a message to <[email-masked]>.

---

From: Dean Anderson <[email-masked]>
Date: Sun, 2 Mar 2003 14:32:10 -0500 (EST)

None of these need to know the first hop router on _another_ network.

Tapping a network for law enforcement is a simple as placing a sniffer at
some convenient point where that customers packets can be grabbed. The
packets are then turned over to law enforcement. It has nothing to do with
"trust" of the sniffer.  Law enforcement is responsible for decoding VOIP
traffic, or paying someone to do it, given packets.  A tap on a customer
network also does not need to involve the first hop router. And a law
enforcement tap should probably NOT involve the first hop router, since it
might be detectable by the customer being tapped!

Your second example, (and I have worked on large VOIP networks using cisco
and other equipment), is just plain wrong.  Voice quality functions for
VOIP are built into the Voice gateway.  You don't have to "listen in".
Telco people can't listen in either, unless they want to lose their job.
In wireline switches, there may be capabilities to listen in, and to
provide audio monitoring ports for law enforcement. (Law enforcement is
usually not allowed in the CO, but get a special line for recording
wiretaps).

And last, the analogous actions in a VOIP network (as I said previously)
do NOT involve the first hop gateway, which typically isn't VOIP _AWARE_

I think you need to learn more about Voice networks before proposing
RFC's to solve problems. The problems you are quoting don't exist.

If you work for Cisco, please consult your VOIP group.

		--Dean

---

From: Ed Warnicke <[email-masked]>
Date: Tue, 04 Mar 2003 21:43:23 -0500

Dean,
    My apology for the delay in getting back to you, I've
been busy with other matters in the last few days.

I will address your points in the following sequence:
1)    I will explain why your description of how to provide tapping to law
      enforcement is insufficient.
2)    I will show that for a large class of service providers the first 
hop router
      is the most logical place to tap ( and is in fact in some cases
      the standard place to tap ).
3)    I will dispell the notion that when I speak of wiretapping I'm 
refering
      exclusively to voice, or that I'm in anyway claiming voice awareness
      on the part of the first hop router.
4)    I will clarify my use of the word "trust", as I was apparently
      insufficiently clear.
5)    I will clarify some of what I meant by voice quality
      measurement. Explain why it might make sense to intercept
      actual voice traffic at the first hop gateway to for
      one class of voice quality measurements.  I will also point
      out the relavent telco specs describing the existence of the
      "monitoring for testing and maintenance" functionality I described.

    In your description of how to tap you suggest placing a sniffer
at "some convenient point where that customers packets can be grabbed".
Let's look at this suggestion.  
    First, you must identify "some convenient point where that customers
packets can be grabbed".  In a typical service provider network, with
appropriate levels of redundancy, the only place you will be able to
reliably grab those packets is from the first hop router(s) at the
edge.  Any further into your network and the possible links/routers
the traffic may traverse multiplies rapidly.
    Second, you must find some means of introducing a sniffer.  This
involves either sticking a sniffer inline on the link, getting the
router/device to replicate packets for you to a local interface, or getting
the router/device to replicate packets for you to a remote collection 
point.  
Sticking a sniffer inline with the link is extremely expensive for high
capacity links. Replicating packets to a local interface is sometimes 
possible,
but in a service provider network it is entirely possible that the first 
hope
router(s) have no interfaces not being used for some other purpose. 
 Additionally
it would require introducing your sniffer to the same physical location 
as the
device you're using as an access point for tapping.  This is operationally
expensive and inconvenient.
    Third the data you can capture and turn over to the LEA (law 
enforcement
agency ) under the warrant issued is frequently constrained.  If you are
a cable operator with as DOCSIS implementation then all of your 
customers have
DHCP assigned IP addresses and thus the IP of the subject of the warrant 
may
change over time.  There is also some chance that the IP of the person 
being
tapped will be assigned to another subscriber, thus contaminating your 
sniffed
traffic with someone elses traffic which you may not be able to legally 
turn
over. So statically sniffing by IP is insufficient.  If the warrant is to
tap their voice communications that you are providing over VoIP, then MUST
turn those over, but typically MUST not turn over the customers data 
traffic
( unless a warrant for the data traffic has also been issued, the US is 
a bit
strange this way ).  Since most VoIP bearer traffic is running over RTP 
on a
arbitrary UDP port this presents additional issues.  All of this is 
challenging
to cope with by just "placing a sniffer at some convenient point where that
customers packets can be grabbed".

    The first hop router(s) is the most logical place to tap
for a large class of service providers.  Consider a Cable network.
Here is a network where the edge consists of a large number of
cable modems ( CM ) which are aggregated up a the Cable Modem Termination
System ( CMTS ).  The CMTS is the first hop router.  The CMTS is also
( usually )the first network device that is not on the customer premise.
The CMTS frequently has reduntant links up to redundant distribution
routers in the network.  One can't be entirely sure which path will
be taken after the first CMTS because multiple valid paths are being
intentionally provided.  While there may be a favored path through the
network, there are many conditions which could cause packets to take an
alternate path.  
    If you look at the Packet Cable Electronic Surveillance
Specification which I refered to in my last email here:
http://www.packetcable.com/downloads/specs/pkt-sp-esp-I01-991229.pdf
you will see that the CMTS ( the first hop gateway ) is specified as the
Intercept Access Point for Call Content.  This means it *IS* the tap point.
Large trunking media gateways may be used as Intercept Access Points for
Call Content, but when tapping customer premise media gateways, 
particularily
when their first link bandwidth is constrained ( like a VoIP enabled
cable modem ) having the media gateway act as the Intercept Access Point
is not reasonable.

    When I speak of wiretapping, I am not exclusively speaking of
tapping VoIP traffic.  Warrants in the US may be issued against
data traffic, or voice traffic.  In many other juristictions they are issued
against "communications".  The first hop gateway need not be VoIP aware 
at all.
It needs only to be capable of replicating the desired packets to a 
collection
point upon request.  See the aforementioned Packet Cable Spec for an 
example.
I repeat, I do not claim, nor have I ever claimed, that the first hop 
gateway
is in any fashion VoIP aware.  I have only specified a means to locate
that gateway.  I have not specified the means by which it will replicate 
traffic
for tapping or be instructed to replicate traffic for tapping.  That's 
beyond
my scope.

    When I refered to "trusting" devices I specifying devices that are 
beyond
the customers capability to either interfere with, or detect, tapping 
activities.
Generally any device that is actually on the customer premise is 
considered to
be potentially comprimisable by the person being tapped, either in terms of
detection or interference.  In consumer grade service provider networks the
first hop router will almost always be under the physical control of the SP,
while the endpoint will almost always be on the customers physical premise.
I'm sorry if I was insufficiently clear on that point.  Think Cable Modem,
DSL, dialup etc.

    When I refered to voice quality I did not mean the jitter, packet loss,
or out of order delivery of RTP packets.  This would generally be data
collected on the media gateway.  I was refering to monitoring for human
perceivable voice quality.  If you look at Bellcore GR-818
"Network Maintenance: Access and Testing - Generic Test Architecture"
section 4.4.5 "Monitor/Talk Line" describes the requirement in the TDM
world that you be able to monitor the line for testing/maintenance purposes.
More detail is available in Bellcore GR-844.  
    Additionally, my local telco guy, who's been an engineer in the TDM
world for over 30 years, claims that these function have beem used 
routinely for
both random monitoring of voice quality in the TDM world and specific 
problem
isolation.  It is possible that some of the human part of this monitoring
have been offloaded recently to automated processes as PAMs, PESQ, and PSQM
have been coming up.  But if you've ever looked at a good PAM score for 
a line
that sounded bad to the human ear you understand why this is not a complete
solution.

    I have worked on service provider class 4, class 5, and GR-303 VoIP
replacements.  I've done a fair bit or work on VoIP over Cable.  I used to
know a thing or two about MGCP ( I wrote the ethereal MGCP plugin among
other things ).  I've moved on to doing other things in last year, but I
like to think I'm not *that* rusty.  
    I am trying to solve a very simple problem of edge router location.
That is the limit of the scope of my solution.  I put wrote this draft 
together
to meet a particular demand, but tried to keep it generic enough to be 
useful
for other applications.
    If you have constructive suggestions I'm happy to hear them :)

Ed

---

From: Dean Anderson <[email-masked]>
Date: Tue, 4 Mar 2003 23:25:57 -0500 (EST)

The crux of the issue is this:  You seem to be arguing that there are
insufficent monitoring facilites.  [that is, it would be helpful to have a
monitor ethernet port on a DOCSIS router, or other service provider 'head
end' routers] That could well be true. However, the provider doesn't need
to have a special protocol to find the DOCSIS head end. The provider
already _knows_ where it is from their facility records.  They don't need
a special protocol to tell them where it is and what its address and
network is. They already know this information.

But of course, rather than a special monitor port, what would be better is
a virtual monitor port that is then replicated via a standard vlan
protocol to an even more conveniently located ethernet monitor port.
This is a router software (maybe hardware in some cases) issue. It doesn't
require new protocols.

---- slightly longer version

Whether the network get complicated after the first hop or not, depends on
the network. In most cases, it doesn't get very complicated right away.
In cases where it does get complicated, then it's up to the provider to
	1) adjust the routing so that all that customers packets will go
somewhere it can be easily tapped
	2) Use multiple sniffers to grab all packets for that customer.

If you are using DHCP, then you can either
	1) track that, and update your sniffer filters as the address
changes
	2) Make sure the customer always gets the same address during the
tap. (not very hard, so probably the popular solution)

VOIP presents no additional issues. All packets contain the customer IP
address.  All you have to is grab them. You don't have to decode them. You
don't have to know anything about VOIP. You don't need to know anything
about UDP, or the codec used.  I just suggested to someone that they write
a proprietary encrypted codec.

Regarding the "placing of the sniffer", most switches have the capacity to
designate a port as a "monitor port", which provides a copy of all the
traffic on the monitored port.  This is sufficient for LEA capabilities.
All that is required is an ethernet.  Most of these switches support
10/100/and Gig ether interfaces.

I haven't looked at the specification you mentioned. I will try to do that
in the next day or so.  However, reading what you said, and taking that at
face value, all that seems necessary is an ethernet monitor port.

You seem to be making surveillance into a much bigger job than it is.
People have been doing it for years already. It could be a pain sometimes,
but it is straightforward.  Just pick up the packets, give them to the
LEA. Done.  Don't overthink it.

---

From: Edward Warnicke <[email-masked]>
Date: Wed, 5 Mar 2003 01:26:00 -0500 (EST)

I am arguing that there are insufficient monitoring functions to
allow for cost effective deployment of wiretapping in some situations,
and that there are efforts underway to improve those monitoring functions.

I am also arguing that it is better to use open methods for
the interoperation of components used for these improved monitoring
function than to cobble things together in one-off or proprientary
way.

The service providers will in most cases *not* build their LEA
functionality inhouse, they'll buy it from a vendor.  In such situations
it's better for there to be one common way for the vendors equipment
to discover the Access Intercept Point using an open protocol than
to have to do a series of one-off intergrations with the various
facilities management systems of the service provider.  This is
what I'm trying to provide.  ( Note I specify no new protocols
in my draft, I just lay out a new way of using existing DNS
implementations to acheive my end ).

I would agree with your suggestion that monitoring be done
with a virtual monitor port on the tapping device.  In practice
though using a standard vlan protocol to send the data back to the
collection point isn't viable because most service provider networks
don't use vlans at their edge.  Vlans and ethernet are mostly seen
in enterprise environments ( although this is starting to change
a bit with longer haul gigE ).  Folks are working on protocols
to transmit the monitored data to the collection point.  I'm
not doing that.

--- longer version

I know that the service providers already know where the first hop
gateway is in their facility records ( or should ).  I don't dispute
that.  What I want to avoid is situations where either this information
must be manually provisioned in the inevidable vendor provided
LEA solution, or where special integration will be required at
any service provider for the LEA servers.  This is why I'm trying to
provide a common open way for LEA servers ( and other applications
which may need to find the first hop gateway ) to extract that
information.

I'm a big fan of using ethernet monitor ports on switches together
with switches to collect and analize traffic.  I've used it a lot
in lab environments.  I've even pointed other folks to how to do
it ( I added this section to the ethereal users guide when I last
revised it: http://www.ethereal.com/docs/user-guide/ch04rouswi.html ).
The problem is that ethernet monitor ports and sniffers don't
scale well operationally in a large service provider environment.
Attaching, filtering, and collecting from multiple sniffers would
be very unpleasant and expensive operationally, and is something
you would want to avoid if at all possible.  Ick.

If you look at the problem from one angle you are correct that VoIP
introduces no new issues for wiretapping.  Wiretapping just needs
to provide the packets to law enforcement that are subject to the
warrant.  Typically though you should only provide those packets
which the warrant requires, and no more.  In the US, where a warrant
may cover voice but not data, you need to be able to deliver only
the VoIP packets to law enforcement.  In order to do this you do
need to know which packets are part of the RTP stream ( which
should be forwarded to LEA ) and which are not.  This does require
you to know which packets are part of the RTP stream.

On a personal note, I'd love to see people provide more encryption
protection for traffic.  The law says the service provider has to hand
over the packets to LEA, it doesn't say they have to be useful :)

I'm not trying to over think this problem.  But I'm not alone
in seeing this as more complicated than you do ( see the Packet
Cable spec for example ).  This doesn't necessarily mean I'm
right, but it does mean I'm not a lone crackpot :)

Since I suspect that we've arrived at an agree to disagree point
on the question of whether this is a useful facility, could you
read over the draft with that objection set temporarily aside and
let me know how it looks.  Put differently, if you assume it is
useful to update RFC 1101, does what I'm proposing seem like a
good or bad way to do it and why?

Ed