Subject: [BEHAVE] Fwd: I-D Action:draft-van-beijnum-behave-ftp64-04.txt

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 9 Jul 2009 19:04:58 +0200

Here's an update of the FTP64 draft. No huge changes with respect to  
03, but things are very different from 00 - 02.

I'll be presenting the FTP issue in the apps area session in  
Stockholm. (This clashes with the first BEHAVE session...)

Begin forwarded message:

> 	Title           : IPv6-to-IPv4 translation FTP considerations
> 	Author(s)       : I. van Beijnum
> 	Filename        : draft-van-beijnum-behave-ftp64-04.txt
> 	Pages           : 9
> 	Date            : 2009-07-09

> The File Transfer Protocol has a very long history, and despite the
> fact that today, other options exist to perform file transfers, FTP
> is still in common use.  As such, it is important that in the
> situation where some client computers are IPv6-only while many
> servers are still IPv4-only and IPv6-to-IPv4 translators are used to
> bridge that gap, FTP is made to work through these translators as
> best it can.

> FTP has an active and a passive mode, both as original commands that
> are IPv4-specific, and as extended, IP version agnostic commands.
> The only FTP mode that works without changes through an IPv6-to-IPv4
> translator is extended passive However, many existing FTP servers
> don't support this mode, and some clients don't ask for it.  This
> document describes the optimal server, client and middlebox (if any)
> behavior to minimize this problem.

> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-van-beijnum-behave-ftp64-04.txt

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 13 Jul 2009 23:19:54 +0200

Here's an update of the FTP64 draft that fixes an incorrect  
interpretation of the default port 20 behavior in the -04 version:

http://www.ietf.org/internet-drafts/draft-van-beijnum-behave-ftp64-05.txt

---

From: liu dapeng <[email-masked]>
Date: Tue, 14 Jul 2009 16:54:14 +0800

Hi,Iljitsch,
  I have couple of questions about the ftp64 draft:

  1) In the introduction section:
     'It appears that in these cases, the server did support EPSV but
a middlebox
     didn't'
     My understanding is that if the FTP server support EPSV, it will then
     response with a '229 Extended Passive Mode Entered' message which contains
     the server's TCP port number.
     I donot see why the middlebox cannot support this command. Could
     you please explain this?

  2) Why do not we just require all the IPv6 FTP client should support
both EPSV and
     PASV command?
     if the server supports EPSV, it will response only a port number, and the
     client can then use it to initiate the data connection.

     if the server only supports PASV, it will response both server's
address and port
     number. but the client can be designed to use the control connection's
     server IP address to initiate the data connection.

    This approach does not need  ALG, and only requires the IPv6 FTP client
     implementation comply with this guideline.

BR,
Dapeng Liu

2009/7/14, Iljitsch van Beijnum <[email-masked]>:
> _______________________________________________
> Behave mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/behave
>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 14 Jul 2009 11:46:12 +0200

On 14 jul 2009, at 10:54, liu dapeng wrote:

>     a middlebox didn't'
>     response with a '229 Extended Passive Mode Entered' message which
>     contains    the server's TCP port number.

After issuing the EPSV command, nothing happens for a long time. If  
the server simply didn't understand EPSV it would send back an error  
and there would be no delay. Under the assumption that the server not  
not supports EPSV and EPSV doesn't work my conclusion is that the  
server sends a 229 response, but some other box in the path of the  
control channel doesn't handle that response, and probably keeps  
waiting for the 227 response, which of course never comes. It's the  
only explanation that makes sense.

>     both EPSV and PASV command?

The draft says:

All FTP clients MUST support EPSV when communicating over IPv6.

It is highly RECOMMENDED that FTP clients react by retrying with PASV
or EPRT when the EPSV command fails

The reason why PASV isn't a MUST is that it requires logic that  
interprets errors and then retries, which could complicate a client a  
lot, depending on the architecture.

>     if the server supports EPSV, it will response only a port  
> number, and
>     the client can then use it to initiate the data connection.

Yes, this is what we want. The draft requires servers to support EPSV,  
but today, about a third of the IPv4 FTP servers don't.

>     server IP address to initiate the data connection.

Yes, that is what the draft specifies, although it's "highly  
recommended", not required.

>     implementation comply with this guideline.

Yes. And the good thing is that for most people, it's not too hard to  
upgrade to a more capable FTP client. So even if the servers aren't  
upgraded to support EPSV and the NAT64 operators don't implement an  
FTP ALG, things can still work.

Are you saying that the server requirements and ALG description should  
be removed from the draft?

Thanks for the comments,

Iljitsch

---

From: liu dapeng <[email-masked]>
Date: Thu, 16 Jul 2009 01:38:59 +0800

> server sends a 229 response, but some other box in the path of the

If the server does not support EPSV, how could it send 229 response?

> only explanation that makes sense.

it still seems not clear for me how this would happen...

> be removed from the draft?

Yes, I prefer the approache not using ALG, as you said, maybe it is
not too hard for the user to upgrade their FTP client to communicate
with a server whether it support EPSV or not even there is a NAT64
without ALG on the path. It also fits the philosophy that the
intelligence should be in the end system and keep the network simple.

I also suggest that maybe we can revise section 4 a little as follows:

4.  Client requirements

   It is highly RECOMMENDED that FTP clients react by retrying with PASV
   or EPRT when the EPSV command fails, either because of an error
   response by the server, because the data connection couldn't be
   created or because the control channel session was terminated.  In
   the latter two cases, a client MAY cache the name or address of the
   FTP server and issue PASV rather than EPSV in future sessions.  In
   that case, the cache entry SHOULD be cleared if older than 7 days and
   the server indicates EPSV support in its FEAT response where it
   previously did not indicate EPSV support in its FEAT response.

   When an FTP client is communicating over IPv6 and it is unable to use
   the EPSV (and possibly EPRT) command successfully, it SHOULD retry
   with PASV.  However, the server will respond to the PASV command with
   an IPv4 address that the client must use to connect to for the data
   connection.

   [I suggest that we revise this part to only recommend that the
client uses the control connection's IP address, since it is not easy
for the client application to determine whether it is running on a
machine with a IPv4 connection or not]

   The client SHOULD discard the IPv4 address in the response message
and  connect to the address used for the control channel session on
the indicated
port number.

   Clients MUST NOT use any arguments with the EPSV command; many IPv4
   FTP servers react adversely to "EPSV 1".  Clients SHOULD assume that
   servers are unaware of the IP version used by clients.  This may be
   the result from a server implementation that uses the updated IPv6
   socket API with IPv4-mapped addresses, or because the server resides
   behind an IPv6-to-IPv4 translator.

BR,
Dapeng Liu

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 15 Jul 2009 20:24:48 +0200

> it still seems not clear for me how this would happen...

It's not important. What's important is that it doesn't work.

Whether this is because of stupid firewall builders/operators (which I  
blame all problems on the internet for until I've seen proof there is  
another reason) or not doesn't make a difference for what we need to do.

> Yes, I prefer the approache not using ALG

Well, that is something we need to have rough consensus on. So if  
others could chime in that would be useful.

My argument for including the ALG description (which is explicitly NOT  
an endorsement of using ALGs) is that some people will almost  
certainly implement them anyway, and maybe not realize some of the  
issues, creating interoperability problems. If everyone does it the  
same way at least the clients and servers can figure out a good way to  
deal with the ALGs.

> and  connect to the address used for the control channel session on

That's exactly what the current text mandates (in the absense of IPv4  
connectivity). Is the current text unclear or otherwise problematic?

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 15 Jul 2009 12:24:40 -0700

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of liu dapeng
> Sent: Wednesday, July 15, 2009 10:39 AM
> To: Iljitsch van Beijnum
> Cc: Behave WG
> Subject: Re: [BEHAVE] Fwd: I-D 
> Action:draft-van-beijnum-behave-ftp64-04.txt
> 
> >>     My understanding is that if the FTP server support 
> EPSV, it will
> >> then

It happens on ftp.dell.com (among other sites).  You can send EPSV
and get a 229 response.  But you cannot establish a TCP connection
to the port that was indicated in the 229 response.

Results from my testing is at ftp://ftpeng.cisco.com/dwing/ftp-epsv,
see especially ftp-site-list.log which has the output of the testing.
 # grep "EPSV: Ok" ftp-site-list.log | wc
   1626    9756   92685
 # grep "EPSV: Timeout" ftp-site-list.log | wc
    157     954    9474

ftp.dell.com wasn't in that list.  Running it manually right now
using my Perl script generates:
  ftp.dell.com              PASV: Ok   EPSV: Timeout   00:23

-d

---

From: liu dapeng <[email-masked]>
Date: Tue, 21 Jul 2009 10:02:07 +0800

> others could chime in that would be useful.

Agree

> same way at least the clients and servers can figure out a good way to

If the FTP client's implementation complies with this draft, do you
think it still need ALG somewhere in the network?

> connectivity). Is the current text unclear or otherwise problematic?

In section 4. Clinet requirements:

"If the client has IPv4 reachability, it SHOULD set up a
   data connection towards the IPv4 address.  If the client doesn't have
   IPv4 reachability, it SHOULD assume that it is communicating through
   an IPv6-to-IPv4 translator, and that the IPv4 address in the server's
   response is the same IPv4 address that the control channel session is
   connected to."

According to above text, It requires that the FTP client to know
whether it has IPv4 connectivity or not, but actually, it is not easy
for the FTP client application to determine whether it is running on a
machine has a IPv4 connectivity or not. It will alos increase the FTP
client's complexity which we can avoid.

So I suggest that the FTP client simply use the control session's IP
address whether it has IPv4 connectivity or not.

BR,
Dapeng

---

From: liu dapeng <[email-masked]>
Date: Tue, 21 Jul 2009 11:39:52 +0800

>  # grep "EPSV: Ok" ftp-site-list.log | wc

Thanks Dan,
I tried ftp.dell.com using my laptop, but cannot establish TCP connection.
DNS query and response seems successful, but after the client send a
TCP SYN (dst port=21) packet to the server, the server has no
response.

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 20 Jul 2009 20:52:02 -0700

> From: liu dapeng [mailto:[email-masked]] 
> Sent: Monday, July 20, 2009 8:40 PM
> To: Dan Wing
> Cc: Iljitsch van Beijnum; Behave WG

Worked for me just now:

# telnet ftp.dell.com 21
Trying 143.166.11.10...
Connected to ftp.dell.com.
Escape character is '^]'.
220 Microsoft FTP Service
user anonymous
331 Anonymous access allowed, send identity (e-mail name) as password.
pass [email-masked]
230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.
    For information about DELL, call +1 800 999 3355 All transfers are logged
wi
th
    your host name and email address. If you don't like this policy please
disco
nnect now.
    Please be advised that use constitutes consent to monitoring (Elec Comm
Priv
 Act,
    18 USC 2701-2711). Please see the file readme.txt for disclaimers
pertaining
 to this
    service. If your FTP client crashes or hangs shortly after login, try
using
a dash
    (-) as the first character of your password. This will turn off the
informat
ional
    messages which may be confusing your ftp client.
    ********IN CASE OF PROBLEMS*************************
    ** File Content: send EMAIL to [email-masked]   **
    ** FTP Server: send EMAIL to [email-masked]  **
    ** WWW Server: send EMAIL to [email-masked]   **
    ****************************************************
230 User logged in.
stat
211-Microsoft FTP Service status:
     Logged in user: anonymous
     TYPE: ASCII; FORM: NONPRINT; STRUcture: FILE; transfer MODE: STREAM
     Data connection: none
211 End of status.
quit
221 Thank you for using the Dell FTP site, please come again.
Connection closed by foreign host.

You might try doing a tcp traceroute to it.

-d

> >> >>    This approach does not need  ALG, and only requires 
> the IPv6 FTP
> >> >> client

---

From: liu dapeng <[email-masked]>
Date: Tue, 21 Jul 2009 15:02:16 +0800

>     ** WWW Server: send EMAIL to [email-masked]   **

Thanks Dan,
It seems something wrong with my fixed network ISP's configuration.
I can login using a WAN wireless card.
It seems that the FTP server dose supports EPSV and could response a
229 message but the client cannot establish data connection to the
server. It could not be the firewall that prevent the connection
useing the appointed port, since PASV can work fine.

BR,
Dapeng Liu

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 21 Jul 2009 10:15:03 +0200

> useing the appointed port, since PASV can work fine.

Based on this, it is my conclusion that the actual server is sitting  
behind a firewall that monitors the FTP control channel and then  
allows incoming connections to the FTP server if it sees a 227 message  
from the server, but not if it sees a 229 message.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 21 Jul 2009 10:21:08 +0200

> machine has a IPv4 connectivity or not.

Not sure that's true... The system knows whether it has an address and  
a default route.

> It will alos increase the FTP

> address whether it has IPv4 connectivity or not.

How about:

"If the client has positive knowledge that it has IPv4 connectivity,  
it SHOULD connect to the transport address provided in the server's  
response. If the client has no IPv4 connectivity or the status of IPv4  
connectivity is unknown, the client SHOULD assume..."

---

From: liu dapeng <[email-masked]>
Date: Thu, 23 Jul 2009 18:53:50 +0800

> connectivity is unknown, the client SHOULD assume..."

Yes, it seems better in my opinion.

But I am still wondering that why we have to use the IPv4 connectivity
to set up the data connection?

Should we encourage the user to use IPv6 address whenever it is
possible? This can encourages the user to tranfer to the IPv6 network.
Besides, we can simify FTP client implemention as well.

BR,
Dapeng Liu

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 23 Jul 2009 13:38:07 +0200

> to set up the data connection?

Wouldn't it be better to use native connectivity (IPv4) rather than go  
through a translator (IPv6)?

> Besides, we can simify FTP client implemention as well.

How about this: MAY connect to the IPv4 address or MAY connect to the  
server's IPv6 address. Then implementers can decide what they prefer.

Iljitsch

---

From: liu dapeng <[email-masked]>
Date: Fri, 24 Jul 2009 00:34:52 +0800

Hi guys,

I find a NetBSD maillist that had dicussed FTP client implementation
issues and they met the same problem as we found. Below is the
maillist archive link for your reference. The topic is "	FTP EPSV and
data connections".

http://mail-index.netbsd.org/tech-net/2005/09/21/0005.html
http://mail-index.netbsd.org/tech-net/2005/09/

BR,
Dapeng Liu

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 23 Jul 2009 20:09:12 +0200

> http://mail-index.netbsd.org/tech-net/2005/09/

Yes.

I'm now wondering what would happen if we did this:

EPSV
227 Entering passive mode 1,2,3,4,5,6
229 Entering extended passive mode (|1286|||)

I guess it couldn't be any worse than a timeout and a failure that we  
get now for those 6%. But would it hurt for the other 94%?

Iljitsch