You are not logged in.

#1 2011-05-03 13:22:11

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,698
Website

[Python2 & ftplib] MDTM command changes on different clients

This is directly related to an issue reported by Ravenman in the Pacserve thread.

When connecting to the same server with the same script (see below) and the same versions of all implicated packages, we do not get the same results. The only thing that might be different is the system architecture, but given the output below, that seems irrelevant.

Here is a simplified debugging script to give an example:

#!/usr/bin/env python2

import ftplib
import urlparse

urls = [
  'ftp://ftp.archlinux.org/core/os/i686/core.db.tar.gz',
#   'ftp://mirror.aarnet.edu.au/pub/archlinux/core/os/i686/core.db.tar.gz',
#   'ftp://mirror.rit.edu/archlinux/core/os/i686/core.db.tar.gz',
#   'ftp://ftp.jaist.ac.jp/pub/Linux/ArchLinux/core/os/i686/core.db.tar.gz',
#   'ftp://ftp.tku.edu.tw/Linux/ArchLinux/core/os/i686/core.db.tar.gz'
]

for url in urls:
  print url
  parsed = urlparse.urlparse(url)
  path = parsed.path
  host = parsed.netloc
  port = None
  try:
    host, port = host.split(':')
  except ValueError:
    pass

  ftp = ftplib.FTP()
  ftp.set_debuglevel(2)
  ftp.connect(host, port)
  ftp.login()
  try:
    ftp.sendcmd('MDTM %s' % path)
  except ftplib.all_errors as e:
    print "error: %s" % e
  ftp.quit()
  print ""

When I run it on my system (x86_64), I get the following output:

ftp://ftp.archlinux.org/core/os/i686/core.db.tar.gz
*get* '220 (vsFTPd 2.1.0)\r\n'
*resp* '220 (vsFTPd 2.1.0)'
*cmd* 'USER anonymous'
*put* 'USER anonymous\r\n'
*get* '331 Please specify the password.\r\n'
*resp* '331 Please specify the password.'
*cmd* 'PASS **********'
*put* 'PASS **********\r\n'
*get* '230 Login successful.\r\n'
*resp* '230 Login successful.'
*cmd* 'MDTM /core/os/i686/core.db.tar.gz'
*put* 'MDTM /core/os/i686/core.db.tar.gz\r\n'
*get* '213 20110503081423\r\n'
*resp* '213 20110503081423'
*cmd* 'QUIT'
*put* 'QUIT\r\n'
*get* '221 Goodbye.\r\n'
*resp* '221 Goodbye.'

When Ravenman runs it on his, he reports the following:

ftp://ftp.archlinux.org/core/os/i686/core.db.tar.gz
*get* '220 (vsFTPd 2.1.0)\r\n'
*resp* '220 (vsFTPd 2.1.0)'
*cmd* 'USER anonymous'
*put* 'USER anonymous\r\n'
*get* '331 Please specify the password.\r\n'
*resp* '331 Please specify the password.'
*cmd* 'PASS **********'
*put* 'PASS **********\r\n'
*get* '230 Login successful.\r\n'
*resp* '230 Login successful.'
*cmd* 'MDTM /core/os/i686/core.db.tar.gz'
*put* 'MDTM /core/os/i686/core.db.tar.gz\r\n'
*get* '500 command not understood\r\n'
*resp* '500 command not understood'
error: 500 command not understood
*cmd* 'QUIT'
*put* 'QUIT\r\n'
*get* '221 Goodbye.\r\n'
*resp* '221 Goodbye.'

Notice the "500 command not understood\r\n" after the MDTM command. The same error occurs on his system when connecting to other FTP servers in the script (see this post) . They all work as expected here.

According to the output we're sending the exact same commands to the same server, yet the server fails to recognize them when he sends them.

Does anyone understand what is going on? What am I missing? I've already wondered a few times if I'm just being cleverly trolled.

I'd also appreciate it others could run the test script and post the results along with the version of Python2 that they're using.

Thanks

Offline

#2 2011-05-03 14:18:06

ataraxia
Member
From: Pittsburgh
Registered: 2007-05-06
Posts: 1,537

Re: [Python2 & ftplib] MDTM command changes on different clients

I have 3 x86_64 boxes, all on python2 2.7.1-9, but on 3 different networks. The test script works as expected for me - same output in all three cases:

$ ./testmdtm.py 
ftp://ftp.archlinux.org/core/os/i686/core.db.tar.gz
*get* '220 (vsFTPd 2.1.0)\r\n'
*resp* '220 (vsFTPd 2.1.0)'
*cmd* 'USER anonymous'
*put* 'USER anonymous\r\n'
*get* '331 Please specify the password.\r\n'
*resp* '331 Please specify the password.'
*cmd* 'PASS **********'
*put* 'PASS **********\r\n'
*get* '230 Login successful.\r\n'
*resp* '230 Login successful.'
*cmd* 'MDTM /core/os/i686/core.db.tar.gz'
*put* 'MDTM /core/os/i686/core.db.tar.gz\r\n'
*get* '213 20110503081423\r\n'
*resp* '213 20110503081423'
*cmd* 'QUIT'
*put* 'QUIT\r\n'
*get* '221 Goodbye.\r\n'
*resp* '221 Goodbye.'

I wonder if there's some kind of transparent proxying going on for Ravenman, with a rather stupid proxy that doesn't implement MDTM. Then again, I don't know if transparent proxies are even possible with FTP.

Offline

#3 2011-05-03 14:25:55

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: [Python2 & ftplib] MDTM command changes on different clients

Could it be some kind of locale/encoding issue?

Last edited by Daenyth (2011-05-03 14:26:25)

Offline

#4 2011-05-03 14:47:29

juster
Forum Fellow
Registered: 2008-10-07
Posts: 195

Re: [Python2 & ftplib] MDTM command changes on different clients

Here is your script being run. It works fine for me.

[juster@artemis ~]$ python2 testftp.py
ftp://ftp.archlinux.org/core/os/i686/core.db.tar.gz
*get* '220 (vsFTPd 2.0.7)\r\n'
*resp* '220 (vsFTPd 2.0.7)'
*cmd* 'USER anonymous'
*put* 'USER anonymous\r\n'
*get* '331 Please specify the password.\r\n'
*resp* '331 Please specify the password.'
*cmd* 'PASS **********'
*put* 'PASS **********\r\n'
*get* '230 Login successful.\r\n'
*resp* '230 Login successful.'
*cmd* 'MDTM /core/os/i686/core.db.tar.gz'
*put* 'MDTM /core/os/i686/core.db.tar.gz\r\n'
*get* '213 20110503081423\r\n'
*resp* '213 20110503081423'
*cmd* 'QUIT'
*put* 'QUIT\r\n'
*get* '221 Goodbye.\r\n'
*resp* '221 Goodbye.'

[juster@artemis ~]$ python2 --version
Python 2.7.1

Notice how I have different server version numbers. WTF? edit: Ok I see ftp.archlinux.org has two addresses, the 143 host uses vsFTPd 2.1.0 and the 144 host runs vsFTPd 2.0.7.

To rule out an obscure bug in ftplib, Ravenman might try using netcat:

nc ftp.archlinux.org 21 <<EOF
USER anonymous
PASS my@email.com
MDTM /core/os/i686/core.db.tar.gz
QUIT
EOF

While messing with arch's FTP server in netcat I could not figure how to get the 500 command not understood error. I would guess that the server noticed the MDTM command (or else it would say unknown command) but had a problem with the command arguments. Maybe like Daenyth said, a subtle encoding problem is occurring.

Last edited by juster (2011-05-03 14:51:32)

Offline

#5 2011-05-07 15:28:53

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,698
Website

Re: [Python2 & ftplib] MDTM command changes on different clients

Thanks for the replies.



@ataraxia
I don't know if a transparent proxy would be possible either. I've suspected his network settings too but I have no idea what to suggest to even test them.



@Daenyth
I don't see how. The characters are all in the ASCII range and I can't find anything online about ftplib depending on locale or encoding issues. It should be sending the exact same thing.

I have even compared Ravenman's output to mine in an editor to see if there's any difference, but I found none. Of course, if there were some difference it might have been lost at some point as it was being copied and pasted.




@juster
I like the nc test idea. I'll ask him to try that.

Offline

Board footer

Powered by FluxBB