Arch Linux

9d8

Member

Registered: 2019-06-02

Posts: 7

Error uploading file to ftp server with lftp and ssl

Sometimes, when uploading a file to my ftp server with lftp, I get the error: "426 Failure reading network stream" and the file either fails to upload or only partially uploads. Interestingly, the file upload will occasionally succeed when lftp tries to reconnect, but never on the first attempt.

Here is the full lftp log:

---- Connecting to ########## (###.###.###.###) port ####
<--- 220 (vsFTPd 3.0.3)          
---> FEAT
<--- 211-Features:
<---  AUTH TLS
<---  EPRT
<---  EPSV
<---  MDTM
<---  PASV
<---  PBSZ
<---  PROT
<---  REST STREAM
<---  SIZE
<---  TVFS
<---  UTF8
<--- 211 End
---> AUTH TLS
<--- 234 Proceed with negotiation.
---> OPTS UTF8 ON
Certificate: CN=##########       
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
<--- 200 Always in UTF8 mode.    
---> USER #######
<--- 331 Please specify the password.
---> PASS ########
<--- 230 Login successful.             
---> PWD
<--- 257 "/" is the current directory  
---> PBSZ 0
<--- 200 PBSZ set to 0.
---> MKD upload
<--- 550 Create directory operation failed.
---> MKD upload/
<--- 550 Create directory operation failed.
---> PROT C
<--- 200 PROT now Clear.
---> TYPE I
<--- 200 Switching to Binary mode.
---> PASV
<--- 227 Entering Passive Mode (###,###,###,###,8,164).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> ALLO 29390
<--- 202 ALLO command ignored.
---> STOR upload/somefile
<--- 522 Data connections must be encrypted.
---- Closing data socket
---> PROT P
<--- 200 PROT now Private.
---> PASV
<--- 227 Entering Passive Mode (###,###,###,###,8,171).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> STOR upload/somefile
<--- 150 Ok to send data.
Certificate: CN=##########
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
---- Closing data socket
<--- 426 Failure reading network stream.                     
---> QUIT
<--- 221 Goodbye.
---- Closing control socket

I have worked around this error by disabling ssl on my server, but obviously this isn't a satisfactory solution. I have also tried other clients and they do not have an issue, which tells me that this error is with lftp and not my ftp server, though I would still like to use lftp. This issue also seems more common with smaller files than with larger ones, but idk if that means anything for sure.

Any ideas as to what the problem is, and how to fix it?

Last edited by 9d8 (2019-06-03 02:05:50)

seth

Member

Registered: 2012-09-03

Posts: 61,185

Re: Error uploading file to ftp server with lftp and ssl

---> MKD upload
<--- 550 Create directory operation failed.
---> MKD upload/
<--- 550 Create directory operation failed.
…
---> STOR upload/somefile

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

9d8

Member

Registered: 2019-06-02

Posts: 7

Re: Error uploading file to ftp server with lftp and ssl

That error only occurs due to the shell script that I run with lftp. It will always run mkdir -pf to ensure that all the necessary directories exist. In this case, it failed since the upload/ directory already exists on my server. I removed the mkdir from the script and that didn't fix the 426 error.

seth

Member

Registered: 2012-09-03

Posts: 61,185

Re: Error uploading file to ftp server with lftp and ssl

Did you try to enforce a specific TLS version?
Inconclusive random suggestions from the interwebs include to use "require_ssl_reuse=NO" "use_sendfile=NO".

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Zod

Member

From: Hoosiertucky

Registered: 2019-03-10

Posts: 634

Re: Error uploading file to ftp server with lftp and ssl

I noticed this in the log you posted...

---> ALLO 29390
<--- 202 ALLO command ignored.

Just something to look at...

To write larger files than the limit, the you (or your FTP client) must tell the server how large the incoming file is by giving the FTP ALLO command before the FTP STOR (put) command. Some clients may do this automatically. If your client does not, you can do this manually:

ftp> quote allo 123456789
200 ALLO command ok.
ftp> put bigfile write.test10
local: bigfile remote: write.test10
200 PORT command ok.
125 Data connection open; transfer starting.
226 Binary Transfer complete.
123456789 bytes sent in 23.547 secs (5.1e+04 Kbytes/sec)

Note:Once set, the size set with ALLO will remain in effect for that FTP session until you change 

seth

Member

Registered: 2012-09-03

Posts: 61,185

Re: Error uploading file to ftp server with lftp and ssl

vsftpd (and probably every ftp server in the wild) unconditionally ignores ALLO: https://github.com/richardcochran/vsftp … ostlogin.c (that's NOT the official source but I can't browse ftp://vsftpd.beasts.org/users/cevans/un … tpd-3.0.2/ - apparently requires login)

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Zod

Member

From: Hoosiertucky

Registered: 2019-03-10

Posts: 634

Re: Error uploading file to ftp server with lftp and ssl

I have also tried other clients and they do not have an issue, which tells me that this error is with lftp

Maybe allo needs to be disabled in lftp, I think it uses it by default.

Edit0: I'm wondering why the trouble with lftp and other clients don't have a problem. I'm also wondering about file size of aborted uploads(file size of what manages to get uploaded during an abort) and what, if any, upload limit is set by vsftpd.

Last edited by Zod (2019-06-03 16:51:41)

9d8

Member

Registered: 2019-06-02

Posts: 7

Re: Error uploading file to ftp server with lftp and ssl

I tried setting use_sendfile=NO and it had no noticeable effect. I have vsftpd set to only accept tlsv1 and I already did not require ssl reuse.

I disabled allo in lftp and it didn't stop the 426 error.

After playing around for a bit, I found that lftp will upload 16kb before failing. By setting net:reconnect-interval-base to 0, lftp will resume and keep uploading 16kb until the entire file is uploaded. This is more of a workaround than a real solution. Strangely, the 426 error is not present when uploading an 18GB archive file, which further supports my idea that this issue only occurs on smaller files. As far as I am aware, I have no upload limits set on lftp or on my remote server.

9d8

Member

Registered: 2019-06-02

Posts: 7

Re: Error uploading file to ftp server with lftp and ssl

I feel like I should mention that my server uses flash drives and micro SDs for storage all with ext4 filesystems (Its on a raspberry pi where there isn't enough power for using external disk drives). Could there be some physical limits on these drives preventing the upload from succeeding?

Zod

Member

From: Hoosiertucky

Registered: 2019-03-10

Posts: 634

Re: Error uploading file to ftp server with lftp and ssl

Could you provide another lftp log after setting net:reconnect-interval-base to 0?

Edit0:
Client closing data socket...

---- Closing data socket[

vsftpd complaining that the client closed the data socket...

<--- 426 Failure reading network stream.

Last edited by Zod (2019-06-03 18:11:11)

9d8

Member

Registered: 2019-06-02

Posts: 7

Re: Error uploading file to ftp server with lftp and ssl

Here it is. This is with allo disabled and net:reconnect-interval-base 0. The new error 500 occurs because I disabled atimes in the server's fstab.

---- Connecting to ########## (###.###.###.###) port ####
<--- 220 (vsFTPd 3.0.3)          
---> FEAT
<--- 211-Features:
<---  AUTH TLS
<---  EPRT
<---  EPSV
<---  MDTM
<---  PASV
<---  PBSZ
<---  PROT
<---  REST STREAM
<---  SIZE
<---  TVFS
<---  UTF8
<--- 211 End
---> AUTH TLS
<--- 234 Proceed with negotiation.
---> OPTS UTF8 ON
Certificate: CN=##########      
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
<--- 200 Always in UTF8 mode.    
---> USER #######
<--- 331 Please specify the password.
---> PASS #######
<--- 230 Login successful.             
---> PWD
<--- 257 "/" is the current directory
---> PBSZ 0
<--- 200 PBSZ set to 0.
---> MKD upload
<--- 550 Create directory operation failed.
---> MKD upload/
<--- 550 Create directory operation failed.
---> PROT C
<--- 200 PROT now Clear.
---> TYPE I
<--- 200 Switching to Binary mode.
---> PASV
<--- 227 Entering Passive Mode (###,###,###,###,8,174).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> STOR upload/somefile
<--- 522 Data connections must be encrypted.
---- Closing data socket
---> PROT P
<--- 200 PROT now Private.
---> PASV
<--- 227 Entering Passive Mode (###,###,###,###,8,174).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> STOR upload/somefile
<--- 150 Ok to send data.
Certificate: CN=##########
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
---- Closing data socket
<--- 426 Failure reading network stream.                     
---> QUIT
<--- 221 Goodbye.
---- Closing control socket
---- Connecting to ########## (###.###.###.###) port ####
<--- 220 (vsFTPd 3.0.3)                         
---> FEAT
<--- 211-Features:
<---  AUTH TLS
<---  EPRT
<---  EPSV
<---  MDTM
<---  PASV
<---  PBSZ
<---  PROT
<---  REST STREAM
<---  SIZE
<---  TVFS
<---  UTF8
<--- 211 End
---> AUTH TLS
<--- 234 Proceed with negotiation.
---> OPTS UTF8 ON
Certificate: CN=##########                
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
<--- 200 Always in UTF8 mode.             
---> USER #######
<--- 331 Please specify the password.
---> PASS #######
<--- 230 Login successful.                      
---> PBSZ 0
<--- 200 PBSZ set to 0.
---> PROT P
<--- 200 PROT now Private.
---> TYPE I
<--- 200 Switching to Binary mode.
---> SIZE upload/somefile
<--- 213 16384
---> PASV
copy: put rolled back to 16384, seeking get accordingly
<--- 227 Entering Passive Mode (###,###,###,###,8,171).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> REST 16384
<--- 350 Restart position accepted (16384).
---> STOR upload/somefile
<--- 150 Ok to send data.
Certificate: CN=##########
 Issued by:        C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Checking against: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
  Trusted
Certificate: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
 Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
  Trusted
---- Closing data socket
<--- 226 Transfer complete.                                  
---> SITE UTIME 20190602151412 upload/somefile
<--- 500 Unknown SITE command.
---> SITE UTIME upload/somefile 20190602151412 20190602151412 20190602151412 UTC
<--- 500 Unknown SITE command.
---> QUIT
<--- 221 Goodbye.
---- Closing control socket

9d8

Member

Registered: 2019-06-02

Posts: 7

Re: Error uploading file to ftp server with lftp and ssl

I just checked the logs on vsftpd. It seems like the SSL errno 32 is the cause of the problem. Searching for errno 32 leads to something called a "Broken pipe error". I'm not yet sure how to fix it.

Mon Jun  3 14:14:58 2019 [pid 2] [######] DEBUG: Client "192.168.1.1", "SSL shutdown state is: NONE"
Mon Jun  3 14:14:58 2019 [pid 2] [######] DEBUG: Client "192.168.1.1", "SSL shutdown state is: SSL_SENT_SHUTDOWN"
Mon Jun  3 14:14:58 2019 [pid 2] [######] DEBUG: Client "192.168.1.1", "SSL ret: 18446744073709551615, SSL error: error:00000000:lib(0):func(0):reason(0), errno: 32"
Mon Jun  3 14:14:58 2019 [pid 3] [######] FTP response: Client "192.168.1.1", "426 Failure reading network stream."
Mon Jun  3 14:14:58 2019 [pid 3] [######] FAIL UPLOAD: Client "192.168.1.1", "/upload/somefile", 16384 bytes, 316.38Kbyte/sec

Zod

Member

From: Hoosiertucky

Registered: 2019-03-10

Posts: 634

Re: Error uploading file to ftp server with lftp and ssl

I'm wondering about your script and what your using to send data from the client, I see this in your last log which seems to correspond to the 16kb transfer you mentioned...

copy: put rolled back to 16384, seeking get accordingly
<--- 227 Entering Passive Mode (###,###,###,###,8,171).
---- Connecting data socket to (###.###.###.###) port ####
---- Data connection established
---> REST 16384
<--- 350 Restart position accepted (16384).

It seems like it sends till it's done then closes the data socket

Closing data socket

which results in...

<--- 226 Transfer complete. 

And

FTP response: Client "192.168.1.1", "426 Failure reading network stream.
DEBUG: Client "192.168.1.1", "SSL shutdown state is: NONE"
Mon Jun  3 14:14:58 2019 [pid 2] [######] DEBUG: Client "192.168.1.1", "SSL shutdown state is: SSL_SENT_SHUTDOWN"
Mon Jun  3 14:14:58 2019 [pid 2] [######] DEBUG: Client "192.168.1.1", "SSL ret: 18446744073709551615, SSL error: error:00000000:lib(0):func(0):reason(0), errno: 32"

On the server side.

I don't know, it seems like it's working but that lftp is not closing the data socket cleanly.

FTP supposedly uses TCP which would call for that socket disconnect being agreed upon by both sides.

Last edited by Zod (2019-06-03 19:09:02)