SSH Timeout - Write failed: Broken pipe?

victorhooi · 2010-05-12 06:18:12

heya,

This isn't a specifically Arch question, but hey, the people on here are usually quite knowledgeable *grins*.

I have a SSH server that I'm connecting to over the internet, via a HTTP proxy (using corkscrew).

I can connect successfully, however, after a while, the SSH connection dies with:

Write failed: Broken pipe

I assumed this was a timeout issue, so I added:

ClientAliveInterval 6000
ClientAliveCountMax 3

to my /etc/ssh/sshd_config file. However, this doesn't seem to change anything. If there's activity on the connection, it's fine. But a minute or so or activity seems to cause the connection to die.

Is there anything I can do to find out more information on what's going on? Or anything I can do to tweak the connection?

Cheers,
Victor

Ashren · 2010-05-12 06:23:45

Add the following to your ssh_config on your client:

ServerAliveInterval 120

victorhooi · 2010-05-12 07:00:38

heya,

Ashren: Thanks for the quick reply .

Yeah, I actually already tried that, my /etc/ssh/ssh_config (on the client) has:

ServerAliveInterval 6000

Still drops out very quickly, unfortunately.

Cheers,
Victor

fukawi2 · 2010-05-12 08:45:05

I'm guessing it's a timeout on the proxy server.... Try leaving 'top' open and see if it still dies.

victorhooi · 2010-05-14 05:25:17

heya,

fukawi2: Aha, that seems to work, keeping top open seems to keep the connection alive. Kinda annoying to have to keep "top" active if I need to leave it alone.

I started "top" in a screen session, and detached it, but that doesn't seem to work ,unfortunately. Any other way of keeping the session alive?

Also, how can you tell if the timeout is in SSHD, or in the proxy server? Or is is just because I've set the ClientAliveInterval/ServerAliveInterval that we definitely know it's the proxy server? It's this one, I believe, btw.

Cheers,
Victor

fukawi2 · 2010-05-14 05:45:47

All proxy servers will timeout eventually if there's no active traffic -- the top session just ensures that there's constantly changing data on the 'screen' thru the tunnel.

I'd try setting your interval a lot lower -- 6000 seconds is over an hour and a half!

victorhooi · 2010-05-14 07:09:46

heya,

Hmm, darn, I suppose there's no other trick you can use to fake activity, while still giving you use of the terminal?

At the moment, it seems to time out if you leave it for around 1 minute, quite annoying. So if I leave it at just a second, it drops out...lol.

Cheers,
Victor

fukawi2 · 2010-05-14 09:42:40

Try setting your ServerAliveInterval to 30 then....

victorhooi · 2010-05-17 01:20:02

heya,

Aha, seems I was a bit silly, I completely misunderstood how the Alive values worked - I assumed, without reading, that it was a timeout window, after which it would auto-close. I really should read the manpages more carefully in the future.

Seems that it's how often it sends the "keepalive" packets. Well, yeah, setting it to a lower value on my client side (/etc/ssh/ssh_config) seems to have worked. And on my Windows PC, I set it to 30 sec under Putty as well, under Connection. I suppose there's no way to find out what the max value is, without experimentation? (I don't have access to the proxy config).

Thanks for your help! .

Cheers,
Victor

fukawi2 · 2010-05-17 02:06:05

victorhooi wrote:

I suppose there's no way to find out what the max value is, without experimentation? (I don't have access to the proxy config).

Yup... Shouldn't be too hard if you bisect it (ie, 30 works, try 90, if that doesn't work, try 60, if that works, try 75)

victorhooi wrote:

Thanks for your help! .

You're welcome

habogay · 2011-05-23 08:03:55

Thanks for your share but i still get error .

Write failed: Broken pipe .
I don't use any proxy .
I already config ServerAliveInterval 6000

but error on config
#ClientAliveInterval 6000
#ClientAliveCountMax 3

fukawi2 · 2011-05-23 09:09:31

habogay wrote:

I don't use any proxy .
I already config ServerAliveInterval 6000

fukawi2 wrote:

I'd try setting your interval a lot lower -- 6000 seconds is over an hour and a half!

alazyrabbit · 2011-10-07 10:02:43

Even though I set the ClientAliveInterval and SeverAliveInterval, sometimes the network is simply unstable, so I wrote a shell script to automatically restart the SSH session. Here is the script: http://nextspaceship.com/2011/09/how-to … h-session/

PeGa! · 2011-10-26 01:34:49

Hi, I came to this post looking for an answer to the same issue, but in my case encrypted imap was compromised too, so a wider search led me to an alternative solution:

echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time

Reference

It worked for me.

Regards.

Last edited by PeGa! (2011-10-26 01:35:10)

synthead · 2011-10-26 02:07:29

The keepalives are basically useless TCP packets sent to the OpenSSH server with the only intention of telling it that the client is still around. If your server is set to timeout after 5 minutes, that's 5 min x 60 sec = 300 seconds. This means that your client needs to send something to the server in less than 300 seconds, like 250, configured as "ServerAliveInterval 250", or you'll get dropped. The smaller you set this interval, the better off you are at the cost of a slight bump in traffic. I work as a Linux admin, and I keep my ServerAliveInterval at 20. I have never seen a pipe break on my servers with this setting unless there are network problems.

man ssh_config:

       ServerAliveInterval
              Sets a timeout interval in seconds after which if no data  has  been
              received  from  the  server,  ssh(1) will send a message through the
              encrypted channel to  request  a  response  from  the  server.   The
              default is 0, indicating that these messages will not be sent to the
              server.  This option applies to protocol version 2 only.

fukawi2 · 2011-10-26 05:56:19

PeGa! wrote:

echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time
Reference

You're basically doing the same thing, but within the kernel TCP/IP stack rather than the application (SSH).

boly38 · 2012-08-06 14:50:16

Hello;
i reply here my own experience because i was unable to find this fix on google
(and this post is on the top of many of my recent historic google results maybe this could help ..)

when trying to make an ssh connection over https proxy i got exactly the same "Write failed: Broken pipe" after 30 sec (20-40 sec)
even after changing ssh(d) (client or server) keep alive directive ...

my case the https proxy config was done by a self-made apache installation..
this apache https proxy enabled the "reqtimeout_module" which is present in the default (apache 2.4.2 distrib) config file

extra/httpd-default.conf extract :

<IfModule reqtimeout_module>
  RequestReadTimeout header=20-40,MinRate=500 body=20,MinRate=500
</IfModule>

the ssh over ssl session was cut by this timeout
(apache debug log say "[reqtimeout:info] (...) Request header read timeout")

to avoid this ; set it to 0 (unlimited)

    <IfModule reqtimeout_module>
      RequestReadTimeout header=0,MinRate=500 body=0,MinRate=500
    </IfModule>

NB: be warn that this module prevent the slow loris ddos attacks (src: http://s121.codeinspot.com/q/235576 ) so maybe you should try different values instead of 0

regards

Last edited by boly38 (2012-08-06 14:52:30)

fukawi2 · 2012-08-06 23:02:49

Well found solution... I just wanted to post about that module protecting against slow loris and re-iterate the warning for future Googlers.... I'd suggest no-one does this on any machine that is important enough to matter if script kiddies to DOS it, at least not without taking other precautions to protect against slow loris.

EDIT: Welcome to the forums

Last edited by fukawi2 (2012-08-06 23:03:05)

archenroot · 2016-03-13 19:56:59

Solution which boly38 suggested fixed such situation with my SSH tunneled via www-servers/apache-2.4.18 running on Gentoo system.

More details about the apache module reqtimout:
https://httpd.apache.org/docs/2.4/mod/m … meout.html

I played little bit with configuration again and following configuration looks stable, but after some time it also fails:

<IfModule reqtimeout_module>
      RequestReadTimeout header=300,MinRate=500 body=300,MinRate=500
</IfModule>

Followed by SSH configs:
Client:

Host *
        ServerAliveInterval 10

And Server:

ClientAliveInterval 10
ClientAliveCountMax 3
TCPKeepAlive no

From previous experiments I still have have also changed Linux kernel TCP KeepAlive 3 user variables by putting them down a lot from default:
http://tldp.org/HOWTO/TCP-Keepalive-HOW … alive.html

18259/24106MB   12.00 12.01 12.05 13/213 12102
[32407:32406 0:609] 09:18:34 Sun Mar 13 [root@prometheus: +1] /etc/apache2
(1:609)# sysctl net.ipv4.tcp_keepalive_time  net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes
net.ipv4.tcp_keepalive_time = 10
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_keepalive_probes = 5

The default values are:

 # cat /proc/sys/net/ipv4/tcp_keepalive_time
  7200

  # cat /proc/sys/net/ipv4/tcp_keepalive_intvl
  75

  # cat /proc/sys/net/ipv4/tcp_keepalive_probes
  9

Finally only DISABLE module for proxy vhost works stable for hours in my case. I would be interested in some balanced configuration here.

Last edited by archenroot (2016-03-13 20:24:13)

ugjka · 2016-03-13 20:00:31

Hell of a necrobump with an empty post.

Edit: archenroot, read the forum rules

Last edited by ugjka (2016-03-13 20:05:45)

ewaller · 2016-03-14 01:57:32

Using this opportunity to close a very old thread

https://wiki.archlinux.org/index.php/Fo … bumping.22

Arch Linux

#1 2010-05-12 06:18:12

SSH Timeout - Write failed: Broken pipe?

#2 2010-05-12 06:23:45

Re: SSH Timeout - Write failed: Broken pipe?

#3 2010-05-12 07:00:38

Re: SSH Timeout - Write failed: Broken pipe?

#4 2010-05-12 08:45:05

Re: SSH Timeout - Write failed: Broken pipe?

#5 2010-05-14 05:25:17

Re: SSH Timeout - Write failed: Broken pipe?

#6 2010-05-14 05:45:47

Re: SSH Timeout - Write failed: Broken pipe?

#7 2010-05-14 07:09:46

Re: SSH Timeout - Write failed: Broken pipe?

#8 2010-05-14 09:42:40

Re: SSH Timeout - Write failed: Broken pipe?

#9 2010-05-17 01:20:02

Re: SSH Timeout - Write failed: Broken pipe?

#10 2010-05-17 02:06:05

Re: SSH Timeout - Write failed: Broken pipe?

#11 2011-05-23 08:03:55

Re: SSH Timeout - Write failed: Broken pipe?

#12 2011-05-23 09:09:31

Re: SSH Timeout - Write failed: Broken pipe?

#13 2011-10-07 10:02:43

Re: SSH Timeout - Write failed: Broken pipe?

#14 2011-10-26 01:34:49

Re: SSH Timeout - Write failed: Broken pipe?

#15 2011-10-26 02:07:29

Re: SSH Timeout - Write failed: Broken pipe?

#16 2011-10-26 05:56:19

Re: SSH Timeout - Write failed: Broken pipe?

#17 2012-08-06 14:50:16

Re: SSH Timeout - Write failed: Broken pipe?

#18 2012-08-06 23:02:49

Re: SSH Timeout - Write failed: Broken pipe?

#19 2016-03-13 19:56:59

Re: SSH Timeout - Write failed: Broken pipe?

#20 2016-03-13 20:00:31

Re: SSH Timeout - Write failed: Broken pipe?

#21 2016-03-14 01:57:32

Re: SSH Timeout - Write failed: Broken pipe?

Board footer