You are not logged in.
- Topics: Active | Unanswered
#1 2019-11-26 05:00:25
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
[SOLVED] ssh works locally, times out over internet
I've been working on setting up an ssh server on my desktop, and I want to be able to ssh into it from my laptop over the internet.
Everything is working when I do the following from my laptop on my local network:
ssh -p <ssh-port> -i <private-key> user@<local-network-ip-of-desktop>To get it exposed on the internet, I went into my router's settings to obtain its current public IP address, and have set up Dynamic DNS to assign a domain name to this router's dynamic IP. I have also set up port forwarding for <ssh-port>, and have ensured that my desktop (my ssh server) has a fixed IP on the local network, so that the port forwarding should always be pushed to the correct machine (desktop/ssh server).
If I try to run the ssh command to either the hard-coded public IP of the router or the dynamic DNS name for the router, I get a connection timeout, though.
Does anyone have any idea why this might be happening and/or any possible solutions? Thanks 
Last edited by hoWlExat (2019-12-02 06:52:42)
Offline
#2 2019-11-26 05:24:01
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
Run `tcpdump -lnnieth0 port 22` on the server while you try to connect from the outside (replace eth0 with your actual network interface).
If you can see the packets there, then your problem is on the box. If the packets don't reach there, then the problem upstream (your router/port forwarding/ISP).
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#3 2019-11-26 05:35:42
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
@fukawi2, I am seeing lines pop up of the following form:
me@laptop $ tcpdump -lnni<network-interface> port <ssh-port>
<time> IP <some-ip> > <local-network-ip-of-desktop>.<ssh-port> Flags [S], seq ..., win 29200, options [mss 1104,sackOK,TS val ... ecr 0,nop,wscale 7], length 0I don't know what would be going wrong, though, if it reaches my machine, since I can ssh into it over the local network just fine.
Last edited by hoWlExat (2019-11-26 05:37:09)
Offline
#4 2019-11-26 05:37:55
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
That's the only output?
What is the output of `iptables-save` and/or `nft list ruleset`?
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#5 2019-11-26 05:53:27
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
There are some chains showing up there that I'd defined before, but I don't know how to get rid of them (LOG_AND_DROP, STATE0, STATE1, STATE2, STATE3). Those were from when I was trying to set up port knocking, but that's going to be my next step. For the purposes of this current question, we can assume it's fine to blow those away (but I don't know how, currently).
Remember that I can ssh in successfully over the local network (I don't know if that applies here). Thank you for the help!
me@laptop $ iptables-save
# Generated by iptables-save v1.8.3 on <date>
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]
:LOG_AND_DROP - [0:0]
:STATE0 - [0:0]
:STATE1 - [0:0]
:STATE2 - [0:0]
:STATE3 - [0:0]
:xvpn - [0:0]
:xvpn_dns - [0:0]
:xvpn_dns_iface_exceptions - [0:0]
:xvpn_dns_ip_exceptions - [0:0]
:xvpn_ks - [0:0]
:xvpn_ks_iface_exceptions - [0:0]
:xvpn_ks_ip_exceptions - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport <ssh-port> -m state --state NEW -m recent --update --seconds 60 --hitcount 4 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
-A INPUT -p tcp -m tcp --dport <ssh-port> -m state --state NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -p tcp -m tcp --dport <ssh-port> -m state --state NEW -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
-A INPUT -j DROP
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
COMMIT
# Completed on <date>Last edited by hoWlExat (2019-11-26 05:54:03)
Offline
#6 2019-11-26 06:03:11
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
I don't see anything there at a quick glance that looks like it would interfere - but just to eliminate it, can you run these commands and try again?
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -FAre you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#7 2019-11-26 06:07:59
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
If I do that, then I get the same output as before with `tcpdump` when trying to ssh over the internet.
My `iptables-save` output is updated to:
# Generated by iptables-save v1.8.3 on <date>
*filter
:INPUT ACCEPT [5:2385]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [5:2385]
:LOG_AND_DROP - [0:0]
:STATE0 - [0:0]
:STATE1 - [0:0]
:STATE2 - [0:0]
:STATE3 - [0:0]
:xvpn - [0:0]
:xvpn_dns - [0:0]
:xvpn_dns_iface_exceptions - [0:0]
:xvpn_dns_ip_exceptions - [0:0]
:xvpn_ks - [0:0]
:xvpn_ks_iface_exceptions - [0:0]
:xvpn_ks_ip_exceptions - [0:0]
COMMIT
# Completed on <date>Last edited by hoWlExat (2019-11-26 06:08:42)
Offline
#8 2019-11-26 06:12:57
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
What's the output of `ss -tunlp`?
And it would be helpful to see the uncensored output of tcpdump along with that if you can.
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#9 2019-11-26 06:12:58
- bulletmark
- Member
- From: Brisbane, Australia
- Registered: 2013-10-22
- Posts: 681
Re: [SOLVED] ssh works locally, times out over internet
Are you running the ssh test from a client from within the same local LAN you are trying to forward back to? Some home routers allow this, some don't, and yet some others require a special setting on the port forward to allow it. My router calls the setting "Enable LAN loopback".
Offline
#10 2019-11-26 06:26:16
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
@fukawi2
me@laptop $ ss -tunlp
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN 0 0 192.168.1.12:68 0.0.0.0:* users:(("dhcpcd",pid=872,fd=11))
udp UNCONN 0 0 192.168.1.12%enp7s0:68 0.0.0.0:* users:(("NetworkManager",pid=788,fd=18))
udp UNCONN 0 0 10.127.0.62:123 0.0.0.0:* users:(("ntpd",pid=878,fd=20))
udp UNCONN 0 0 192.168.1.12:123 0.0.0.0:* users:(("ntpd",pid=878,fd=19))
udp UNCONN 0 0 127.0.0.1:123 0.0.0.0:* users:(("ntpd",pid=878,fd=18))
udp UNCONN 0 0 0.0.0.0:123 0.0.0.0:* users:(("ntpd",pid=878,fd=17))
udp UNCONN 0 0 0.0.0.0:40610 0.0.0.0:* users:(("openvpn",pid=1173,fd=4))
udp UNCONN 0 0 [::]:123 [::]:* users:(("ntpd",pid=878,fd=16))
tcp LISTEN 0 128 127.0.0.1:9150 0.0.0.0:* users:(("tor",pid=3497,fd=6))
tcp LISTEN 0 128 127.0.0.1:9151 0.0.0.0:* users:(("tor",pid=3497,fd=7))
tcp LISTEN 0 128 127.0.0.1:34463 0.0.0.0:* users:(("expressvpn-agen",pid=1441,fd=7))
tcp LISTEN 0 128 127.0.0.1:2015 0.0.0.0:* users:(("expressvpnd",pid=907,fd=7))
tcp LISTEN 0 128 0.0.0.0:<ssh-port> 0.0.0.0:* users:(("sshd",pid=875,fd=3))
tcp LISTEN 0 1 127.0.0.1:44391 0.0.0.0:* users:(("openvpn",pid=1173,fd=3))
tcp LISTEN 0 128 [::]:<ssh-port> [::]:* users:(("sshd",pid=875,fd=4)) In the below block, 192.168.1.12 is equivalent to <local-network-ip-of-desktop>:
me@laptop $ tcpdump -lnnienp7s0 port <ssh-port>
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp7s0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:19:43.611542 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353520933 ecr 0,nop,wscale 7], length 0
06:19:44.617505 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353521938 ecr 0,nop,wscale 7], length 0
06:19:46.629511 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353523950 ecr 0,nop,wscale 7], length 0
06:19:50.885805 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353528206 ecr 0,nop,wscale 7], length 0
06:19:59.078590 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353536398 ecr 0,nop,wscale 7], length 0
06:20:15.206988 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353552526 ecr 0,nop,wscale 7], length 0
06:20:47.462208 IP 173.239.199.183.34340 > 192.168.1.12.<ssh-port>: Flags [S], seq 511406441, win 29200, options [mss 1104,sackOK,TS val 2353584782 ecr 0,nop,wscale 7], length 0
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel@bulletmark
Yes, I am running the ssh test from my laptop on the same local LAN that my desktop resides on. My router is a Netgear R6220, and I'm not seeing an "Enable LAN loopback" setting, initally. I don't know if NAT loopback is the same, but this page says my router doesn't have a NAT loopback feature.
Offline
#11 2019-11-26 06:32:31
- bulletmark
- Member
- From: Brisbane, Australia
- Registered: 2013-10-22
- Posts: 681
Re: [SOLVED] ssh works locally, times out over internet
Try running your ssh test from an external site. The forward may actually be working. It only costs a few cents and a minute to spin up a VPS (e.g. Ubuntu or Debian) to try this, e.g. on DigitalOcean or Vultr.
Last edited by bulletmark (2019-11-26 06:33:05)
Offline
#12 2019-11-26 06:34:31
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
What is the output of `ip r s`?
My router calls the setting "Enable LAN loopback".
Good thought - I didn't think of that. The tcpdump confirms that the packet is coming to the desktop from the public IP address, and the desktop just isn't responding with the expected SYN ACK packet though.
I'm running out of ideas here - the desktop is seeing the incoming SYN packet, but isn't responding for whatever reason. Usually it's because of iptables. Are you positive you don't have any nftable rules?
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#13 2019-11-26 06:37:53
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
@fukawi2
me@laptop $ ip r s
0.0.0.0/1 via 10.127.0.61 dev tun0
default via 192.168.1.1 dev enp7s0 proto dhcp src 192.168.1.12 metric 202
10.0.0.0/8 via 192.168.1.1 dev enp7s0
10.127.0.1 via 10.127.0.61 dev tun0
10.127.0.61 dev tun0 proto kernel scope link src 10.127.0.62
45.135.184.11 via 192.168.1.1 dev enp7s0
128.0.0.0/1 via 10.127.0.61 dev tun0
172.16.0.0/12 via 192.168.1.1 dev enp7s0
192.168.0.0/16 via 192.168.1.1 dev enp7s0
192.168.1.0/24 dev enp7s0 proto kernel scope link src 192.168.1.12 metric 100
192.168.1.0/24 dev enp7s0 proto dhcp scope link src 192.168.1.12 metric 202@bulletmark
I'll try a VPS!
Offline
#14 2019-11-26 07:07:40
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
You have a VPN setup - it's routing the return traffic via the VPN which then gets dropped by their firewall probably.
Turn your VPN off and try again.
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#15 2019-11-26 07:11:44
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
@fukawi2 @bulletmark I get the same `tcpdump` results on my ssh server when trying to ssh in from a VPS...
And I've tried with and without my VPN enabled on my laptop. Same `tcpdump` results.
Last edited by hoWlExat (2019-11-26 07:13:59)
Offline
#16 2019-11-26 23:48:52
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
What is the output of `ip r s` after disabling the VPN?
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#17 2019-11-28 08:02:15
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
@fukawi2, not sure if you were asking for output on desktop (server) or laptop (client), so I've included before and afters for both.
me@desktop $ ip r s
0.0.0.0/1 via 10.127.0.21 dev tun0
default via 192.168.1.1 dev enp7s0 proto dhcp src 192.168.1.12 metric 202
10.0.0.0/8 via 192.168.1.1 dev enp7s0
10.127.0.1 via 10.127.0.21 dev tun0
10.127.0.21 dev tun0 proto kernel scope link src 10.127.0.22
45.135.184.11 via 192.168.1.1 dev enp7s0
128.0.0.0/1 via 10.127.0.21 dev tun0
172.16.0.0/12 via 192.168.1.1 dev enp7s0
192.168.0.0/16 via 192.168.1.1 dev enp7s0
192.168.1.0/24 dev enp7s0 proto kernel scope link src 192.168.1.12 metric 100
192.168.1.0/24 dev enp7s0 proto dhcp scope link src 192.168.1.12 metric 202
me@desktop $ expressvpn disconnect
Disconnecting...
Disconnected
me@desktop $ ip r s
default via 192.168.1.1 dev enp7s0 proto dhcp src 192.168.1.12 metric 202
192.168.1.0/24 dev enp7s0 proto kernel scope link src 192.168.1.12 metric 100
192.168.1.0/24 dev enp7s0 proto dhcp scope link src 192.168.1.12 metric 202
#########################################
me@laptop $ ip r s
0.0.0.0/1 via 10.22.0.201 dev tun0
default via 192.168.1.1 dev wlp110s0 proto dhcp metric 600
10.0.0.0/8 via 192.168.1.1 dev wlp110s0
10.22.0.1 via 10.22.0.201 dev tun0
10.22.0.201 dev tun0 proto kernel scope link src 10.22.0.202
128.0.0.0/1 via 10.22.0.201 dev tun0
169.254.0.0/16 dev docker0 scope link metric 1000 linkdown
172.16.0.0/12 via 192.168.1.1 dev wlp110s0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
173.214.168.253 via 192.168.1.1 dev wlp110s0
192.168.0.0/16 via 192.168.1.1 dev wlp110s0
192.168.1.0/24 dev wlp110s0 proto kernel scope link src 192.168.1.16 metric 600
me@laptop $ expressvpn disconnect
Disconnecting...
Disconnected.
me@laptop $ ip r s
default via 192.168.1.1 dev wlp110s0 proto dhcp metric 600
169.254.0.0/16 dev docker0 scope link metric 1000 linkdown
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.1.0/24 dev wlp110s0 proto kernel scope link src 192.168.1.42 metric 600Last edited by hoWlExat (2019-11-30 02:38:57)
Offline
#18 2019-11-28 08:12:42
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
EDIT: sorry, on mobile; misread your last post.
Your laptop appears to have no networking at all after turning off the VPN?
Last edited by fukawi2 (2019-11-28 08:14:44)
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#19 2019-11-30 02:40:21
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
Ah, that was because my VPN had a network lock on when it was disabled. I tried it with the network lock disabled, and I got the same results. That was expected, given it didn't work via the VPS, either.
I updated my previous message with the `ip r s` results on my laptop with the network lock disabled.
Last edited by hoWlExat (2019-11-30 02:40:39)
Offline
#20 2019-11-30 06:46:31
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
Alright, @fukawi2, I have some exciting news! I figured out what was causing the issue, but I don't yet understand why. I'm wondering if you might have some insight, here.
## Solution
I had a VPN enabled on *both* my laptop and my desktop. I had forgotten that the VPN was enabled on my desktop. As soon as I disconnected from the VPN on my desktop, the ssh from the laptop over the internet to the desktop worked! And it worked both when my laptop was connected and disconnected from its VPN.
## Follow up
To wrap this up, would you be able to help me troubleshoot why the VPN on the desktop was preventing ssh from working? Below are the results of executing `iptables -L` on the desktop after having completely opened up my firewall and enabling the VPN:
me@desktop $ iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
xvpn all -- anywhere anywhere
Chain xvpn (1 references)
target prot opt source destination
xvpn_dns all -- anywhere anywhere
xvpn_ks all -- anywhere anywhere
Chain xvpn_dns (1 references)
target prot opt source destination
xvpn_dns_iface_exceptions all -- anywhere anywhere
xvpn_dns_ip_exceptions all -- anywhere anywhere
DROP udp -- anywhere anywhere udp dpt:domain
Chain xvpn_dns_iface_exceptions (1 references)
target prot opt source destination
Chain xvpn_dns_ip_exceptions (1 references)
target prot opt source destination
ACCEPT udp -- anywhere 10.127.0.1 udp dpt:domain
Chain xvpn_ks (1 references)
target prot opt source destination
xvpn_ks_iface_exceptions all -- anywhere anywhere
xvpn_ks_ip_exceptions all -- anywhere anywhere
ACCEPT udp -- anywhere anywhere udp dpts:bootps:bootpc
DROP all -- anywhere anywhere
Chain xvpn_ks_iface_exceptions (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere
Chain xvpn_ks_ip_exceptions (1 references)
target prot opt source destination
ACCEPT all -- anywhere 10.0.0.0/8
ACCEPT all -- anywhere 172.16.0.0/12
ACCEPT all -- anywhere 192.168.0.0/16
ACCEPT all -- anywhere 169.254.0.0/16
ACCEPT all -- anywhere base-address.mcast.net/24
ACCEPT all -- anywhere 45.135.184.11Last edited by hoWlExat (2019-11-30 06:53:55)
Offline
#21 2019-12-01 23:12:28
- fukawi2
- Ex-Administratorino
- From: .vic.au
- Registered: 2007-09-28
- Posts: 6,231
- Website
Re: [SOLVED] ssh works locally, times out over internet
With the VPN enabled on your desktop, the return packets (from desktop to laptop) are being routed via the VPN. This causes asynchronous routing. It's kind of like someone ringing your doorbell on the front door of your house, but you open the back door. You're both equally confused because neither of you are getting the result you expect from the doorbell.
You'll need to setup policy-based routing to make return packets be sent via the interface they were received on. This article should be able to help you: https://blog.scottlowe.org/2013/05/29/a … y-routing/
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
#22 2019-12-02 06:52:08
- hoWlExat
- Member
- Registered: 2019-11-15
- Posts: 35
Re: [SOLVED] ssh works locally, times out over internet
That makes sense. Since my networking skills/knowledge is lackluster, and I don't understand half of the jargon/concepts, I'll ask a separate question in this room about that, while I try to learn what I can in my free time. I appreciate all the help! Thanks!
Offline