You are not logged in.

#1 2021-11-24 14:33:52

archuser1
Member
Registered: 2021-11-24
Posts: 11

[SOLVED]Suddenly can't connect to my local webserver from same network

All of a sudden I can't connect from my smartphone to my local apache webserver. Both are in the same network: 192.168.0.99 (smartphone) <-> 192.168.0.100 (apache pc). The apache webserver is running and working as I can connect to it from my local pc, where it is running on. I can ping the IP from my smartphone. The only thing I can think of which I did "change", was adding/setting up QEMU/KVM, no change to apache conf files.

For QEMU/KVM I think I installed

 libvirt virt-manager qemu bridge-utils iptables-nft dnsmasq

, so maybe that somehow screwed and blocked all incoming traffic to my pc?

Searching around, several times I read that maybe a firewall is blocking connections, but I don't see a firewall running. The only one I can think of is iptables, and its rules are empty:

sudo iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Also, iptables seems not to be running, which should not block incoming traffic?:

systemctl status iptables
○ iptables.service - IPv4 Packet Filtering Framework
     Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled; vendor preset: disabled)
     Active: inactive (dead)

Another thing I did was setting up a network bridge for the VM, but I'm not sure how this can block incoming traffic. I also removed the network bridge in hope that this is the reason for the blocking, but unfortunately no change.

To rule out apache configuration issue, I run sudo python -m http.server 8080 and tried to connect to it, but unfortunately still can't connect from external devices.

ss -antu shows apache running (I changed from "Listen 80" to "Listen 0.0.0.0:80" in hope that IPv6 somehow was the reason, but unfortunately still can't connect from external devices):

tcp       LISTEN     0          511                            0.0.0.0:80                 0.0.0.0:*                    

Last edited by archuser1 (2021-11-26 09:59:43)

Offline

#2 2021-11-24 15:10:43

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

nft list tables # eg. firewalld doesn't operate on the iptables layer

I can't connect … apache webserver.

I can ping the IP

How does the connection failure manifest? Do you get an http error status?
Can you "nmap 192.168.0.100" from the phone? ( https://secwiki.org/w/Nmap/Android and there's a bunch of network tools wrapping nmap in the playstores)

Offline

#3 2021-11-24 20:43:19

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

sudo nft list tables
table inet filter

Firefox gives me an error like the IP would be wrong (same error if I e.g. try .111):

Unable to connect

An error occurred during a connection to 192.168.0.100. 

nmap 192.168.0.100 took almost 3h to complete and showed nothing regarding the first 999 ports (only 2 upper ports are shown as "filtered"). If I use nmap  -p 80 192.168.0.100 I get instantly

80/tcp closed http

. (I used Termux and it supports installing nmap in it)

Offline

#4 2021-11-24 21:00:46

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

ip a; ip r

When you run

sudo tcpdump -i eth0 icmp # replace eth0 w/ the relevant NIC

and then ping the server, are those events logged by tcpdump?

Offline

#5 2021-11-25 19:05:01

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

ip a; ip r gives me:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master nm-bridge state UP group default qlen 1000
    link/ether 2c:f0:5d:d9:8a:72 brd ff:ff:ff:ff:ff:ff
3: nm-bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ce:e3:a1:e4:83:5d brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute nm-bridge
       valid_lft forever preferred_lft forever
    inet6 2003:c0:3735:8a00:8455:4d25:e094:1eed/64 scope global dynamic noprefixroute 
       valid_lft 7171sec preferred_lft 1771sec
    inet6 fe80::4762:db96:cfe3:4e6f/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
default via 192.168.0.1 dev nm-bridge proto static metric 425 
192.168.0.0/24 dev nm-bridge proto kernel scope link src 192.168.0.100 metric 425 

tcpdump gives me:

19:51:29.930853 IP SMARTPHONENAME.fritz.box > test-pc: ICMP echo request, id 2, seq 1, length 64
19:51:29.930870 IP test-pc > SMARTPHONENAME.fritz.box: ICMP echo reply, id 2, seq 1, length 64
(these two lines are repeating every ping second then)

I can also ping my pc from my local VM, but get the same "Unable to connect" error message when I try to browse the webserver.

I tried to browse the webserver while running the same tcpdump command and I get:

19:57:07.838101 IP test-pc > SMARTPHONENAME.fritz.box: ICMP host test-pc unreachable - admin prohibited filter, length 68
19:57:15.129332 IP test-pc > _gateway: ICMP host test-pc unreachable - admin prohibited filter, length 68

Last edited by archuser1 (2021-11-25 19:06:00)

Offline

#6 2021-11-25 21:33:10

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

So 192.168.0.100 is the bridge and "admin prohibited filter" smells like firewall(d), https://unix.stackexchange.com/question … g-rejected
Likely because your trying to reach the bridged segment (The VM?)

Offline

#7 2021-11-26 08:15:53

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

With the smartphone I'm trying to reach my local pc (192.168.0.100) where the apache webserver is running. Later I setup the VM on my local pc for not webserver related stuff (no webserver is running on the VM). (but maybe installing the VM related software screwed and installed some software running firewall, except I haven't found anything so far.)

I checked for firewalld before creating my initial post and it seems not to be running:

$ systemctl status firewalld.service 
Unit firewalld.service could not be found.

It says "..could not be found", but gets auto-completed when I type "systemctl status firew" hit the tab key, so maybe it was once installed, removed but its rules are still being applied by some program? But in general I also suspect a local software firewall thing.

Locating firewalld:

$ locate firewalld
/usr/lib/firewalld
/usr/lib/firewalld/zones
/usr/lib/firewalld/zones/libvirt.xml
/usr/lib/firewalld/zones/nm-shared.xml
/var/log/firewalld
$ grep -i "firewall\|nftables" /var/log/pacman.log 
[...]
[PACMAN] Running 'pacman -R firewalld'
[ALPM] removed firewalld (1.0.2-1)
[ALPM] upgraded nftables (1:1.0.0-1 -> 1:1.0.1-1)

firewalld was indeed once installed and later removed by me, because I read here wiki.archlinux.org/title/libvirt that it's not required.

I grepped for nftables because it's part of the installed nftables-nft and I suspect that it may be blocking traffic:

sudo nft list ruleset
table inet filter {
	chain input {
		type filter hook input priority filter; policy drop;
		ct state invalid drop comment "early drop of invalid connections"
		ct state { established, related } accept comment "allow tracked connections"
		iifname "lo" accept comment "allow from loopback"
		ip protocol icmp accept comment "allow icmp"
		meta l4proto ipv6-icmp accept comment "allow icmp v6"
		tcp dport 22 accept comment "allow sshd"
		meta pkttype host limit rate 5/second counter packets 9 bytes 600 reject with icmpx admin-prohibited
		counter packets 26 bytes 4502
	}

	chain forward {
		type filter hook forward priority filter; policy drop;
	}
}

According to "tcp dport 22 accept comment "allow sshd"" port 22 should be open, but "nmap -p 22 192.168.0.100" from my smartphone shows unfortunately "22/tcp closed ssh" and "systemctl status nftables" shows:

systemctl status nftables.service 
○ nftables.service - Netfilter Tables
     Loaded: loaded (/usr/lib/systemd/system/nftables.service; enabled; vendor preset: disabled)
     Active: inactive (dead) since Fri 2021-11-26
       Docs: man:nft(8)
   Main PID: 463 (code=exited, status=0/SUCCESS)
        CPU: 10ms

, so maybe nftables is inactive and is not the reason for the blocking? "systemctl stop nftables.service" unfortunately doesn't help and nmap from smartphone still shows port 80 is closed.

Last edited by archuser1 (2021-11-26 08:31:18)

Offline

#8 2021-11-26 09:58:37

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Success
Searching site:bbs.archlinux.org "admin prohibited filter" there was only one result:
https://bbs.archlinux.org/viewtopic.php?id=271258
  "So I flushed the nftables and now it works:"
So I did: "sudo nft flush ruleset" and it works now! Thanks seth, your tcpdump command helped reveal that it's a firewall issue, I used a different one and in mine there was no "admin prohibited filter" indication, which made me look after the firewalls again.

After I flushed iptables I should have flushed nftables. My suggestion would be making iptables and nftables not installable both at the same time.

In /var/log/pacman.log I can see that

[2021-11-22..] [ALPM] upgraded nftables (1:1.0.0-1 -> 1:1.0.1-1)

so maybe that enabled nftables (nothing suggests that a new rules conf file was added).

ssh works after doing "systemctl start sshd", so it probably wasn't a firewall thing after all, as the rule in my previous post shows its access was enabled.

Last edited by archuser1 (2021-11-26 10:20:23)

Offline

#9 2021-11-26 13:34:25

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Does the "meta pkttype host limit rate 5/second counter packets 9 bytes 600 reject with icmpx admin-prohibited" rule show up again after a reboot?

Offline

#10 2021-11-26 15:19:33

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Good call, after reboot all 9 lines are still there. In iptables the flush is temporary and needs to be saved to be permanent, maybe it's similar with nftables.

Offline

#11 2021-11-26 15:21:25

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

The flush is not, the tables are.
So something™ adds that during the boot and I assume it's the same thing that adds the bridge and I assume it's the VM?

Offline

#12 2021-11-26 16:26:16

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

I tried disabling VM related services but that didn't help. It seems to be a known issue, but I can understand that it's undesirable if "sudo nft flush ruleset" deletes all rules (although there's a protective sudo hurdle). I edited /etc/nftables.conf and removed its contents, no output after "sudo nft list ruleset" after reboot. Maybe the nftables.service adds the rules from the config, as the config stays empty after reboot.

Offline

#13 2021-11-26 21:29:51

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Those rules are actually provided by default by the arch package… hmm
https://github.com/archlinux/svntogit-p … ables.conf

With an empty ruleset, you might as well dump the nftables package.

Offline

#14 2021-11-27 10:05:42

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Yes, I saw they are provided (I implied that it's weird that they became only active after the "[ALPM] upgraded nftables (1:1.0.0-1 -> 1:1.0.1-1)"). Another suggestion I would make is to comment them all out in the provided .conf.

sudo pacman -R nftables
Abhängigkeiten werden geprüft …
Fehler: Vorgang konnte nicht vorbereitet werden (Kann Abhängigkeiten nicht erfüllen)
:: Entfernen von nftables verletzt Abhängigkeit »nftables«, benötigt von iptables-nft

Can't remove it the easy way because iptables-nft depends on it.

Offline

#15 2021-11-27 10:07:55

seth
Member
Registered: 2012-09-03
Posts: 25,134

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Almost nothing directly depends on iptables-nft, you could replace it w/ iptables

Offline

#16 2021-11-27 10:59:18

progandy
Member
Registered: 2012-05-17
Posts: 4,641

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Or you could set up a basic firewall (with nftables or legacy iptables), allow access to the webserver ports similar to the ssh port. The default nftables-nft config is restrictive and does not allow any remote access except ssh and icmp/ping.

Almost nothing directly depends on iptables-nft, you could replace it w/ iptables

There is no real difference between using iptables with an empty ruleset and nftables (and iptables-nft) with an empty ruleset. The only difference is an additional package of about 920KiB and the kernel modules used for handling the rules.

Last edited by progandy (2021-11-27 11:04:24)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#17 2021-12-04 14:09:13

archuser1
Member
Registered: 2021-11-24
Posts: 11

Re: [SOLVED]Suddenly can't connect to my local webserver from same network

Removing iptables-nft didn't work because it said it depends on systemd and one other thing (doesnt mention systemd when I do pacman -Si iptables-nft after it's removed), but replacing is using sudo pacman -S iptables works indeed: I thought iptables-nft was required for VM's internet access. Then sudo pacman -S nftables also worked. VM also still has internet access working because it's received through a bridged network. (But for NAT/DHCP networking, iptables-nft, and dnsmasq may still be required: https://wiki.archlinux.org/title/libvirt)

Last edited by archuser1 (2021-12-04 14:10:38)

Offline

Board footer

Powered by FluxBB