[SOLVED] Device server unreachable on local network

atlimar · 2024-10-22 12:48:33

I'm unable to access a server running on the Arch device on my local network.

I can ping the Arch device on the local network (192.168.1.254).
The Arch device can see and ping other devices on the network (e.g. my phone at 192.168.1.179).
I can SSH onto the Arch device from my phone on the local network.

I cannot access the Arch device on, for example, http://192.168.1.254:9000 (or any other port) while having a server running. The Arch device itself can resolve http://192.168.1.254:9000.

I have nftables and iptables-nft installed to manage ports, with ufw as frontend. I went a bit overboard trying to configure ufw in an attempt to get this working, haven't deleted any rules yet:

➜ sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW       192.168.1.0/24
1234                       ALLOW       192.168.1.0/24
192.168.1.179 9000         ALLOW       192.168.1.0/24
192.168.1.0/24 9000        ALLOW       192.168.1.179

ip a;ip r

➜ ip a; ip r
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether b4:0e:de:28:ae:b5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.254/24 metric 600 brd 192.168.1.255 scope global dynamic wlan0
       valid_lft 71428sec preferred_lft 71428sec
    inet6 fe80::b60e:deff:fe28:aeb5/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:a1:8b:0e:ee brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev wlan0 proto dhcp src 192.168.1.254 metric 600
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.254 metric 600
192.168.1.1 dev wlan0 proto dhcp scope link src 192.168.1.254 metric 600

➜ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

avahi-daemon.service                     | multi-user.target.wants
avahi-daemon.socket                      | sockets.target.wants
bluetooth.service                        | bluetooth.target.wants
dbus-org.bluez.service                   | system
dbus-org.freedesktop.Avahi.service       | system
dbus-org.freedesktop.network1.service    | system
dbus-org.freedesktop.resolve1.service    | system
dbus-org.freedesktop.timesync1.service   | system
display-manager.service                  | system
docker.service                           | multi-user.target.wants
fstrim.timer                             | timers.target.wants
getty@tty1.service                       | getty.target.wants
iwd.service                              | multi-user.target.wants
mongodb.service                          | multi-user.target.wants
nftables.service                         | multi-user.target.wants
nvidia-hibernate.service                 | systemd-hibernate.service.wants
nvidia-resume.service                    | systemd-hibernate.service.wants
nvidia-resume.service                    | systemd-suspend.service.wants
nvidia-suspend.service                   | systemd-suspend.service.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire-pulse.socket                    | sockets.target.wants
pipewire-session-manager.service         | user
pipewire.socket                          | sockets.target.wants
power-profiles-daemon.service            | graphical.target.wants
remote-fs.target                         | multi-user.target.wants
supergfxd.service                        | getty.target.wants
switcheroo-control.service               | graphical.target.wants
systemd-networkd.service                 | multi-user.target.wants
systemd-networkd.socket                  | sockets.target.wants
systemd-networkd-wait-online.service     | network-online.target.wants
systemd-network-generator.service        | sysinit.target.wants
systemd-resolved.service                 | sysinit.target.wants
systemd-timesyncd.service                | sysinit.target.wants
ufw.service                              | multi-user.target.wants
wireplumber.service                      | pipewire.service.wants

I have no idea how to proceed, or what the next step to try is. I've heard this is supposed to just work out of the box (no ports are closed on a clean Arch install), so I assume I must have installed or done something at some point that completely blocked this off, and I'm unsure what.

Last edited by atlimar (2024-10-28 18:42:44)

seth · 2024-10-22 14:01:00

nmap 192.168.1.254

nftables.service                         | multi-user.target.wants

https://wiki.archlinux.org/title/Nftabl … e_firewall

atlimar · 2024-10-22 15:02:07

➜ nmap 192.168.1.254
Starting Nmap 7.95 ( https://nmap.org ) at 2024-10-22 16:59 CEST
Nmap scan report for dragnipur (192.168.1.254)
Host is up (0.00017s latency).
Not shown: 997 closed tcp ports (conn-refused)
PORT     STATE SERVICE
22/tcp   open  ssh
1234/tcp open  hotline
9000/tcp open  cslistener

Nmap done: 1 IP address (1 host up) scanned in 0.05 seconds

I'm a bit unsure what you want me to do with the nftables wiki link

seth · 2024-10-22 15:06:09

Did you run nmap on the same host? Cause 9000 is open.
The nftables package comes with a predefined set of rules that are loaded by that service.
If you've not changed them, they'll probably kill the traffic (and just looking at the ufw tables won't tell you that)

Trilby · 2024-10-22 15:22:20

atlimar wrote:

I cannot access the Arch device on, for example, http://192.168.1.254:9000

What exactly do you mean you cannot access it? How do you try to access it? What is the command and output for / from such attempts?

What service is listening on port 9000? Is this php{,-fpm}? If so, is the service itself configured to accept connections from other than the localhost?

atlimar · 2024-10-22 15:40:02

The servers at port 1234 and 9000 are webpack dev servers (http file server for javascript development). They are both hosting react apps.

I can open them in the browser on the Arch machine, but not from any other machine on the same local network. If I open the url in e.g. Firefox on my phone it says "Unable to connect".

Using termux on the phone to ping the Arch device works fine.

The server itself is configured to be accessible.

Last edited by atlimar (2024-10-22 16:04:33)

atlimar · 2024-10-22 16:00:46

seth wrote:

Did you run nmap on the same host? Cause 9000 is open.
The nftables package comes with a predefined set of rules that are loaded by that service.
If you've not changed them, they'll probably kill the traffic (and just looking at the ufw tables won't tell you that)

Yes, I ran it on the same host. I agree, by the looks of it, and with the settings I have applied on the firewall, it should be open. Still not accessible from another device on the same local network.

If you check the opening post you can see nftables is configured via ufw, to make sure all ports (as well as 9000/1234 specifically) are accessible on the local network. No settings done via ufw seem to have an effect on the problem.

Trilby · 2024-10-22 16:54:32

Test with some simpler server like a netcat implementation running "echo Connected" or similar on each incoming connection on port 9000 (after disabling these webpack servers). Then test telnet (or similar) from the other system. This will confirm whether or not this is a networking / firewall issue or a server software issue. If it is a server software issue, check if the server(s) have logs and if so check what they are reporting.

EDIT: these appear to use node as the actual server, is this correct? How have you configured or how are you starting the actual node server? What is the output from the node server itself? I believe the first line should be a confirmation of what IP and port it is listening on.

Last edited by Trilby (2024-10-22 17:06:42)

atlimar · 2024-10-22 17:27:30

Yes, these are node servers from a git repo for developing a js sdk. The servers are used by multiple developers in the same team and work as expected on other devices (the node server is accessible on the local network to other devices). I have tried running them on other devices myself and it works as expected.

The server script outputs the ip and port that the server is available on (localhost as well as the 192 ip). Both addresses work fine on the Arch device itself. I am certain the server itself works as expected, but there is something around it in the operating system making sure it isn't accessible from the outside.

As per the opening post I have tried pinging my Arch device (works) and I have tried to ssh onto the device via port 22 (works). I'll have to run more tests with other servers next time I'm at the laptop, thanks for the tip.

I'm purely guessing, but I'm wondering if there could be a conflict between avahi and systemd-resolved, or if I have more than one firewall somehow closing down every tcp port in the device and I'm only managing to configure one of them. Or if I only think I'm configuring nftables, but in actuality I'm not... or something else that I'm just not aware of. Everywhere I look it looks like the ports are open and accessible, with a server correctly listening.

AFAIK Arch, when freshly installed, blocks nothing. I have not, to my knowledge, explicitly tried to block off access to the arch device.

Last edited by atlimar (2024-10-22 17:29:00)

atlimar · 2024-10-22 17:33:49

seth wrote:

The nftables package comes with a predefined set of rules that are loaded by that service.
If you've not changed them, they'll probably kill the traffic (and just looking at the ufw tables won't tell you that)

I will look into this, I believed using ufw as the frontend for configuring nftables would achieve this, but now I guess maybe not!

ewaller · 2024-10-22 17:58:41

Can you connect to other machines on the net besides .254?

I ask because I had a strange problem this week in which ARP stuff at the link layer worked (ping uses ARP), but higher level protocols like USP and TCP failed. It was a bad cable, I have no clue how that could happen.

Also, what happens if you stop all the firewalls on your system?

atlimar · 2024-10-22 18:04:51

ewaller wrote:

Can you connect to other machines on the net besides .254?

From the Arch device? I believe so, but I'll test to make sure asap.

ewaller wrote:

I ask because I had a strange problem this week in which ARP stuff at the link layer worked (ping uses ARP), but higher level protocols like USP and TCP failed. It was a bad cable, I have no clue how that could happen.

For this particular case I'm on wifi, so it should not be the exact same issue.

ewaller wrote:

Also, what happens if you stop all the firewalls on your system?

As far as I know, I only have nftables. I installed iptables-nft and ufw today to simplify configuration of nftables. I've tried stopping and disabling nftables with no result.

seth · 2024-10-22 18:05:14

atlimar wrote:

I will look into this, I believed using ufw as the frontend for configuring nftables would achieve this, but now I guess maybe not!

netfilter happens in the kernel and can be controlled from multiple userspace tools at the same time (not a good idea, but possible)
=> disable the netfilter.service and reboot if you want to use ufw.

atlimar · 2024-10-23 07:12:45

seth wrote:

atlimar wrote:
I will look into this, I believed using ufw as the frontend for configuring nftables would achieve this, but now I guess maybe not!
netfilter happens in the kernel and can be controlled from multiple userspace tools at the same time (not a good idea, but possible)
=> disable the netfilter.service and reboot if you want to use ufw.

I have a list in the opening post of running services on my system, but I checked just to make sure:

➜ sudo systemctl status netfilter
Unit netfilter.service could not be found.

seth · 2024-10-23 07:15:20

Sorry,

nftables.service | multi-user.target.wants

atlimar · 2024-10-23 07:39:44

seth wrote:

Sorry,
nftables.service | multi-user.target.wants

I don't think I should do this, ufw is a frontend for managing a netfilter, like nftables (or iptables). You can read about frontends for nftables on the wiki link you posted above.

I will be disabling ufw and nftables to test if I can access dev servers from my Arch device, but I don't think I can disable nftables and expect ufw to work.

Here is the ACCEPT output for iptables-nft:

 sudo iptables -S | grep ACCEPT

-P OUTPUT ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A ufw-before-forward -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-input -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A ufw-before-input -d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT
-A ufw-before-input -d 239.255.255.250/32 -p udp -m udp --dport 1900 -j ACCEPT
-A ufw-before-output -o lo -j ACCEPT
-A ufw-before-output -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-skip-to-policy-output -j ACCEPT
-A ufw-track-output -p tcp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-track-output -p udp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-user-input -s 192.168.1.0/24 -j ACCEPT
-A ufw-user-input -s 192.168.1.0/24 -p tcp -m tcp --dport 1234 -j ACCEPT
-A ufw-user-input -s 192.168.1.0/24 -p udp -m udp --dport 1234 -j ACCEPT
-A ufw-user-input -s 192.168.1.0/24 -d 192.168.1.179/32 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ufw-user-input -s 192.168.1.0/24 -d 192.168.1.179/32 -p udp -m udp --dport 9000 -j ACCEPT
-A ufw-user-input -s 192.168.1.179/32 -d 192.168.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ufw-user-input -s 192.168.1.179/32 -d 192.168.1.0/24 -p udp -m udp --dport 9000 -j ACCEPT
-A ufw-user-limit-accept -j ACCEPT

The IP ranges at the bottom are all configured via ufw, like so:

sudo ufw allow from 192.168.1.0/24 to any port 1234

I could also post the nftables ruleset, but it's a lot more verbose.

seth · 2024-10-23 07:45:07

I don't think I should do this

Don't thing do what?
Disable that service.

I'll read the previously linked wiki to:

https://wiki.archlinux.org/title/Nftables#Simple_firewall wrote:

Simple firewall
nftables comes with a simple and secure firewall configuration stored in the /etc/nftables.conf file.
The nftables.service will load rules from that file when started or enabled.

What do you think https://gitlab.archlinux.org/archlinux/ … type=heads does?

atTheShikhar · 2024-10-23 07:46:20

I am also facing a similar issue.
I develop using FastAPI(python framework) on archlinux. the server if running fine and the routes are accessible locally in my laptop.
but i'm not able to access the server from my phone, even when both the devices are connected to the same network. Earlier it used to work. Sorry i'm a newbie.

atlimar · 2024-10-23 08:24:53

seth wrote:

I don't think I should do this
Don't thing do what?
Disable that service.
I'll read the previously linked wiki to:
https://wiki.archlinux.org/title/Nftables#Simple_firewall wrote:
Simple firewall
nftables comes with a simple and secure firewall configuration stored in the /etc/nftables.conf file.
The nftables.service will load rules from that file when started or enabled.
What do you think https://gitlab.archlinux.org/archlinux/ … type=heads does?

It has some default rules when freshly installed, which I have used ufw to override. I can post the full nftables rules that I have if you'd like to have a look at the current setup. Either way, I will be disabling them both to try if I can access the device from the outside. As I mentioned previously, I have tried disabling the nftables service without success

atlimar · 2024-10-23 08:26:02

atTheShikhar wrote:

I am also facing a similar issue.
I develop using FastAPI(python framework) on archlinux. the server if running fine and the routes are accessible locally in my laptop.
but i'm not able to access the server from my phone, even when both the devices are connected to the same network. Earlier it used to work. Sorry i'm a newbie.

Do you have iptables, nftables, firewalld, or ufw installed?

Could you post the list of your running services? (see the command in my opening post)

seth · 2024-10-23 12:27:27

which I have used ufw to override

No.

As I mentioned previously, I have tried disabling the nftables service without success

This will only have any effect after a reboot.

atlimar · 2024-10-28 09:11:33

Finally have the device in an environment where I can test local network access again. Thanks seth, you were right all along. Disabling nftables entirely made access work again.

From the wiki I understand that if I want a firewall I should keep nftables running, since ufw is a frontend for the nftables netfilter. I want to make sure I do this correctly.

I have the default nft configuration in /etc/nftables.conf, which appears to be the one that's blocking everything.

If I run "nft list ruleset" I get a separate list of configuration, but this does not seem to be related to the default config loaded above.

I do want the default policy to be to block/drop every request except ones from the local network (or approved IPs). I believe this is how I have configured nftables via ufw, is it then safe to remove the default config in /etc/nftables.conf?

seth · 2024-10-28 13:06:31

I should keep nftables running

No.
Again: netfilter happens completely in the kernel.
That service has *nothing* to do with you being able to use ufw to configure the netfilter rules/firewall.

ufw will run some of the binaries or link some of the libraries provided by the nftables package. IT DOES NOT RELY ON THE NFTABLES.SERVICE!
The service just loads /etc/nftables.conf into netfilter and an empty /etc/nftables.conf is equivalent to disabling the service and
==> disabling the nftables.service is therefore exactly what you should do!

Please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

atlimar · 2024-10-28 18:41:37

seth wrote:

IT DOES NOT RELY ON THE NFTABLES.SERVICE!

Perfect, thanks a lot for the help! It pushed me in the right direction to try your suggestions and made me learn along the way.

The way ufw is described by the wiki as a frontend for nftables makes me intuitively think it's relying on a running nftables service (in order to have something to front). I gather frontend means something different, in the context of Linux, compared to what I'm used to.

Last edited by atlimar (2024-10-28 18:47:14)

progandy · 2024-10-28 19:04:49

nftables runs in the kernel. You can think of it as a driver and UFW is a frontend / UI for that driver. That is not completely accurate, but should be good enough for a first though model.
The nftables.service is just another basic tool that talks to the nftables module to set it up using a simple configuration file.

Last edited by progandy (2024-10-28 19:06:26)

Arch Linux

#1 2024-10-22 12:48:33

[SOLVED] Device server unreachable on local network

#2 2024-10-22 14:01:00

Re: [SOLVED] Device server unreachable on local network

#3 2024-10-22 15:02:07

Re: [SOLVED] Device server unreachable on local network

#4 2024-10-22 15:06:09

Re: [SOLVED] Device server unreachable on local network

#5 2024-10-22 15:22:20

Re: [SOLVED] Device server unreachable on local network

#6 2024-10-22 15:40:02

Re: [SOLVED] Device server unreachable on local network

#7 2024-10-22 16:00:46

Re: [SOLVED] Device server unreachable on local network

#8 2024-10-22 16:54:32

Re: [SOLVED] Device server unreachable on local network

#9 2024-10-22 17:27:30

Re: [SOLVED] Device server unreachable on local network

#10 2024-10-22 17:33:49

Re: [SOLVED] Device server unreachable on local network

#11 2024-10-22 17:58:41

Re: [SOLVED] Device server unreachable on local network

#12 2024-10-22 18:04:51

Re: [SOLVED] Device server unreachable on local network

#13 2024-10-22 18:05:14

Re: [SOLVED] Device server unreachable on local network

#14 2024-10-23 07:12:45

Re: [SOLVED] Device server unreachable on local network

#15 2024-10-23 07:15:20

Re: [SOLVED] Device server unreachable on local network

#16 2024-10-23 07:39:44

Re: [SOLVED] Device server unreachable on local network

#17 2024-10-23 07:45:07

Re: [SOLVED] Device server unreachable on local network

#18 2024-10-23 07:46:20

Re: [SOLVED] Device server unreachable on local network

#19 2024-10-23 08:24:53

Re: [SOLVED] Device server unreachable on local network

#20 2024-10-23 08:26:02

Re: [SOLVED] Device server unreachable on local network

#21 2024-10-23 12:27:27

Re: [SOLVED] Device server unreachable on local network

#22 2024-10-28 09:11:33

Re: [SOLVED] Device server unreachable on local network

#23 2024-10-28 13:06:31

Re: [SOLVED] Device server unreachable on local network

#24 2024-10-28 18:41:37

Re: [SOLVED] Device server unreachable on local network

#25 2024-10-28 19:04:49

Re: [SOLVED] Device server unreachable on local network

Board footer