You are not logged in.

Pages: 1

#1 2023-11-11 21:27:31

jayrock
Member
Registered: 2021-12-19
Posts: 11

Port Forwarding from physical host server into VM client [SOLVED]

Imagine a configuration where you have a Home Computer and a Server connected into the same LAN network. The Home Server is running Arch Linux as a host OS and it has Virtual Machine running Debian inside (emulated by QEMU). The VMs are behind a virtual NAT. The host system has `ufw` Firewall installed.

Now, I'm trying fully understand how to get the Port forwarding working so, that I can for example SSH from my Home Computer directly into the Debian VM. Below is what I have figured out so far, and how I understand the situation...

Network

1. I enabled the `vhost_net` kernel module to allow guest VMs to process packets directly in the kernel (instead of user space)

# -- /etc/modules-load.d/vhost_net.conf
vhost_net

2. I created a virtual NAT configuration ('virbr1') with static IP for the 'debsrv' VM. I also updated the debsrv QEMU config to use the created 'my-nat' network.

# -- Created new NAT configuration with static IP for 'debsrv'
virsh net-create ./my-nat.xml
virsh net-autostart my-nat

# -- Updated the debsrv QEMU configuration to use the `my-nat` network
virsh dumpxml debsrv > debsrv-config.xml
virsh define debsrv-config.xml
<network>
    <name>my-nat</name> 
    <forward mode='nat'> 
        <nat> 
            <port start='1024' end='65535'/> 
        </nat>
    </forward>
    <bridge name='virbr1' stp='on' delay='0'/> 
    <ip address='192.168.100.1' netmask='255.255.255.0'> 
        <dhcp> 
            <range start='192.168.100.2' end='192.168.100.200'/> 
            <host mac='52:54:00:00:00:01' name='debsrv' ip='192.168.100.201'/>
        </dhcp> 
    </ip>
</network>

3. I ensured that the Port Forwarding is enabled in the Linux Kernel

cat /proc/sys/net/ipv4/ip_forward
sudo sysctl -w net.ipv4.ip_forward=1

4. UFW manual suggested to uncomment the following lines in `/etc/ufw/sysctl.conf` to enable port forwarding for both IPv4 and IPv6, so I did.

# -- /etc/ufw/sysctl.conf
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

5. I also did change the `DEFAULT_FORWARD_POLICY` to `ACCEPT` as suggested by many sources.

# --/etc/default/ufw
DEFAULT_FORWARD_POLICY="ACCEPT"

6. I poked some port holes in the UFW itself

  • tcp/22 is for SSH to archsrv (host server)

  • tcp/8022 is meant to be forwarded to tcp/22 port for SSH to debsrv (client VM)

ufw allow ssh
ufw allow from any port 8022 proto tcp
ufw status
-------------------------------------------------------
Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       Anywhere
Anywhere                   ALLOW       8022/tcp
22 (v6)                    ALLOW       Anywhere (v6)
Anywhere (v6)              ALLOW       8022/tcp (v6)

7. Now, this is where I am stuck. I was able to configure the following `iptables` rules like this...

iptables -I FORWARD -m state -d 192.168.100.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -t nat -I PREROUTING -p tcp -d 192.168.50.128 --dport 8022 -j DNAT --to-destination 192.168.100.201:22

...which indeed worked so that I was able to SSH from my home PC directly into the Debian client VM.

@home-pc ~/ ssh 192.168.50.128 -p 8022

...however, I have not been able to find a way how to get these iptable configurations persistent, so that they would also be configured during the next reboot. Some old post at Serverfault forum suggested creating a script that would run these commands at boot time, but for some reason this does not feel like a "correct" solution to configure this.

I tried to play around with the `/etc/ufw/before.rules` but at least the below configuration got me only half-way there. I still need to manually command the iptables...

# -- /etc/ufw/before.rules
# This is configure _after_ the *filter table
*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 8022 -j DNAT --to-destination 192.168.100.201:22
COMMIT

Any ideas?

Last edited by jayrock (2023-11-13 16:41:05)

Offline

#2 2023-11-11 23:43:39

twelveeighty
Member
From: Alberta, Canada
Registered: 2011-09-04
Posts: 1,225

Re: Port Forwarding from physical host server into VM client [SOLVED]

Since you do not mention it at all in your post, I'll ask the obvious question: why would you not use Bridged networking on the VM guest?

Offline

#3 2023-11-12 08:14:16

jayrock
Member
Registered: 2021-12-19
Posts: 11

Re: Port Forwarding from physical host server into VM client [SOLVED]

twelveeighty wrote:

Since you do not mention it at all in your post, I'll ask the obvious question: why would you not use Bridged networking on the VM guest?

That is a good question. The thing is that I'm a total n00b what comes to maintaining a Linux server. Many sources recommend ready solutions such as Proxmox to handle the server virtualization. However, this all is sort of a hobby for me (learning Linux systems in general) and I kinda want to first learn more deeply how these things work in Linux systems "under the hood", before installing any kind of ready solution.

Now, why NAT instead of a bridged networking? You're right that in this scenario, bridged networking makes more sense. But, I figured that since the NAT was the default configuration in QEMU, I would start learning to configure it first.

Offline

#4 2023-11-12 08:41:28

-thc
Member
Registered: 2017-03-15
Posts: 775

Re: Port Forwarding from physical host server into VM client [SOLVED]

jayrock wrote:

5. I also did change the `DEFAULT_FORWARD_POLICY` to `ACCEPT` as suggested by many sources.

# --/etc/default/ufw
DEFAULT_FORWARD_POLICY="ACCEPT"

The policy change makes the first one of those two commands useless:

jayrock wrote:

7. Now, this is where I am stuck. I was able to configure the following `iptables` rules like this...

iptables -I FORWARD -m state -d 192.168.100.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -t nat -I PREROUTING -p tcp -d 192.168.50.128 --dport 8022 -j DNAT --to-destination 192.168.100.201:22

This at least looks O.K.:

jayrock wrote:
# -- /etc/ufw/before.rules
# This is configure _after_ the *filter table
*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 8022 -j DNAT --to-destination 192.168.100.201:22
COMMIT

I would suggest dumping the iptables ruleset ("iptables-save") after a reboot (without working port forwarding) and after issuing the iptables commands (working port forwarding) and comparing them - line by line.

Please do not post the dumped rulesets here - they're mostly a convoluted mess of empty chains making the packets jump around like balls i a pinball machine.

Offline

#5 2023-11-12 11:31:12

jayrock
Member
Registered: 2021-12-19
Posts: 11

Re: Port Forwarding from physical host server into VM client [SOLVED]

For some reason I still need to run the below iptables rule to get the port forwarding to work. (Even though DEFAULT_FORWARD_POLICY="ACCEPT" is configured). However, I noticed that I do not have to run the other iptables rule. Appraently that is handled now by the `/etc/ufw/before.rules`.

iptables -I FORWARD -m state -d 192.168.100.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

Diffing the iptables "filter" table before, and after running the above command I get the following...

----- *filter BEFORE -------> 
> :INPUT DROP [5:180]
> :OUTPUT ACCEPT [2:80]
----- *filter AFTER -------> 
< :INPUT DROP [1:36]
< :OUTPUT ACCEPT [0:0]
< -A FORWARD -d 192.168.100.0/24 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT

So, I figured I need to configure these into the *filter table in `/etc/ufw/before.rules`. I tried something like this:

# -- /etc/ufw/before.rules
# Allow Forward rules for specific interface
-A ufw-before-forward -i eno1 -j ACCEPT
-A ufw-before-forward -o eno1 -j ACCEPT
-A ufw-before-forward -i virbr0 -j ACCEPT
-A ufw-before-forward -o virbr0 -j ACCEPT
-A ufw-before-input -i eno1 -j ACCEPT
-A ufw-before-output -o virbr0 -j ACCEPT
-A FORWARD -d 192.168.100.0/24 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT

...But the port forwarding was still not working. Now I again run the iptables rules manually, and noticed that it added that `-A FORWARD -d 192.168.100.0....` rule above those other rules. (Notice, how the same rule is applied twice, the bottom one originating from before.rules). Also, there is that one `ufw-reject-forward` rule, which I was unable to find from my `before.rules`

-A FORWARD -d 192.168.100.0/24 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A FORWARD -j ufw-track-forward
-A FORWARD -d 192.168.100.0/24 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT

Offline

#6 2023-11-12 12:03:27

-thc
Member
Registered: 2017-03-15
Posts: 775

Re: Port Forwarding from physical host server into VM client [SOLVED]

All of this looks like the default for ufw forwarding/routing is still the default "deny".

# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
[...]

If that's the case, change it via

ufw default allow routed

If that works, the only needed "before" rule should be the nat rule.

Offline

#7 2023-11-12 19:29:31

jayrock
Member
Registered: 2021-12-19
Posts: 11

Re: Port Forwarding from physical host server into VM client [SOLVED]

% sudo ufw status verbose
Status: active
Logging: on (full)
Default: deny (incoming), allow (outgoing), allow (routed)
New Profiles: skip
...

Running below did not help as I believe the setting was already configured (as seen above) sad

ufw default allow routed

Also, I noticed that...

# This does NOT work (without NEW)
sudo iptables -I FORWARD -m state -d 192.168.100.0/24 --state RELATED,ESTABLISHED -j ACCEPT

# This does work (with NEW)
sudo iptables -I FORWARD -m state -d 192.168.100.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

...So maybe the key is that  `NEW` parameter in the `iptables` rule. Which got me thinking that the issue might not be in the routing/forwarding settings? But the ufw/iptables is blocking connections that are established from outside?

Offline

#8 2023-11-12 20:30:25

-thc
Member
Registered: 2017-03-15
Posts: 775

Re: Port Forwarding from physical host server into VM client [SOLVED]

jayrock wrote:

...So maybe the key is that  `NEW` parameter in the `iptables` rule. Which got me thinking that the issue might not be in the routing/forwarding settings? But the ufw/iptables is blocking connections that are established from outside?

Thats the job of the ufw rules. I somehow missed the fact that your "8022" rules' direction is inverted:

jayrock wrote:
ufw allow ssh
ufw allow from any port 8022 proto tcp
ufw status
-------------------------------------------------------
Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       Anywhere
Anywhere                   ALLOW       8022/tcp
22 (v6)                    ALLOW       Anywhere (v6)
Anywhere (v6)              ALLOW       8022/tcp (v6)

Why?

Offline

#9 2023-11-12 20:36:58

jayrock
Member
Registered: 2021-12-19
Posts: 11

Re: Port Forwarding from physical host server into VM client [SOLVED]

Ok. I might've figured out what is happening....

1) First off... I found this web page helpful, as it explained pretty clearly how iptables work: https://www.booleanworld.com/depth-guid … -firewall/

2) I learned that I can actually insert the iptables rule in specific index in the table. So instead of appending the list (with -A), I inserted the rule (with -I <index-number>). However, even though inserting rule at #1 index I still found from the 4th index in the table. Apparently LIBVIRT had overwritten the table with it's own rules...

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination
1    LIBVIRT_FWX  all  --  anywhere             anywhere
2    LIBVIRT_FWI  all  --  anywhere             anywhere
3    LIBVIRT_FWO  all  --  anywhere             anywhere
4   ACCEPT     all  --  anywhere             192.168.100.0/24     state NEW,RELATED,ESTABLISHED
5    ufw-before-logging-forward  all  --  anywhere             anywhere
...

I then checked the LIBVIRT table and noticed, that while it also did include my FORWARD rule, it was missing the NEW parameter, which I earlier found important as this state represents the very first packet of a connection.

Chain LIBVIRT_FWI (1 references)
num  target     prot opt source               destination
1    ACCEPT     all  --  anywhere             192.168.100.0/24     ctstate RELATED,ESTABLISHED
2    REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

So, I believe the libvirt is the one causing these issues, and after Googling around, I found others struggling with the same issue...

Offline

#10 2023-11-12 20:57:54

jayrock
Member
Registered: 2021-12-19
Posts: 11

Re: Port Forwarding from physical host server into VM client [SOLVED]

-thc wrote:

I somehow missed the fact that your "8022" rules' direction is inverted:

Why?

Good catch! I tried adding the rule otherway around, but connection is still refused, without manually adding that NEW iptables rule...

Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22                         ALLOW IN    Anywhere
Anywhere                   ALLOW IN    8022/tcp
Anywhere                   ALLOW IN    22/tcp
8022/tcp                   ALLOW IN    Anywhere

I guess I need to start reading libvirt handbook:
https://jamielinux.com/docs/libvirt-net … index.html

Last edited by jayrock (2023-11-12 21:04:39)

Offline

Pages: 1

Board footer

Powered by FluxBB