You are not logged in.

Pages: 1

#1 2023-03-17 19:03:08

Brocellous
Member
Registered: 2017-11-27
Posts: 147

udp fails over wireguard tunnel

I'm trying to diagnose a bandwidth issue between me and a remote server I connect to over wireguard. For some reason, a tcp connection works ok but udp completely fails when testing with iperf3. Here is what that looks like:

TCP working:

[client.here ~]$ iperf3 -c server.far   
Connecting to host server.far, port 5201
[  5] local 10.9.0.8 port 47818 connected to 10.1.0.253 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[...]   
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.0 MBytes  9.27 Mbits/sec  271             sender
[  5]   0.00-10.04  sec  10.4 MBytes  8.67 Mbits/sec                  receiver
[server.far ~]$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 10.9.0.8, port 47804
[  5] local 10.1.0.253 port 5201 connected to 10.9.0.8 port 47818
[ ID] Interval           Transfer     Bitrate
[...]              
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec  10.4 MBytes  8.67 Mbits/sec                  receiver

UDP not working:

[client.here ~]$ iperf3 -c server.far -u            
Connecting to host server.far, port 5201
iperf3: error - unable to read from stream socket: Resource temporarily unavailable
[server.far ~]$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201 (test #2)
-----------------------------------------------------------
Accepted connection from 10.9.0.8, port 56818
[  5] local 10.9.0.1 port 5201 connected to 10.9.0.8 port 40610
iperf3: error - unable to read from stream socket: Connection refused

Here's what the udp test looks like in tcpdump:

[client.here ~]$ sudo tcpdump -ni wg-tcma udp or icmp
[...]
11:47:41.391397 IP 10.9.0.8.33322 > 10.1.0.253.5201: UDP, length 4
11:47:41.428714 IP 10.9.0.1.5201 > 10.9.0.8.33322: UDP, length 4
11:47:41.428788 IP 10.9.0.8 > 10.9.0.1: ICMP 10.9.0.8 udp port 33322 unreachable, length 40
[server.far ~]$ sudo tcpdump -ni wg-tcma udp or icmp
[...]
11:47:41.392511 IP 10.9.0.8.33322 > 10.1.0.253.5201: UDP, length 4
11:47:41.392563 IP 10.9.0.1.5201 > 10.9.0.8.33322: UDP, length 4
11:47:41.429574 IP 10.9.0.8 > 10.9.0.1: ICMP 10.9.0.8 udp port 33322 unreachable, length 40

I cleared all firewall rules on both hosts for this test. The routing is straightforward, I think:

[client.here ~]$ dig @_localdnsstub +noall +ans server.far 
server.far.	0	IN	A	10.1.0.253
[client.here ~]$ ip route get 10.1.0.253
10.1.0.253 dev wg-tcma src 10.9.0.8 uid 1000 
    cache
[client.here ~]$ ip -4 -br addr show dev wg-tcma 
wg-tcma          UNKNOWN        10.9.0.8/32
[server.far ~]$ ip route get 10.9.0.8
10.9.0.8 dev wg-tcma src 10.9.0.1 uid 1001
    cache

What's going on?

Last edited by Brocellous (2023-03-17 19:12:46)

Offline

#2 2023-03-17 21:22:01

-thc
Member
Registered: 2017-03-15
Posts: 775

Re: udp fails over wireguard tunnel

The WireGuard endpoint (10.9.0.1) needs to forward your packets to server.far via "masquerade" or SNAT.
It looks like this is only enabled/configured for TCP.

Offline

#3 2023-03-17 21:45:32

Brocellous
Member
Registered: 2017-11-27
Posts: 147

Re: udp fails over wireguard tunnel

In this case, server.far is the wireguard endpoint as well.

$ parallel ip -4 -br addr show dev ::: eno1 wg-tcma
eno1             UP             10.1.0.253/22 metric 256 
wg-tcma          UNKNOWN        10.9.0.1/24

Interestingly, I usually do have a masquerade rule on server.far:

[server.far ~]$ sudo nft list ruleset
table inet nat {
	chain postrouting {
		type nat hook postrouting priority srcnat; policy accept;
		iifname "wg-tcma" oifname != "wg-tcma" masquerade
	}
}

which I use to forward traffic from server.far to other hosts in the .far lan. Since I wasn't sure if it was related, I tried the above tests without the firewall rule just in case. However UDP works fine when forwarded to other.far from server.far in the .far lan using the above rule:

[client.here ~]$ iperf3 -c other.far -u
Connecting to host other.far, port 5201
[  5] local 10.9.0.8 port 41070 connected to 10.1.0.151 port 5201
[...]  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.000 ms  0/959 (0%)  sender
[  5]   0.00-10.04  sec  1.25 MBytes  1.05 Mbits/sec  0.211 ms  0/959 (0%)  receiver
[other.far ~]$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 10.1.0.253, port 58696
[  5] local 10.1.0.151 port 5201 connected to 10.1.0.253 port 41070
[...]
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  1.25 MBytes  1.05 Mbits/sec  0.211 ms  0/959 (0%)  receiver

Notice that other.far (10.1.0.151) accepts the connection from server.far (10.1.0.253) because of the masquerade.

In any case, a udp packet definitely reaches client.here, because the tcpdumps in OP show it has the chance to return ICMP port unreachable. If it were a problem with client.here I would except client.here > other.far udp to also fail though.

Last edited by Brocellous (2023-03-17 21:46:02)

Offline

#4 2023-03-18 07:05:12

-thc
Member
Registered: 2017-03-15
Posts: 775

Re: udp fails over wireguard tunnel

I tried to replicate this - I use WireGuard endpoints with masquerading too.

Using your commands my WireGuard endpoint reacted like this:

thc@server.far:~$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.7.7.2, port 33684
[  5] local 10.7.7.1 port 5201 connected to 10.7.7.2 port 60929
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  10.00-11.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  11.00-12.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  12.00-13.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  13.00-14.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  14.00-15.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  15.00-16.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  16.00-17.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  17.00-18.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  18.00-19.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  19.00-20.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  20.00-21.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  21.00-22.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  22.00-23.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  23.00-24.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  24.00-25.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  25.00-26.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  26.00-27.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  27.00-28.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  28.00-29.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
[  5]  29.00-30.00  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

I had to bind iperf3 to the physical network adapter to get a result:

thc@server.far:~$ iperf3 -s -B 192.168.6.4
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.7.7.2, port 55204
[  5] local 192.168.6.4 port 5201 connected to 10.7.7.2 port 58888
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec   122 KBytes   996 Kbits/sec  0.155 ms  0/91 (0%)
[  5]   1.00-2.00   sec   127 KBytes  1.04 Mbits/sec  0.147 ms  0/95 (0%)
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  0.197 ms  0/96 (0%)
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  0.186 ms  0/96 (0%)
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  0.178 ms  0/96 (0%)
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  0.215 ms  0/96 (0%)
[  5]   6.00-7.00   sec   127 KBytes  1.04 Mbits/sec  0.238 ms  0/95 (0%)
[  5]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  0.238 ms  0/96 (0%)
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  0.164 ms  0/96 (0%)
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  0.206 ms  0/96 (0%)
[  5]  10.00-10.06  sec  6.68 KBytes   935 Kbits/sec  0.184 ms  0/5 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.06  sec  1.25 MBytes  1.04 Mbits/sec  0.184 ms  0/958 (0%)  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

It seems by default iperf3 binds to the virtual WireGuard adapter and that doesn't work.

Offline

#5 2023-03-18 09:44:47

Brocellous
Member
Registered: 2017-11-27
Posts: 147

Re: udp fails over wireguard tunnel

I suppose in the first case your setup either doesn't generate the port unreachable message or icmp is dropped somewhere.

I tried binding the server to the local address with iperf -s -B 10.1.0.253 and it works as you describe, that's a good find. Still, I don't understand why tcp should work but udp not here, especially since it is apparently rejected on the client host.

Offline

Pages: 1

Board footer

Powered by FluxBB