Some traffic not using openvpn tunnel?

ce1984 · 2021-02-15 01:03:11

Hi all, I'm having a hard time figuring out what's going on here and could really use some pointers. I admit my understanding on some of this is limited, so please bare with me if I have any errors in the explanation. I believe I have discovered that my bittorrent traffic is not going through my VPN tunnel.

Relevant info:
I use a VPN provider with a generated conf file, started by openvpn-client.service. After startup, I can tell that my browser traffic is being routed through the VPN, using a geoip tool that identifies the server, and checking with wireshark, that shows all of the ip traffic on the eno1 interface has the vpn server ip as the destination or source.

However, when I start a bittrorent client, I see connections appear on wireshark directly to peer IPs on the eno1 device. Given the routing table, I don't understand why these connections are not being routed via the tun0 device.

My routing table, as modified by openvpn, with comments :

[user@archlinux ~]$ ip route
0.0.0.0/1 via 10.33.84.1 dev tun0 ##all IPs in range 0.0.0.0 through 127.255.255.255 
default via 192.168.0.1 dev eno1 proto dhcp src 192.168.0.110 metric 1002 ## should be ignored, as 0.0.0.0/1 and 128.0.0.0/1 take precedence
10.33.84.0/24 dev tun0 proto kernel scope link src 10.33.84.246 ## private network of vpn
128.0.0.0/1 via 10.33.84.1 dev tun0  ##all IPs in range 128.0.0.0 through 255.255.255.255 
192.168.0.0/24 dev eno1 proto dhcp scope link src 192.168.0.110 metric 1002 ## local network traffic via router
193.37.254.34 via 192.168.0.1 dev eno1 ## reach VPN server through router

Sample wireshark frame captured from eno1:

Frame 101: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) on interface eno1, id 0
    Interface id: 0 (eno1)
    Encapsulation type: Ethernet (1)
    Arrival Time: Feb 14, 2021 17:19:26.568617270 MST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1613348366.568617270 seconds
    [Time delta from previous captured frame: 0.000028661 seconds]
    [Time delta from previous displayed frame: 0.000028661 seconds]
    [Time since reference or first frame: 22.819882091 seconds]
    Frame Number: 101
    Frame Length: 62 bytes (496 bits)
    Capture Length: 62 bytes (496 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:udp:data]
    [Coloring Rule Name: UDP]
    [Coloring Rule String: udp]
Ethernet II, Src: 
    Destination: 
    Source: 
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 192.168.0.110, Dst: 62.149.20.45
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 48
    Identification: 0x7b16 (31510)
    Flags: 0x40, Don't fragment
    Fragment Offset: 0
    Time to Live: 64
    Protocol: UDP (17)
    Header Checksum: 0xabce [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 192.168.0.110
    Destination Address: 62.149.20.45
User Datagram Protocol, Src Port: 49359, Dst Port: 230
    Source Port: 49359
    Destination Port: 230
    Length: 28
    Checksum: 0x1406 [unverified]
    [Checksum Status: Unverified]
    [Stream index: 11]
    [Timestamps]
    UDP payload (20 bytes)
Data (20 bytes)
    Data: 41007cddf049cca20000000000000000c5690000
    [Length: 20]

Really hope someone can point out what I'm missing here.

Thanks

Last edited by ce1984 (2021-02-15 05:51:49)

fukawi2 · 2021-02-15 05:42:06

What is the output of `ip route get 62.149.20.45`?

(Not going to be the problem, but the comments in your routes is wrong - the comment that starts with "should be ignored..." says /24 but should be /1)

ce1984 · 2021-02-15 05:57:57

fukawi2 wrote:

What is the output of `ip route get 62.149.20.45`?
(Not going to be the problem, but the comments in your routes is wrong - the comment that starts with "should be ignored..." says /24 but should be /1)

Thanks for the correction, just fixed in the original post.

Output of ip route get:

[user@archlinux ~]$ ip route get 62.149.20.45
62.149.20.45 via 10.33.84.1 dev tun0 src 10.33.84.246 uid 1000 
    cache

I've also played around with iptables and by dropping outbound traffic except to the VPN server, routing now works as I would expect. I'd still like to try and understand why this happens in the first place though.

fukawi2 · 2021-02-15 06:32:28

Hard to say now that's it working. My first guess would be cached routes. Run `ip route flush cache` after connecting to VPN to clear the cache.

ce1984 · 2021-02-15 06:53:57

fukawi2 wrote:

Hard to say now that's it working. My first guess would be cached routes. Run `ip route flush cache` after connecting to VPN to clear the cache.

By stopping the iptables service, it's easy to recreate. No apparent change after 'ip route flush cache' .

Koatao · 2021-02-15 20:55:10

Hello,

I have no real clue about what might cause a leak but a few questions.

How do you make it work with iptables? What rules are you using?

Does it only happens with bittrorrent protocol? Did you test other peer to peer protocol?

Do you start your bittorent client after starting the VPN client or before?

If you remove the default route (leaving only the two routes with the 1 bit mask), does it still leak?

Last edited by Koatao (2021-02-15 20:56:01)

ce1984 · 2021-02-15 22:12:43

So my iptables setup right now is blocking outbound packets except those going to the vpn server:

$ sudo iptables -L -v --line-numbers 
Chain INPUT (policy DROP 4 packets, 144 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1    90673   26M ACCEPT     all  --  lo     any     anywhere             anywhere            
2      25M   32G ACCEPT     all  --  tun0   any     anywhere             anywhere            
3      25M   34G ACCEPT     udp  --  eno1   any     193.37.254.34        anywhere             state NEW,RELATED,ESTABLISHED udp spt:https

Chain FORWARD (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1      20M   11G ACCEPT     all  --  any    tun0    anywhere             anywhere            
2        0     0 ACCEPT     all  --  any    any     anywhere             dns.google          
3    90673   26M ACCEPT     all  --  any    lo      anywhere             anywhere            
4      22M   12G ACCEPT     udp  --  any    any     anywhere             193.37.254.34       
5        0     0 ACCEPT     udp  --  any    any     archlinux            10.33.84.0/24       
6    88540   11M LOGGING    all  --  any    any     anywhere             anywhere            

Chain LOGGING (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1     1833  238K LOG        all  --  any    any     anywhere             anywhere             limit: avg 2/min burst 5 LOG level warning prefix "IPTables-Dropped: "
2    88540   11M DROP       all  --  any    any     anywhere             anywhere            

Chain TCP (0 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain UDP (0 references)
num   pkts bytes target     prot opt in     out     source               destination

I still see these packets show up on the log, but at least now they're not making it out, and they no longer appear on wireshark.
A section from the log showing these packets being dropped:

Feb 14 19:33:31 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=71.92.63.124 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=46087 DF PROTO=UDP SPT=49359 DPT=30075 LEN=28 
Feb 14 19:33:39 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=91.247.193.2 LEN=132 TOS=0x00 PREC=0x00 TTL=64 ID=22711 DF PROTO=UDP SPT=49359 DPT=15671 LEN=112 
Feb 14 19:34:10 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=180.245.178.67 LEN=132 TOS=0x00 PREC=0x00 TTL=64 ID=33127 DF PROTO=UDP SPT=49359 DPT=6881 LEN=112 
Feb 14 19:34:45 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=185.65.134.169 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=56436 DF PROTO=UDP SPT=49359 DPT=32443 LEN=28 
Feb 14 19:35:28 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=94.79.4.14 LEN=132 TOS=0x00 PREC=0x00 TTL=64 ID=14314 DF PROTO=UDP SPT=49359 DPT=49436 LEN=112 
Feb 14 19:35:49 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=36.106.166.53 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=18913 DF PROTO=UDP SPT=49359 DPT=8999 LEN=28 
Feb 14 19:36:09 archlinux kernel: IPTables-Dropped: IN= OUT=eno1 SRC=192.168.0.110 DST=176.209.234.51 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=42088 DF PROTO=UDP SPT=49359 DPT=29077 LEN=28

Everything still seems to work fine by the way, even with these being dropped.

I haven't tested with any other protocols, but I could give it a shot if you think that would help figure this out. Any recommendations?

I just tested with the default route removed, and that does seem to fix it, as I don't get any leaked packets even with iptables turned off. So I guess that explains part of it, but I don't undertand why that routing rule was being followed instead of the more specific 1-bitmask entries.

Thanks for your help!

Zod · 2021-02-15 22:31:15

I may have missed it, but, can't you bind the bittorrent client to the tunnel adapter (tun0) in the client settings?

ce1984 · 2021-02-15 22:41:18

Zod wrote:

I may have missed it, but, can't you bind the bittorrent client to the tunnel adapter (tun0) in the client settings?

I hadn't thought of that, but yes that looks like it's an option in the client (deluge). It was blank by default. I also notice there are options for UPnP and NAT-PMP that are turned on, not sure if they could have anything to do with this.

fukawi2 · 2021-02-16 02:18:43

Interesting... With iptables turned off and the VPN connected, can you post the output of `mtr -c1 -r 62.149.20.45`? (You might need to install mtr)

Last edited by fukawi2 (2021-02-16 02:19:02)

ce1984 · 2021-02-16 02:45:11

fukawi2 wrote:

Interesting... With iptables turned off and the VPN connected, can you post the output of `mtr -c1 -r 62.149.20.45`? (You might need to install mtr)

Okay, so:
iptables off
default route restored
confirmed issue still persists (wireshark showing outbound packets to lots of addresses other than the vpn server)

$ mtr -c1 -r 62.149.20.45
Start: 2021-02-15T19:38:22-0700
HOST: archlinux                   Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 10.33.84.1                 0.0%     1   25.1  25.1  25.1  25.1   0.0
  2.|-- vlan177.as09.lax1.us.m247  0.0%     1   25.7  25.7  25.7  25.7   0.0
  3.|-- ???                       100.0     1    0.0   0.0   0.0   0.0   0.0
  4.|-- 217.138.223.104            0.0%     1   25.3  25.3  25.3  25.3   0.0
  5.|-- 37.120.220.80              0.0%     1   25.9  25.9  25.9  25.9   0.0
  6.|-- te0-0-1-0.201.nr11.b00269  0.0%     1   26.1  26.1  26.1  26.1   0.0
  7.|-- te0-4-1-11.agr21.lax04.at  0.0%     1   28.2  28.2  28.2  28.2   0.0
  8.|-- be2963.ccr41.lax04.atlas.  0.0%     1   26.1  26.1  26.1  26.1   0.0
  9.|-- be3271.ccr41.lax01.atlas.  0.0%     1   26.4  26.4  26.4  26.4   0.0
 10.|-- be2931.ccr31.phx01.atlas.  0.0%     1   37.9  37.9  37.9  37.9   0.0
 11.|-- be2929.ccr21.elp01.atlas.  0.0%     1   46.3  46.3  46.3  46.3   0.0
 12.|-- be2928.ccr42.iah01.atlas.  0.0%     1   61.6  61.6  61.6  61.6   0.0
 13.|-- be2690.ccr42.atl01.atlas.  0.0%     1   75.7  75.7  75.7  75.7   0.0
 14.|-- be2113.ccr42.dca01.atlas.  0.0%     1   86.5  86.5  86.5  86.5   0.0
 15.|-- be2807.ccr42.jfk02.atlas.  0.0%     1   92.6  92.6  92.6  92.6   0.0
 16.|-- be2490.ccr42.lon13.atlas.  0.0%     1  164.2 164.2 164.2 164.2   0.0
 17.|-- be12488.ccr42.ams03.atlas  0.0%     1  171.9 171.9 171.9 171.9   0.0
 18.|-- be2816.ccr42.ham01.atlas.  0.0%     1  212.1 212.1 212.1 212.1   0.0
 19.|-- be2484.ccr21.waw01.atlas.  0.0%     1  212.0 212.0 212.0 212.0   0.0
 20.|-- be2478.ccr22.bts01.atlas.  0.0%     1  211.9 211.9 211.9 211.9   0.0
 21.|-- be2047.ccr22.kbp01.atlas.  0.0%     1  214.0 214.0 214.0 214.0   0.0
 22.|-- be2497.rcr13.b020975-2.kb  0.0%     1  212.2 212.2 212.2 212.2   0.0
 23.|-- 149.6.190.90               0.0%     1  215.1 215.1 215.1 215.1   0.0
 24.|-- 62.149.20.45               0.0%     1  211.6 211.6 211.6 211.6   0.0

So it looks to me like this is running over the vpn, even though other traffic apparently isn't.

fukawi2 · 2021-02-16 03:17:24

OK, well that seems to narrow down the problem being with your torrent client since the kernel is routing the traceroute packets correctly. I'm not sure why or how it's doing the "wrong thing" though.

You haven't done anything before with multiple routing tables or networking namespaces or something exotic like that?

ce1984 · 2021-02-16 03:26:04

fukawi2 wrote:

You haven't done anything before with multiple routing tables or networking namespaces or something exotic like that?

Nope, nothing out of the ordinary like that. I don't know all that much about how this works, but it seems to me that something like this has some significant security implications if processes can bypass a vpn that's set up for the purpose of privacy.

When I get some time I might see if I can replicated on a new install with just deluge and openvpn.

Zod · 2021-02-16 13:50:16

Thought I'd add, I'm not entirely convinced that traffic is 'leaking'.

tun0 is a virtual adapter that physically exists on eno1.

The VPN tunnel is actually just encapsulation, meaning routing information is added to the packet when it's processed by openvpn .

Seeing, via wireshark, a packet on eno1 with the routing information posted in the op isn't entirely convincing (at least to me). What's really needed is to capture that same packet at least one hop removed in the direction of the target from eno1/tun0.

Edit0: https://www.pcwdld.com/port-mirroring-d … d-tutorial

Last edited by Zod (2021-02-16 13:57:32)

ce1984 · 2021-02-16 19:57:00

Zod wrote:

Thought I'd add, I'm not entirely convinced that traffic is 'leaking'.
tun0 is a virtual adapter that physically exists on eno1.
The VPN tunnel is actually just encapsulation, meaning routing information is added to the packet when it's processed by openvpn .
Seeing, via wireshark, a packet on eno1 with the routing information posted in the op isn't entirely convincing (at least to me). What's really needed is to capture that same packet at least one hop removed in the direction of the target from eno1/tun0.
Edit0: https://www.pcwdld.com/port-mirroring-d … d-tutorial

So while I don't have the capability to watch this packet as it traverses the router, I'm fairly confident that what we're looking at is the raw ethernet frame and bittorent packet without the openvpn-provided wrapper. As I understand it tun is supposed to be a layer3 (IP) device, so the fact that this showing up as an ethernet frame is good evidence that this is the actual packet going out on the wire. The MACs match my NIC and router, the port number is that being used by deluge, and the destination address is that of a peer as shown in deluge.

In contrast, I can see that packets routed to tun0 then go out via eno1, encapsulated and with the detination IP that matches my VPN server. So I think I see how it's supposed to work, but some packets were just bypassing tun0.

fukawi2 · 2021-02-16 22:57:07

Zod wrote:

Thought I'd add, I'm not entirely convinced that traffic is 'leaking'.
tun0 is a virtual adapter that physically exists on eno1.
The VPN tunnel is actually just encapsulation, meaning routing information is added to the packet when it's processed by openvpn .
Seeing, via wireshark, a packet on eno1 with the routing information posted in the op isn't entirely convincing (at least to me). What's really needed is to capture that same packet at least one hop removed in the direction of the target from eno1/tun0.
Edit0: https://www.pcwdld.com/port-mirroring-d … d-tutorial

If you can see the IP headers for non-VPN packets on the physical device, traffic is definitely leaking. The only packets you should see on the physical device is the encrypted encapsulation packets destined to the OpenVPN server (ie, single destination IP, on the port that OpenVPN is using).

giacomo.c · 2021-02-18 06:03:00

It was probably UPnP issue where the client was asking for connections via your router and not being tunneled through your VPN.
Try just disabling that and see if your torrent traffic then goes through the VPN.

Arch Linux

#1 2021-02-15 01:03:11

Some traffic not using openvpn tunnel?

#2 2021-02-15 05:42:06

Re: Some traffic not using openvpn tunnel?

#3 2021-02-15 05:57:57

Re: Some traffic not using openvpn tunnel?

#4 2021-02-15 06:32:28

Re: Some traffic not using openvpn tunnel?

#5 2021-02-15 06:53:57

Re: Some traffic not using openvpn tunnel?

#6 2021-02-15 20:55:10

Re: Some traffic not using openvpn tunnel?

#7 2021-02-15 22:12:43

Re: Some traffic not using openvpn tunnel?

#8 2021-02-15 22:31:15

Re: Some traffic not using openvpn tunnel?

#9 2021-02-15 22:41:18

Re: Some traffic not using openvpn tunnel?

#10 2021-02-16 02:18:43

Re: Some traffic not using openvpn tunnel?

#11 2021-02-16 02:45:11

Re: Some traffic not using openvpn tunnel?

#12 2021-02-16 03:17:24

Re: Some traffic not using openvpn tunnel?

#13 2021-02-16 03:26:04

Re: Some traffic not using openvpn tunnel?

#14 2021-02-16 13:50:16

Re: Some traffic not using openvpn tunnel?

#15 2021-02-16 19:57:00

Re: Some traffic not using openvpn tunnel?

#16 2021-02-16 22:57:07

Re: Some traffic not using openvpn tunnel?

#17 2021-02-18 06:03:00

Re: Some traffic not using openvpn tunnel?

Board footer