Arch Linux

svalee

Member

Registered: 2019-01-20

Posts: 44

Website

[SOLVED] Zscaler creates a dns resolving issue when turned on

I'm using network-manager, systemd-resolved and systemd-resolvconf

This is what I see in resolvectl status  when the zscaler is active

Link 14 (zcctun0)
    Current Scopes: DNS LLMNR/IPv4 mDNS/IPv4
         Protocols: +DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 100.64.0.2
       DNS Servers: 100.64.0.2
        DNS Domain: ~.
     Default Route: yes

I interpret it as route all traffic through the private dns server. Do I understand it right?

Zscaler up

dig google.com         
;; communications error to 127.0.0.53#53: timed out
;; communications error to 127.0.0.53#53: timed out
;; communications error to 127.0.0.53#53: timed out

; <<>> DiG 9.20.3 <<>> google.com
;; global options: +cmd
;; no servers could be reached

zscaler up but specify the dns server explicitly

dig @8.8.8.8 google.com                    

; <<>> DiG 9.20.3 <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2150
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		184	IN	A	172.217.25.206

;; Query time: 43 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Wed Dec 11 20:13:20 +07 2024
;; MSG SIZE  rcvd: 55

resolvectl status before zscaler up
https://pastebin.com/iJ1mxfWs

resolvectl after zscaler up
https://pastebin.com/zsCFA6bL

journal after I bring zscaler up
https://pastebin.com/ir97ZfSZ

Does anyone know what can be the issue and how to solve it?
Let me know if I can provide any additional info

Last edited by svalee (2025-03-12 08:19:12)

-thc

Member

Registered: 2017-03-15

Posts: 853

Re: [SOLVED] Zscaler creates a dns resolving issue when turned on

Yes - systemd-resolved states that this connection/network has an active default gateway.
Be aware that - according to system-resolved - your transport/physical network (wlo1) retains the default gateway too.

This looks like a routing problem. please post the output of

ip -4 route

before and after connecting to zscaler.

Last edited by -thc (2025-03-11 16:10:41)

svalee

Member

Registered: 2019-01-20

Posts: 44

Website

Re: [SOLVED] Zscaler creates a dns resolving issue when turned on

What should I be aware of with regards to the wlo1? Is it an error that it retains the default gateway as well? What are the implications of that?

Zscaler down

default via 192.168.1.1 dev wlo1 proto dhcp src 192.168.1.8 metric 600 
172.17.0.0/16 dev br-5bd0d1c27100 proto kernel scope link src 172.17.0.1 linkdown 
192.168.1.0/24 dev docker0 proto kernel scope link src 192.168.1.5 linkdown 
192.168.1.0/24 dev wlo1 proto kernel scope link src 192.168.1.8 metric 600 

Zscaler up

default via 192.168.1.1 dev wlo1 proto dhcp src 192.168.1.8 metric 600 
12.232.214.226 via 192.168.1.1 dev docker0 linkdown 
38.142.189.163 via 192.168.1.1 dev docker0 linkdown 
63.158.119.230 via 192.168.1.1 dev docker0 linkdown 
66.162.147.188 via 192.168.1.1 dev docker0 linkdown 
100.64.0.0/16 via 100.64.0.1 dev zcctun0 
100.64.0.0/16 dev zcctun0 proto kernel scope link src 100.64.0.1 
104.129.196.48 via 192.168.1.1 dev docker0 linkdown 
104.129.196.49 via 192.168.1.1 dev docker0 linkdown 
136.226.95.70 via 192.168.1.1 dev docker0 linkdown 
136.226.95.96 via 192.168.1.1 dev docker0 linkdown 
147.161.208.82 via 192.168.1.1 dev docker0 linkdown 
147.161.208.88 via 192.168.1.1 dev docker0 linkdown 
165.225.246.70 via 192.168.1.1 dev docker0 linkdown 
165.225.246.71 via 192.168.1.1 dev docker0 linkdown 
165.225.247.89 via 192.168.1.1 dev docker0 linkdown 
165.225.247.97 via 192.168.1.1 dev docker0 linkdown 
172.16.176.0/20 via 100.64.0.1 dev zcctun0 
172.17.0.0/20 via 100.64.0.1 dev zcctun0 
172.17.0.0/16 dev br-5bd0d1c27100 proto kernel scope link src 172.17.0.1 linkdown 
172.17.176.0/20 via 100.64.0.1 dev zcctun0 
172.17.192.0/20 via 100.64.0.1 dev zcctun0 
172.17.208.0/20 via 100.64.0.1 dev zcctun0 
189.201.147.187 via 192.168.1.1 dev docker0 linkdown 
192.168.1.0/24 dev docker0 proto kernel scope link src 192.168.1.5 linkdown 
192.168.1.0/24 dev wlo1 proto kernel scope link src 192.168.1.8 metric 600 
216.150.237.52 via 192.168.1.1 dev docker0 linkdown 

-thc

Member

Registered: 2017-03-15

Posts: 853

Re: [SOLVED] Zscaler creates a dns resolving issue when turned on

What should I be aware of with regards to the wlo1? Is it an error that it retains the default gateway as well? What are the implications of that?

Just be aware of the fact. Nothing more.

For a "full tunnel" VPN to work the VPN client either has to define a second routing table (rule based routing) or define "routing exemptions" for the VPN endpoints. zscaler seems to do the latter.

There is a collision of network routes that leads zscaler to believe "docker0" is the default network:

192.168.1.0/24 dev docker0 proto kernel scope link src 192.168.1.5 linkdown 
192.168.1.0/24 dev wlo1 proto kernel scope link src 192.168.1.8 metric 600 

It sets routes to private networks through it's own tunnel:

100.64.0.0/16 via 100.64.0.1 dev zcctun0 
172.16.176.0/20 via 100.64.0.1 dev zcctun0 
172.17.0.0/20 via 100.64.0.1 dev zcctun0 
172.17.176.0/20 via 100.64.0.1 dev zcctun0 
172.17.192.0/20 via 100.64.0.1 dev zcctun0 
172.17.208.0/20 via 100.64.0.1 dev zcctun0

It sets host routes as exemptions for the local network "docker0" - which is wrong (it's wlo1):

12.232.214.226 via 192.168.1.1 dev docker0 linkdown 
38.142.189.163 via 192.168.1.1 dev docker0 linkdown 
63.158.119.230 via 192.168.1.1 dev docker0 linkdown 
66.162.147.188 via 192.168.1.1 dev docker0 linkdown 
104.129.196.48 via 192.168.1.1 dev docker0 linkdown 
104.129.196.49 via 192.168.1.1 dev docker0 linkdown 
136.226.95.70 via 192.168.1.1 dev docker0 linkdown 
136.226.95.96 via 192.168.1.1 dev docker0 linkdown 
147.161.208.82 via 192.168.1.1 dev docker0 linkdown 
147.161.208.88 via 192.168.1.1 dev docker0 linkdown 
165.225.246.70 via 192.168.1.1 dev docker0 linkdown 
165.225.246.71 via 192.168.1.1 dev docker0 linkdown 
165.225.247.89 via 192.168.1.1 dev docker0 linkdown 
165.225.247.97 via 192.168.1.1 dev docker0 linkdown 
189.201.147.187 via 192.168.1.1 dev docker0 linkdown 
216.150.237.52 via 192.168.1.1 dev docker0 linkdown 

This cannot work. Since I despise containers I cannot say if your docker network setup is incorrect or not.

As long as "docker0" has the same network "192.168.0.1/24" as wlo1 this zscaler hiccup will remain.

svalee

Member

Registered: 2019-01-20

Posts: 44

Website

Re: [SOLVED] Zscaler creates a dns resolving issue when turned on

Yeap, that was it, thank you so much -thc.

The fix was to change the default subnet for docker in /etc/docker/daemon.json

I've changed it to

{
    "bip": "172.17.0.1/16"
}

For people with the same issue you'd probably want to stop the docker and docker socket services first, remove the existing interface and then change the docker config file

systemctl stop docker docker.socket
ip link delete docker0