You are not logged in.
Hello.
As in a previous post said i have set up DNScrypt with DNSmasq in arch linux host which also has virtual machines based on KVM.
I changed the configuration of DNSmasq a little bit from the previous time so to be able to work with libvirtd
bogus-priv
no-resolv
server=127.0.0.1#2053
user=nobody
interface=enp3s0 #host iface
interface=virbr1 #guest iface
bind-interfaces
I then open a VM and tried to ping google.com but a "Network unreachable appeared"
The guest OS is debian 8 and it uses Network manager.
So i tried to debug it by run a journalctl -f on the host.
A simple dig command from host returns this result:
; <<>> DiG 9.10.3 <<>>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20727
;; flags: qr rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 0;; QUESTION SECTION:
;. IN NS;; ANSWER SECTION:
. 85194 IN NS ns7.opennic.glue.
. 85194 IN NS ns6.opennic.glue.
. 85194 IN NS ns2.opennic.glue.
. 85194 IN NS ns8.opennic.glue.
. 85194 IN NS ns9.opennic.glue.
. 85194 IN NS ns3.opennic.glue.
. 85194 IN NS ns4.opennic.glue.
. 85194 IN NS ns10.opennic.glue.;; Query time: 71 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Nov 17 13:21:30 EET 2015
;; MSG SIZE rcvd: 174
and those logs
Nov 17 13:22:40 nucleus.cell.org dnsmasq[4837]: query[NS] . from 127.0.0.1
Nov 17 13:22:40 nucleus.cell.org dnsmasq[4837]: forwarded . to 127.0.0.1
The same command from guest returns the same result :
....
....
;; Query time: 71 msec
;; SERVER: 192.168.101.1#53(192.168.101.1)
;; WHEN: Tue Nov 17 13:21:30 EET 2015
;; MSG SIZE rcvd: 174
and logs
Nov 17 13:25:38 nucleus.cell.org dnsmasq[4837]: query[NS] . from 127.0.0.1
Nov 17 13:25:38 nucleus.cell.org dnsmasq[4837]: forwarded . to 127.0.0.1
Lets dig google.com now from host :
Logs:
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: query[A] google.com from 127.0.0.1
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: forwarded google.com to 127.0.0.1
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.102
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.100
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.101
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.138
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.113
Nov 17 13:26:59 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.139
While from guest
Logs:
Nov 17 13:28:20 nucleus.cell.org dnsmasq[4837]: query[A] google.com from 127.0.0.1
Nov 17 13:28:20 nucleus.cell.org dnsmasq[4837]: forwarded google.com to 127.0.0.1
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.101
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.139
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.138
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.100
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.113
Nov 17 13:28:21 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.102
So DNS requests seems working good but what's going on with ping ?
Ping from host 'ping google.com'
Logs:
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: query[A] google.com from 127.0.0.1
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.102
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.113
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.100
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.138
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.139
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: cached google.com is 64.233.166.101
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: query[PTR] 102.166.233.64.in-addr.arpa from 127.0.0.1
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: forwarded 102.166.233.64.in-addr.arpa to 127.0.0.1
Nov 17 13:30:00 nucleus.cell.org dnsmasq[4837]: reply 64.233.166.102 is wm-in-f102.1e100.net
while from guest
Logs
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: query[A] google.com from 127.0.0.1
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: forwarded google.com to 127.0.0.1
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.138
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.139
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.101
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.113
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.100
Nov 17 13:33:33 nucleus.cell.org dnsmasq[4837]: reply google.com is 64.233.166.102
It seems to me that it cannot forward the reply to the guest but why's that ? Any idea ?
Thank you.
EDIT :
When VM is starting i can see these data at logs :
Nov 17 14:24:30 nucleus.cell.org dnsmasq-dhcp[4937]: DHCPREQUEST(virbr1) 192.168.101.183 52:54:00:a8:fb:97
Nov 17 14:24:30 nucleus.cell.org dnsmasq-dhcp[4937]: DHCPACK(virbr1) 192.168.101.183 52:54:00:a8:fb:97 webdev
and tcpdumping guest's network interface
sudo tcpdump -i virbr1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on virbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:28:15.248042 IP 192.168.101.183.40297 > nucleus.cell.org.domain: 62932+ A? www.google.com. (32)
14:28:15.248177 IP nucleus.cell.org.domain > 192.168.101.183.40297: 62932 6/0/0 A 64.233.166.106, A 64.233.166.104, A 64.233.166.103, A 64.233.166.105, A 64.233.166.99, A 64.233.166.147 (128)
14:28:20.251765 ARP, Request who-has 192.168.101.183 tell nucleus.cell.org, length 28
14:28:20.251919 ARP, Reply 192.168.101.183 is-at 52:54:00:a8:fb:97 (oui Unknown), length 28
Hmm an ARP reply problem ?
Last edited by netpumber (2015-11-26 17:14:45)
Offline
Hmm an ARP reply problem ?
Run "arp" in the guest, to answer your own question.
You're using VMs, presumably Qemu, but you've shown no networking/iptables info for those VMs to be able to communicate with the host, and the Internet.
Offline
Run "arp" in the guest, to answer your own question.
Actually i cannot run arp on the guest because it isn't installed and i cannot install it because it has no internet.
Yes i'm using KVM with QEMU without any iptables. What i've done is to create a new network interface from the virt-manager and use that one for VM.
It's configuration is :
Network :192.168.101.0/24
DHCP range : 192.168.101.128 - 192.168.101.254
Forwarding : NAT to enp3s0
The thing is that before try DNScrypt with DNSmasq, both host and guest had internet access.
I don't know what else from network configuration should i post.
Last edited by netpumber (2015-11-17 21:32:20)
Offline
Actually i cannot run arp on the guest
Alternative:
ip neigh
Offline
ip neigh
Returns nothing.
Something else that i should say.
Both dnsmasq and libvirtd are enabled to run at startup. But i realized that dnsmasq didn't started because another dnsmasq process has been running. Under those conditions Host PC hasn't got internet access too. To solve it i have to killall dnsmasq as root and then run
systemctl start dnsmasq
with that way only host has internet access.
Maybe libvirtd is responsible for the first ran of dnsmasq because if i
systemctl disable libvirtd
there is no dnsmasq process running. But still, under those conditions, host PC hasn't got internet access too because dnsmasq service cannot find the virbr1 device (libvirtd create this). Same occurs even if i tell the libvirtd service run after dnsmasq service.
Last edited by netpumber (2015-11-17 21:35:53)
Offline
How exactly is the network setup? Did you activate ip forwarding on the host?
Offline
I'm using systemd-networkd and here is the configuration of the host
[Match]
Name=enp3s0
[Network]
#DNS=192.168.1.1
# DNS is setted to 127.0.0.1 to use dnsmaq through dnscrypt-proxy
DNS=127.0.0.1
# Need this to forward network traffic from VMs.
IPForward=yes
[Address]
Address=192.168.1.2/24
[Route]
Gateway=192.168.1.1
Last edited by netpumber (2015-11-18 18:00:00)
Offline
ip neigh
Returns nothing.
Your guest needs to know the MAC address of your host, so that it can communicate to the host.
You need to fix this basic routing/firewalling issue. Forget about DNS until you have *this* basic connectivity working.
Offline
This is only the host-network. How do you activate NAT? In virt-manager? Can you show the rules which are actually active (iptables -L)?
Offline
Yes, i have create a NAT with virt-manager.
iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootpsChain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
ACCEPT all -- anywhere 192.168.101.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.101.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachableChain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:bootpc
ACCEPT udp -- anywhere anywhere udp dpt:bootpc
Offline
"iptables -L" is a crappy summary that does not show the rule order - and the rule order is critically important. Otherwise, the order of these 2 lines is nonsensical:
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
ACCEPT all -- anywhere 192.168.101.0/24 ctstate RELATED,ESTABLISHED
Instead, show the output of:
iptables-save
Offline
# Generated by iptables-save v1.4.21 on Wed Nov 18 21:00:22 2015
*mangle
:PREROUTING ACCEPT [1382:1111819]
:INPUT ACCEPT [1381:1111787]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1164:163686]
:POSTROUTING ACCEPT [1169:163846]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Wed Nov 18 21:00:22 2015
# Generated by iptables-save v1.4.21 on Wed Nov 18 21:00:22 2015
*nat
:PREROUTING ACCEPT [53:7950]
:INPUT ACCEPT [13:1279]
:OUTPUT ACCEPT [274:18090]
:POSTROUTING ACCEPT [274:18090]
-A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Wed Nov 18 21:00:22 2015
# Generated by iptables-save v1.4.21 on Wed Nov 18 21:00:22 2015
*filter
:INPUT ACCEPT [1381:1111787]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1164:163686]
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
COMMIT
# Completed on Wed Nov 18 21:00:22 2015
Offline
thanks for pointing that one out.
Offline
It's configuration is :
Network :192.168.101.0/24
DHCP range : 192.168.101.128 - 192.168.101.254
Forwarding : NAT to enp3s0
According to your post, there's only NAT for 192.168.122.0/24.
You could test if this is the problem by adding some according rules for that ip range.
Last edited by c0mm0ner (2015-11-18 19:25:17)
Offline
I 'll try tomorrow to setup a bridge connection as this document describes (but using systemd-networkd) and i will inform you.
You could test if this is the problem by adding some according rules for that ip range.
What kind of rules do you mean ? Can you give me an example ?
Last edited by netpumber (2015-11-18 22:00:56)
Offline
I would simply copy the rules which are in effect for 192.168.122.0/24 and alter the ip range to 192.168.101.0/24.
-A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
For the first line it would be
#iptables -A POSTROUTING -s 192.168.101.0/24 -d 224.0.0.0/24 -j RETURN
Do this for all the rules and try again.
Maybe it would be more likely to work out, if you wouldn't use a helper application but setup and connect your vms by hand first, so that you can exactly work out, what is going on.
Last edited by c0mm0ner (2015-11-19 11:44:35)
Offline
Hello again. I took it from the start and tried to set up a bridged network for the host and VM.
I totally disable DNSCrypt and DNSmasq :
# systemctl disable dnscrypt-proxy
# systemctl disable dnsmasq
Then i turn the physical device into bridge mode and create a new one to link with it (all done using systemd-networkd) :
/etc/systemd/network/wired.network:
[Match]
Name=enp3s0
[Network]
Bridge=br0
/etc/systemd/network/br0.netdev:
[NetDev]
Name=br0
Kind=bridge
/etc/systemd/network/br0.network:
[Match]
Name=br0
[Network]
DNS=192.168.1.1
Address=192.168.1.2/24
Gateway=192.168.1.1
After a reboot i had that :
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether d0:50:99:63:ab:44 brd ff:ff:ff:ff:ff:ff
inet6 fe80::d250:99ff:fe63:ab44/64 scope link
valid_lft forever preferred_lft forever
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 8a:ae:05:74:3e:2e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::88ae:5ff:fe74:3e2e/64 scope link
valid_lft forever preferred_lft forever
# networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp3s0 ether degraded configured
3 br0 ether routable configured
Then i tried to somehow connect VM's NIC with the bridge device 'br0'. Before do that i disable all virtual networks i had created and default to. I don't know if what i did is correct but i couldn't find a tutorial online on how to set a bridge device in virt-manager.
So what i did is shown below.
After that configuration i run the VM and here are some results
@Host :
A new net device has been created
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether d0:50:99:63:ab:44 brd ff:ff:ff:ff:ff:ff
inet6 fe80::d250:99ff:fe63:ab44/64 scope link
valid_lft forever preferred_lft forever
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 8a:ae:05:74:3e:2e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::88ae:5ff:fe74:3e2e/64 scope link
valid_lft forever preferred_lft forever
4: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 500
link/ether fe:54:00:a8:fb:97 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc54:ff:fea8:fb97/64 scope link
valid_lft forever preferred_lft forever
# networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp3s0 ether degraded configured
3 br0 ether routable configured
4 vnet0 ether degraded unmanaged
# ip neigh
192.168.1.106 dev br0 lladdr 52:54:00:a8:fb:97 STALE
192.168.1.1 dev br0 lladdr 00:18:39:2b:2d:5a REACHABLE
I can also ping the VM
# ping 192.168.1.106
PING 192.168.1.106 (192.168.1.106) 56(84) bytes of data.
64 bytes from 192.168.1.106: icmp_seq=1 ttl=64 time=0.138 ms
64 bytes from 192.168.1.106: icmp_seq=2 ttl=64 time=0.170 ms
@VM :
# ip neigh
192.168.1.1 dev eth0 lladdr 00:18:39:2b:2d:5a STALE
192.168.1.2 dev eth0 lladdr 8a:ae:05:74:3e:2e STALE
and also i can ping the host and the gateway, but i cannot ping google.com. The problem still exists.
iptables-save @ host returns :
# Generated by iptables-save v1.4.21 on Thu Nov 19 23:23:16 2015
*filter
:INPUT ACCEPT [9805105:54442413063]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [9795280:54389590419]
COMMIT
# Completed on Thu Nov 19 23:23:16 2015
What might be now the problem ?
Offline
If you are able to ping the gateway from your VM, you do have access to your network. I would now assume either a missing route to the Internet or a problem with name resolution.
Did you assign the gateway inside the VM? Did you properly assign namenservers inside the VM?
Offline
Sorry for being late to answer but my PSU burned out.
So it was a route problem.
At host PC command 'ip route show table main' returned:
default via 192.168.1.1 dev br0 proto static
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.2
While at VM it returned :
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.4
I solved that problem by following that info i found on another forum talking about network-manager:
Edit connections -> Wireless -> Choose your connection -> Edit -> Ipv4 Settings
Replace DHCP by Manual in method selection.
Add a line for IP address. Fill all the fields including Gateway. If you leave gateway=0.0.0.0, you will have no default gateway.
Make sure you do it while the applet is disabled (right click and uncheck "Networking enabled").
Do not touch the "routes" button unless you need to add some additional routing.Press enter and make sure you press the "Apply" button, otherwise your settings will not be saved. After you close the dialog, open it again and check that your changes are saved.
If they are not, try several times - I noticed that the GUI sometimes drop your changes.Now, activate the applet and you should be ok.
Last edited by netpumber (2015-11-26 17:14:16)
Offline