You are not logged in.

#1 2022-08-03 21:05:37

parski
Member
Registered: 2021-10-24
Posts: 7

[Solved] Using host gateway for DHCP for LXC containers

Hello,

I'm a FreeBSD user playing around with LXD and LXC as an alternative to jails. They seem to be very similar but more flexible.

One thing I'm used to with jails is that upon creating one you can enable vnet, a virtual network stack for each jail. By default they are exposed to the LAN like any other machine, I can assign them an IP address and it gets leased by the host's gateway. For example: If my FreeBSD host 10.110.110.5 is connected to my router 10.110.110.1 I can create a jail with vnet and assigned IP of for example 10.110.110.6 and I will see it in my LAN. If I host a web server in it I can access it from any machine connected to the same network. Very simple.

I'm having problems doing the same thing with LXC and I'm not sure what I'm doing wrong. I've been looking at the Linux Containers documentation but I'm not that good at network things. I saw one interesting bit in the Arch Wiki that says:

There are several main setups to consider:

    1) A host bridge
    2) A NAT bridge

The host bridge requires the host's network manager to manage a shared bridge interface. The host and any lxc will be assigned an IP address in the same network (for example 192.168.1.x). This might be more simplistic in cases where the goal is to containerize some network-exposed service like a webserver, or VPN server. The user can think of the lxc as just another PC on the physical LAN, and forward the needed ports in the router accordingly. The added simplicity can also be thought of as an added threat vector, again, if WAN traffic is being forwarded to the lxc, having it running on a separate range presents a smaller threat surface.

I created a bridge interface when I initalized LXD:

[parski@mango ~]$ lxd init
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]: no
Would you like to connect to a MAAS server? (yes/no) [default=no]:
[ins]Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=lxdbr0]:[/ins]
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the LXD server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]:
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

When I list them I have a bridge interface as well as my host's ethernet and WLAN interfaces.

[parski@mango ~]$ lxc network list
+--------+----------+---------+---------------+---------------------------+-------------+---------+---------+
|  NAME  |   TYPE   | MANAGED |     IPV4      |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+--------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| enp3s0 | physical | NO      |               |                           |             | 0       |         |
+--------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| lxdbr0 | bridge   | YES     | 10.52.83.1/24 | fd42:806b:fbe4:ccd7::1/64 |             | 2       | CREATED |
+--------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| wlan0  | physical | NO      |               |                           |             | 0       |         |
+--------+----------+---------+---------------+---------------------------+-------------+---------+---------+

If I launch a container it get's an IP address in the same subnet as the bridge which is different from my LAN that the host is connected to. My host can access the containers via their IP addresses but another machine on my LAN can't because my router can't route to the subnet used by LXD. Does that make sense?

This is different from what I'm reading in the Arch Wiki, quoted above, in particular the part "lxc as just another PC on the physical LAN". How can I let my router assign IP addresses to the LXC containers so that they are just like another PC on the physical LAN?

If I run a web server in a container I am able to proxy that to a port on my host, like so:

lxc config device add <container-name> <proxy-name> proxy "listen=tcp:<host-ip-address>:<destination-port>" connect=tcp:127.0.0.1:<source-port>

But if I want to run five different web applications on the host in different containers I'd have to 10.110.110.5:1111, 10.110.110.5:2222, 10.110.110.5:3333, and so on. With my jails I can just let them have their own dedicated IP addresses on my network like 10.110.110.6, 10.110.110.7, 10.110.110.8. I much prefer this.

As a final note this is a competely fresh install. No Docker to interfere or anything like that. I installed git, yay, vim and make I think. Then I installed lxd and lxc. When I installed lxd it replaced the iptables package with iptables-nft or something like that. That's it.

Last edited by parski (2022-08-04 11:26:14)

Offline

#2 2022-08-04 11:25:21

parski
Member
Registered: 2021-10-24
Posts: 7

Re: [Solved] Using host gateway for DHCP for LXC containers

I got help from LXC people. Apparently, to have the gateway of your host handle a container you don't want to use a bridge at all. You want to use a macvlan NIC. Now it works great!

Last edited by parski (2022-08-04 11:26:01)

Offline

Board footer

Powered by FluxBB