You are not logged in.

#1 2021-10-06 23:20:30

borgermatthew
Member
Registered: 2021-10-06
Posts: 2

[SOLVED] Vagrant lxd connection issues

My setup might be unique but I'm posting to see if anyone either has this combination working or has an idea what might be going wrong.

I have been using (it's been a couple of months since I think this setup worked) vagrant, lxd and a vagrant plugin called vagrant-lxd. Vagrant and lxd are installed from the official repos. The vagrant-lxd plugin is installed using vagrant's internal machinery which dumps the gems in a home dot folder. Currently the plugin throws a stack trace from openssl about a bad certificate when trying to communicate with the lxd rest endpoint. I'm aware lxd uses a self signed certificate but from what I can tell the plugin is configured to not verify that. I can manually connect to the endpoint using the openssl s_client and from Firefox if I accept the self signed cert. I've tried downgrading vagrant but I think the issue is more related to openssl or the plugin machinery. Although the vagrant plugins run within a virtual environment against ruby 2.7 so that hasn't changed in a while.

I spun up a few containers to test this setup in a clean environment. The following reproduces the issue within a fresh arch vm.
pacman -S lxd vagrant
lxd init --auto
lxc config set core.https_address 127.0.0.1
vagrant init --minimal archlinux/current
vagrant up --provider=lxd

This setup works on Ubuntu 21.04. Lxd is the same version, vagrant-lxd plugin is the same version. Ubuntu has Vagrant 2.2.9 and their Openssl version is 1.1.1.j

Below is the tail of the debug log from Vagrant. I modified the faraday source to add some additional output about the http get requests it was making. You can see the line starting with "DEBUG perform_request"

DEBUG syncedfolder: Checking in: lxd
INFO warden: Calling IN action: #<VagrantLXD::Action::ConnectionValidate:0x000055a46cd2f558>
DEBUG certificate: Checking for existing client certificate in /home/matthew/.vagrant.d/data/lxd...
DEBUG certificate: Found usable certificate under /home/matthew/.vagrant.d/data/lxd
DEBUG perform_request GET: 127.0.0.1:8443/1.0/images use_ssl? true verify_mode 0
OpenSSL::SSL::SSLError SSL_read: sslv3 alert bad certificate
ERROR warden: Error occurred: SSL_read: sslv3 alert bad certificate
INFO warden: Beginning recovery process...
INFO warden: Recovery complete.
INFO warden: Beginning recovery process...
INFO warden: Recovery complete.
INFO environment: Released process lock: machine-action-17e28eada5064c4b680c0ba26d333e0a
INFO environment: Running hook: environment_unload
INFO runner: Running action: environment_unload #<Vagrant::Action::Builder:0x000055a46c93c838>
/usr/lib/ruby/2.7.0/openssl/buffering.rb:182:in `sysread_nonblock': SSL_read: sslv3 alert bad certificate (Faraday::SSLError)
    from /usr/lib/ruby/2.7.0/openssl/buffering.rb:182:in `read_nonblock'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:210:in `rbuf_fill'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:191:in `readuntil'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:201:in `readline'
    from /usr/lib/ruby/2.7.0/net/http/response.rb:42:in `read_status_line'
    from /usr/lib/ruby/2.7.0/net/http/response.rb:31:in `read_new'
    from /usr/lib/ruby/2.7.0/net/http.rb:1528:in `block in transport_request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1519:in `catch'
    from /usr/lib/ruby/2.7.0/net/http.rb:1519:in `transport_request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1492:in `request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1485:in `block in request'
    from /usr/lib/ruby/2.7.0/net/http.rb:933:in `start'
    from /usr/lib/ruby/2.7.0/net/http.rb:1483:in `request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1241:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:88:in `perform_request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:44:in `block in call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:99:in `with_net_http_connection'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:39:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/response.rb:8:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/middleware/follow_redirects.rb:72:in `perform_with_redirection'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/middleware/follow_redirects.rb:60:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/rack_builder.rb:143:in `build_response'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/connection.rb:387:in `run_request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/connection.rb:138:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/sawyer-0.8.2/lib/sawyer/agent.rb:94:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/connection.rb:139:in `request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/connection.rb:38:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/client/images.rb:21:in `images'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/driver.rb:455:in `connection_usable?'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/driver.rb:158:in `validate!'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/action.rb:93:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/warden.rb:48:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/warden.rb:48:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/builder.rb:149:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/runner.rb:89:in `block in run'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/runner.rb:89:in `run'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:246:in `action_raw'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:215:in `block in action'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/environment.rb:614:in `lock'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:201:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:201:in `action'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/batch_action.rb:86:in `block (2 levels) in run'
/usr/lib/ruby/2.7.0/openssl/buffering.rb:182:in `sysread_nonblock': SSL_read: sslv3 alert bad certificate (OpenSSL::SSL::SSLError)
    from /usr/lib/ruby/2.7.0/openssl/buffering.rb:182:in `read_nonblock'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:210:in `rbuf_fill'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:191:in `readuntil'
    from /usr/lib/ruby/2.7.0/net/protocol.rb:201:in `readline'
    from /usr/lib/ruby/2.7.0/net/http/response.rb:42:in `read_status_line'
    from /usr/lib/ruby/2.7.0/net/http/response.rb:31:in `read_new'
    from /usr/lib/ruby/2.7.0/net/http.rb:1528:in `block in transport_request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1519:in `catch'
    from /usr/lib/ruby/2.7.0/net/http.rb:1519:in `transport_request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1492:in `request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1485:in `block in request'
    from /usr/lib/ruby/2.7.0/net/http.rb:933:in `start'
    from /usr/lib/ruby/2.7.0/net/http.rb:1483:in `request'
    from /usr/lib/ruby/2.7.0/net/http.rb:1241:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:88:in `perform_request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:44:in `block in call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:99:in `with_net_http_connection'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/adapter/net_http.rb:39:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/response.rb:8:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/middleware/follow_redirects.rb:72:in `perform_with_redirection'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/middleware/follow_redirects.rb:60:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/rack_builder.rb:143:in `build_response'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/connection.rb:387:in `run_request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/faraday-0.17.4/lib/faraday/connection.rb:138:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/sawyer-0.8.2/lib/sawyer/agent.rb:94:in `call'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/connection.rb:139:in `request'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/connection.rb:38:in `get'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/hyperkit-1.3.0/lib/hyperkit/client/images.rb:21:in `images'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/driver.rb:455:in `connection_usable?'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/driver.rb:158:in `validate!'
    from /home/matthew/.vagrant.d/gems/2.7.4/gems/vagrant-lxd-0.5.4/lib/vagrant-lxd/action.rb:93:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/warden.rb:48:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/warden.rb:48:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/builder.rb:149:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/runner.rb:89:in `block in run'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/action/runner.rb:89:in `run'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:246:in `action_raw'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:215:in `block in action'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/environment.rb:614:in `lock'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:201:in `call'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/machine.rb:201:in `action'
    from /opt/vagrant/embedded/gems/2.2.18/gems/vagrant-2.2.18/lib/vagrant/batch_action.rb:86:in `block (2 levels) in run'

Has anyone encountered this or a similar issue? Or have an idea about a systemic change to archlinux within the past two months which would influence this failure?

Last edited by borgermatthew (2021-10-11 21:32:51)

Offline

#2 2021-10-11 21:31:37

borgermatthew
Member
Registered: 2021-10-06
Posts: 2

Re: [SOLVED] Vagrant lxd connection issues

I figured out the issue. It stems from the client certificated used to authenticate with LXD. The certificate generated by vagrant-lxd at .vagrant.d/data/lxd uses a SHA1 signature which LXD refuses to trust now. After creating my own certificate using SHA256, the certificate issues went away. I have opened a pull request with the vagrant-lxd project to use SHA256 going forward. I tried doing some digging to see what component no longer accepts SHA1, be it OpenSSL, Arch's configuration of OpenSSL or LXD but I couldn't track that down from a search of the project's commit logs. If anyone has knowledge on that I would love to know, otherwise I'm just posting this for anyone who stumbles across this issue.

Offline

Board footer

Powered by FluxBB