My letsencrypt ssl certs on my VPS are not trusted on other machines

Doron.Behar · 2018-03-23 10:00:27

I have a website which I host on a VPS (running Arch Linux BTW) and I have a domain which I registered an SSL certificate with Lets Encrypt. I run on my VPS a Nextcloud instance which I have a `nextcloud` sub-domain pointed to the server's address. This nextcloud sub-domain also has another certificate issued with Lets Encrypt.

As some of you may already know, every 3 months Lets-encrypt's certificates have to be updated. I have this server up and running for more then 3 months now so I've updated my nextcloud sub-domain certificate 2 months ago.

My problem is that for some reason, for more then 4-5 months now (I have updated my certificate once since), some programs fail to recognize my certificates. I don't think it's relevant to list all of these programs here so I'll focus just on `curl`:

$ curl [url]https://nextcloud.mydomain.com[/url]
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: [url]https://curl.haxx.se/docs/sslcerts.html[/url]:

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

My question is this:

How is it possible that every day or so someone in the world is buying a new domain and is registering a new SSL certificate for it, whether it is done every 3 months with Lets Encrypt or not and the Arch Linux packages starting with `ca-certificates` are updated every two months or so? Why is there no automatic mechanism (using cron or systemd) for updating the certificates automatically by downloading them straight from the CAs?

Perhaps I don't understand yet how the whole business works and I haven't investigated how is this working on other distros but my only wish is that my `nextcloud` sub-domain certificate will be trusted natively.

Trilby · 2018-03-23 12:01:24

Doron.Behar wrote:

Why is there no automatic mechanism (using cron or systemd) for updating the certificates automatically by downloading them straight from the CAs?

You can, and should be, automating the updating of your server's certs:
https://wiki.archlinux.org/index.php/Le … ic_renewal

This is completely different from the client machine's CA certs. Your servers certs would not be in packages like ca-certs, that's not how it works ... for what should be obvious reasons: you'd not have enough inodes on your drive to store an individual cert file for every server you might interact with.

Doron.Behar wrote:

Perhaps I don't understand yet how the whole business works and I haven't investigated how is this working on other distros but my only wish is that my `nextcloud` sub-domain certificate will be trusted natively.

We can help with that, but not if you provide only obfuscated output. What commands have you run to generate certificates? Are you using apache or nginx (or something else)? Have you properly configured apache/nginx to use your current certs?

On a slightly tangential note that may simplify the process: you do not need separate certificates for each subdomain/virtual-host. Letsencrypt/certbot allows many virtual hosts and/or subdomains on a single certificate which makes configuration a bit simpler: there's only one certificate and one key file to point to for the configuration of every subdomain.

Last edited by Trilby (2018-03-23 12:01:48)

Doron.Behar · 2018-03-23 12:51:50

Trilby wrote:

Doron.Behar wrote:
Why is there no automatic mechanism (using cron or systemd) for updating the certificates automatically by downloading them straight from the CAs?
You can, and should be, automating the updating of your server's certs:
https://wiki.archlinux.org/index.php/Le … ic_renewal
This is completely different from the client machine's CA certs. Your servers certs would not be in packages like ca-certs, that's not how it works ... for what should be obvious reasons: you'd not have enough inodes on your drive to store an individual cert file for every server you might interact with.

I know and I read the article about Lets Encrypt. what I meant in my original post was the client certificates, anyway that answered this part of my question.

Trilby wrote:

Doron.Behar wrote:
Perhaps I don't understand yet how the whole business works and I haven't investigated how is this working on other distros but my only wish is that my `nextcloud` sub-domain certificate will be trusted natively.
We can help with that, but not if you provide only obfuscated output. What commands have you run to generate certificates? Are you using apache or nginx (or something else)? Have you properly configured apache/nginx to use your current certs?

In order to generate the certificates and complete the ACME challenge, I disabled temporarily the nextcloud instance and I've manually put the acme challenge file in the webroot directory. I have used eventually this command:

certbot certonly --email email@example.com -d nextcloud.domain.tld --manual

First of all, I use Apache. Plus, I don't think there is something wrong with the way I configured the Nextcloud web-server because on the browser, the certificate is well trusted since I don't get any warning from Firefox about it. I've used the default settings for through almost all of the configuration process and I followed the WiKi articles about apache and Nextcloud, specifically the sections regarding the SSL certification for Apache.

Trilby wrote:

On a slightly tangential note that may simplify the process: you do not need separate certificates for each subdomain/virtual-host. Letsencrypt/certbot allows many virtual hosts and/or subdomains on a single certificate which makes configuration a bit simpler: there's only one certificate and one key file to point to for the configuration of every subdomain.

Yea I read about it on lets encrypt's forums while I was trying to investigate my problem prior to this post. I will investigate that as well but I would rather do this after I'd solve the problem with the separate certificate for the nextcloud sub-domain.

Trilby · 2018-03-23 14:00:08

Doron.Behar wrote:

I don't think there is something wrong with the way I configured the Nextcloud web-server because on the browser, the certificate is well trusted since I don't get any warning from Firefox about it.

I'd like to pin down where the problem is. In your tests your machine(s) are both the client and the server, so it is not easy to narrow down where to look. This is why I asked you to stop obfuscating your information: what is the server name, how can we attempt to replicate this error?

Are you sure it's actually all good in the browser, some browsers are not particularly verbose about minor certificate errors. Try w3m. But if it is fine in a browser but fails with curl, then the problem is likely client side. The next steps in troubleshooting diverge enormously whether we are tracking a client-side or server-side problem, so help pin that down.

Doron.Behar · 2018-03-23 14:20:50

My domain is doronbehar.com and the nextcloud instance is sitting on nextcloud.doronbehar.com . w3m fails with the same error as curl:

unable to get local issuer certificate: accept? (y/n)

Trilby · 2018-03-23 14:30:22

Thanks. I can confirm the error with nextcloud.doronbehar.com while doronbehar.com works fine. So it is a server-side issue specific to your nextcloud subdomain. It looks like the certificate isn't even provided. Can you post your /etc/httpd/conf/extra/httpd-vhosts.conf (or whatever file configures your nextcloud subdomain).

Also note that firefox seemingly not taking issue with the nextcloud site is a diagnostic red-herring. Perhaps there was a different color or some such in the url bar to indicate the problem, but overall firefox is far too quite about certificate issues - it is easy to not notice when you visit a site with a bad or missing certificate (at least with firefox's default configuration).

EDIT: scratch that. The cert is there. I'm not yet sure why curl is complaining:
https://www.ssllabs.com/ssltest/analyze … nbehar.com

EDIT2: the vhost config would still be useful. I'm a bit out of my depth on this speculation, but I'm wondering if perhaps your config has the certificate but not the key file (or perhaps even vice-versa) - both are required. The SSL labs report above shows an "extra download" indicator for the second step in the certification path which I'm speculating would be the key file. It will say "Sent by server" there on properly configured servers.

Your vhosts config should have 2 lines like the following for each server block:

SSLCertificateKeyFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/privkey.pem"
SSLCertificateFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/fullchain.pem"

Last edited by Trilby (2018-03-23 14:49:40)

Doron.Behar · 2018-03-23 14:56:43

I have the file `/etc/httpd/conf/extra/nextcloud.conf which looks like that:

<Directory /usr/share/webapps/nextcloud/>
    Options FollowSymlinks
    AllowOverride all
    Require all granted
    php_admin_value open_basedir "/srv/http/:/dev/urandom:/tmp/:/usr/share/pear/:/usr/share/webapps/nextcloud/:/etc/webapps/nextcloud"
</Directory>

<VirtualHost _default_:80>
    ServerAdmin me@doronbehar.com
    DocumentRoot /usr/share/webapps/nextcloud
    ServerName nextcloud.doronbehar.com
    ErrorLog /var/log/httpd/nextcloud.info-error_log
    CustomLog /var/log/httpd/nextcloud.info-access_log common
</VirtualHost>

And the file /etc/httpd/conf/extra/httpd-ssl.conf which is also included in /etc/httpd/conf/httpd.conf has this content:

Listen 443
SSLCipherSuite HIGH:MEDIUM:!MD5:!RC4:!3DES
SSLProxyCipherSuite HIGH:MEDIUM:!MD5:!RC4:!3DES
SSLHonorCipherOrder on 
SSLProtocol all -SSLv3
SSLProxyProtocol all -SSLv3
SSLPassPhraseDialog  builtin
SSLSessionCache        "shmcb:/run/httpd/ssl_scache(512000)"
SSLSessionCacheTimeout  300
<VirtualHost _default_:443>
DocumentRoot "/usr/share/webapps/nextcloud"
ServerName nextcloud.doronbehar.com:443
ServerAdmin doron.behar@gmail.com
ErrorLog "/var/log/httpd/error_log"
TransferLog "/var/log/httpd/access_log"
SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/cert.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/privkey.pem"
<FilesMatch "\.(cgi|shtml|phtml|php)$">
    SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/srv/http/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0
CustomLog "/var/log/httpd/ssl_request_log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

/srv/http is linked to /usr/share/webapps/nextcloud and /etc/httpd/conf/extra/httpd-vhosts.conf is not included in /etc/httpd/conf/httpd.conf.

EDIT:

Only after I posted my reply I saw your edit. the problem was this line:

SSLCertificateFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/cert.pem"

It should be:

SSLCertificateFile "/etc/letsencrypt/live/nextcloud.doronbehar.com/fullchain.pem"

I guess firefox knows how to work around such situations that's why it did not complain. Thanks for you guidance!

Last edited by Doron.Behar (2018-03-23 15:03:26)

Trilby · 2018-03-23 15:05:16

That's the problem. You are sending the cert.pem rather than the fullchain.pem. Fix that and restart httpd.

Alternatively add a third line for the chain file if you want to keep these separate:
https://certbot.eff.org/docs/using.html … rtificates

EDIT: you beat me to it in your edit! If all is well now, please mark your thread as SOLVED.

Last edited by Trilby (2018-03-23 15:09:31)

Arch Linux

#1 2018-03-23 10:00:27

My letsencrypt ssl certs on my VPS are not trusted on other machines

#2 2018-03-23 12:01:24

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#3 2018-03-23 12:51:50

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#4 2018-03-23 14:00:08

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#5 2018-03-23 14:20:50

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#6 2018-03-23 14:30:22

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#7 2018-03-23 14:56:43

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

#8 2018-03-23 15:05:16

Re: My letsencrypt ssl certs on my VPS are not trusted on other machines

Board footer