[solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

cfr · 2024-12-06 03:55:47

I am trying to learn Lua. I have no programming background whatsoever. So this is probably a very stupid question, but I am completely stuck. I'm asking here because I *think*, though I'm not sure that the specific configuration of `ssl/tsl` may be relevant, because I'm possibly trying to use the wrong `.pem` to verify the server's certificate. I might be completely wrong about the cause, though.

I'm trying to adapt an example from *Programming in Lua*. which is designed for Lua 5.0 That is more-or-less OK. However, it also seems to be designed for a rather different internet, where servers didn't automatically switch requests to `https`.

This works:

socket = require("socket")
do
  local https = require ("ssl.https")
  local body, code, headers, status = https.request("https://www.google.com")
  print(status)
end

But I'd like to figure out how to use `ssl.wrap` to wrap a connection using `socket.tcp`. If I try to connect to google and disable verifying the certificate, I can get a connection, but that obviously defeats the purpose. Moreover, this fails utterly with every other site I've tried. I assume I'm failing to specify the parameters correctly when calling `ssl.wrap()`.

The library's wiki provides an example configuration:

Bruno Silvestre wrote:

local params = {
  mode = "client",
  protocol = "any",
  key = "/etc/certs/clientkey.pem",
  certificate = "/etc/certs/client.pem",
  cafile = "/etc/certs/CA.pem",
  verify = "peer",
  options = {"all", "no_sslv3"}
}

but this is aimed at connecting to a server on the same machine - at least, that's how the example there is set up.

I also looked at this blog post, which suggests

Paul Kulchenko wrote:

local params = {
  mode = "client",
  protocol = "tlsv1",
  cafile = "/path/to/downloaded/cacert.pem", --<-- added cafile parameters
  verify = "peer", --<-- changed "none" to "peer"
  options = "all",
}

However, it suggests downloading the certificates from a site I know nothing about, rather than using those already installed, and I would prefer not to do that.

So I've been trying to figure out how to use the certificates already installed. Based on the `README`s in the relevant directories under `/etc` and the manual page for `update-ca-trust`, I came up with the following:

socket = require("socket")
ssl = require("ssl")
local params = {
  mode = "client",
  protocol = "any",    -- fails with "unsupported protocol" for any of tsl* versions ????
  -- cafile = "/etc/ca-certificates/extracted/tls-ca-bundle.pem",
  cafile = "/etc/ssl/cert.pem", 
  verify = "peer",
  options = "all",  
}
host = "www.google.com"         -
local conn = socket.tcp()
local res = conn:connect(host,443) 
print(res)
conn, more = ssl.wrap(conn, params)
print(conn, more)
local res, more = conn:dohandshake()
print(res, more)
conn:close()

which prints

1.0
SSL connection: 0x62178ff899f8  nil
false   certificate verify failed

So the connection is wrapped, but the subsequent handshake fails because my code fails to verify the certificate provided by the server (I *think*?).

Can somebody give me a hint as to what I need to do here?

Last edited by cfr (2024-12-08 16:19:09)

lmn · 2024-12-07 09:49:38

The problem you're running into is that you need to provide a valid certificate of the relevant CA for the given server certificate.
These are usually bundled together into a single file called a ca-bundle (provided by ca-certificates-utils). These are then to be used by some of the software on the system as a source of truth.

However, it suggests downloading the certificates from a site I know nothing about, rather than using those already installed, and I would prefer not to do that.

The site in question is https://curl.se/docs/caextract.html. This is part of the website that implements curl and provides an extracted CA-bundle from Mozilla.
But you can use the system ca-bundle by specifying it's location:

/etc/ssl/certs/ca-bundle.crt

Using this in your code leads to a successful connection:

1.0
SSL connection: 0x59e6012692f8  nil
true    nil

The ca-bundle is essentially just a concatanation of some ca-root-certs. So you could find all relevant root certs beforehand and group them into a bundle yourself and use that. In the case of Google

cat /etc/ssl/certs/GTS_Root_R* > google.pem

Also see https://man.archlinux.org/man/openssl.1ssl for more information.

cfr · 2024-12-08 07:15:16

lmn wrote:

The site in question is https://curl.se/docs/caextract.html. This is part of the website that implements curl and provides an extracted CA-bundle from Mozilla.

Thanks.

lmn wrote:

But you can use the system ca-bundle by specifying it's location:
/etc/ssl/certs/ca-bundle.crt

This is a symbolic link which points to the same file as the symbolic link specified in the code I posted above:

$ ls -l /etc/ssl/cert.pem /etc/ssl/certs/ca-bundle.crt /etc/ssl/certs/ca-certificates.crt
lrwxrwxrwx 1 root root 46 Meh  18 19:36 /etc/ssl/cert.pem -> ../ca-certificates/extracted/tls-ca-bundle.pem
lrwxrwxrwx 1 root root 49 Meh  18 19:36 /etc/ssl/certs/ca-bundle.crt -> ../../ca-certificates/extracted/tls-ca-bundle.pem
lrwxrwxrwx 1 root root 49 Meh  18 19:36 /etc/ssl/certs/ca-certificates.crt -> ../../ca-certificates/extracted/tls-ca-bundle.pem

So, if that's the correct file, I'm apparently doing something else wrong.

lmn wrote:

Using this in your code leads to a successful connection:
1.0
SSL connection: 0x59e6012692f8  nil
true    nil

Obviously I'm doing something stupid, but substituting `certs/ca-bundle.crt` for `cert.pem` (just in case the precise name matters, which seemed unlikely) does not produce `true` for me:

socket = require("socket")
ssl = require("ssl")
local params = {
  mode = "client",
  protocol = "any",    -- fails with "unsupported protocol" for any of tsl* versions ????
  -- cafile = "/etc/ca-certificates/extracted/tls-ca-bundle.pem",
  -- cafile = "/etc/ssl/cert.pem",
  -- https://bbs.archlinux.org/viewtopic.php?pid=2212901#p2212901
  cafile = "/etc/ssl/certs/ca-bundle.crt",
  verify = "peer",
  options = "all",
}
host = "www.google.com"
local conn = socket.tcp()
local res = conn:connect(host,443)
print(res)
conn, more = ssl.wrap(conn, params)
print(conn, more)
local res, more = conn:dohandshake()
print(res, more)
conn:close()

returns:

1.0
SSL connection: 0x63db85e3e168  nil
false   certificate verify failed

rather than the `true` you report. What am I missing?

lmn wrote:

The ca-bundle is essentially just a concatanation of some ca-root-certs. So you could find all relevant root certs beforehand and group them into a bundle yourself and use that. In the case of Google
cat /etc/ssl/certs/GTS_Root_R* > google.pem

I concatenated the certificates in `./gwgl.pem` and tried that:

socket = require("socket")
ssl = require("ssl")
local params = {
  mode = "client",
  protocol = "any",    -- fails with "unsupported protocol" for any of tsl* versions ????
  -- cafile = "/etc/ca-certificates/extracted/tls-ca-bundle.pem",
  -- cafile = "/etc/ssl/cert.pem",
  -- https://bbs.archlinux.org/viewtopic.php?pid=2212901#p2212901
  -- cafile = "/etc/ssl/certs/ca-bundle.crt",
  cafile = "./gwgl.pem",
  verify = "peer",
  options = "all",
}
host = "www.google.com"
local conn = socket.tcp()
local res = conn:connect(host,443)
print(res)
conn, more = ssl.wrap(conn, params)
print(conn, more)
local res, more = conn:dohandshake()
print(res, more)
conn:close()

but I'm still missing whatever basic thing is obvious to you ,

1.0
SSL connection: 0x608c73107158  nil
false   certificate verify failed

lmn wrote:

Also see https://man.archlinux.org/man/openssl.1ssl for more information.

Thanks. I'd already tried looking at `openssl`'s manual pages locally. Unfortunately, they presuppose a level of background knowledge I just don't have. I have figured out how to concatenate and manipulate certificates using those pages in the past. (Maybe not precisely that one.) But I am not sure that's the problem here. `gwgl.pem` doesn't look obviously foolish - it isn't empty, for example, and it has a similar format to a public gpg keys:

But I still get the verification failures shown above.

lmn · 2024-12-08 14:57:52

cfr wrote:

This is a symbolic link which points to the same file as the symbolic link specified in the code I posted above:

You are right. I did not check this.

My first concern was this line:

cfr wrote:

protocol = "any",    -- fails with "unsupported protocol" for any of tsl* versions ????

Which TLS version string did you supply? Maybe a typo tls <-> tsl ? The valid options can be found here.

I did change the TLS Version from any to TLS1_2 for testing and forgot about the change. Using TLS1_3 while connecting does fail the TLS verification.
My suspicion is that the any parameter tries the highest available TLS version first (you can see the implementation here with the relevant parameters found in the man page) and as TLS1_3 fails the connection fails. What I don't understand is why the next lower version (TLS1_2) is not tried next.

Here is my working code:

socket = require("socket")
ssl = require("ssl")

local params = {
        mode = "client",
        protocol = "tlsv1_2", -- "any" does not work as expected and "tlsv1_3" fails verification
        cafile = "/etc/ca-certificates/extracted/tls-ca-bundle.pem",
--        cafile = "google.pem",
        verify = "peer",
        options = "all",
}

host = "www.google.com"

local conn = socket.tcp()
local res = conn:connect(host,443)
print(res)

conn, more = ssl.wrap(conn, params)
print(conn, more)
-- conn:sni(host) -- setting the SNI manually works.

local res, more = conn:dohandshake()
print(res, more)

print("INFO", conn:info("protocol"))

s, errs = conn:getpeerverification()
print(s, errs)

conn:close()

Edit:
www.google.com does support TLS1_2 and TLS1_3 but not TLS1_1. You can check this with

openssl s_client -tls1_1 -connect "www.google.com:443" < /dev/null
openssl s_client -tls1_2 -connect "www.google.com:443" < /dev/null
openssl s_client -tls1_3 -connect "www.google.com:443" < /dev/null

Edit 2:
Setting the SNI seems to work with TLS1_3. An example can be found in the lua samples.

OpenSSL wiki wrote:

If the SNI extension is not sent the server's options are to either disconnect or select a default hostname and matching certificate.

www.google.com and archlinux.org seem to choose the disconnect option. debian.org on the other hand seems to set a default SNI.

Last edited by lmn (2024-12-08 15:25:13)

cfr · 2024-12-08 16:18:27

lmn wrote:

Which TLS version string did you supply? Maybe a typo tls <-> tsl ?

Thank you!

Dammit, how could I not see that?

Arch Linux

#1 2024-12-06 03:55:47

[solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

#2 2024-12-07 09:49:38

Re: [solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

#3 2024-12-08 07:15:16

Re: [solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

#4 2024-12-08 14:57:52

Re: [solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

#5 2024-12-08 16:18:27

Re: [solved] lua-sec: Verify ssl.wrap() connection w/ dohandshake()

Board footer