mTLS not working with FastAPI and Uvicorn

Question

I'm completely new to ssl, so sorry if this is a super obvious issue. My project requires mTLS authentication, and I have to admit, I'm really confused! (this is about hour 10 into this rabbit hole). I am using a simple FastAPI application with Uvicorn

I had setup https with Let's Encrypt with no issues, just added the following to my uvicorn command:

ssl_keyfile="mtls/letsencrypt/privkey.pem", and ssl_certfile="mtls/letsencrypt/fullchain.pem"

I was pointed to this tutorial: https://medium.com/@rob.blackbourn/how-to-use-cfssl-to-create-self-signed-certificates-d55f76ba5781 to create some self-signed certificates,

and this tutorial to enable mTLS: https://ahaw021.medium.com/mutual-tls-mtls-with-fastapi-and-uvicorn-3b9e91bdf5a6

However the waters are still very murky. I used

cfssl gencert -ca intermediate_ca.pem -ca-key intermediate_ca-key.pem -config cfssl.json -profile=client host.json | cfssljson -bare client

to create client.pem,client-key.pem and client.csr. Then I used the following command to create client.crt from my client.pem file:

openssl x509 -outform der -in mtls/cert/client.pem -out mtls/cert/client.crt

and installed it from chrome's settings. However when I run the uvicorn command and go to my domain, chrome doesn't show client.crt, and obviously when I click cancel, it gives me ERR_EMPTY_RESPONSE.

I tried calling an endpoint with Postman, so I selected client.crt for the crt file and client-key.pem for the key file under certificates, and enabled SSL under general settings and the settings tab for my endpoint. I either get one of the two errors, depending on my troubleshooting methods:

Error: error:0900006e:PEM routines:OPENSSL_internal:NO_START_LINE

Error: socket hang up I am at a complete lost on what to do, I have obviously gone wrong at some point, but no idea where. I had tested the app before trying mTLS, and HTTPS worked fine. My distilled code is below:

#main.py
from fastapi import FastAPI
import uvicorn
import ssl

from routers import router1,router2

app = FastAPI(root_path="/api/v1",)
app.include_router(router1.router)
app.include_router(router2.router)

@app.post("/auth")
async def auth() -> Token:
    #oauth2 authentication
    pass

@app.get("/a")
async def endpoint_a():
    return {"a":"b"}

if __name__ == "__main__":
    uvicorn.run("main:app", 
                host="0.0.0.0", 
                port=8000,
                ssl_keyfile="mtls/letsencrypt/privkey.pem", # Provided by Let's Encrypt
                ssl_certfile="mtls/letsencrypt/fullchain.pem",

                ssl_ca_certs="mtls/certs/fullchain.pem", # combined ca.pem and intermediate_ca.pem
                ssl_cert_reqs=ssl.CERT_REQUIRED
                )

config files used by cfssl:

#cfssl
{
    "signing": {
      "default": {
        "expiry": "8760h"
      },
      "profiles": {
        "intermediate_ca": {
          "usages": [
              "signing",
              "digital signature",
              "key encipherment",
              "cert sign",
              "crl sign",
              "server auth",
              "client auth"
          ],
          "expiry": "8760h",
          "ca_constraint": {
              "is_ca": true,
              "max_path_len": 0, 
              "max_path_len_zero": true
          }
        },
        "peer": {
          "usages": [
              "signing",
              "digital signature",
              "key encipherment", 
              "client auth",
              "server auth"
          ],
          "expiry": "8760h"
        },
        "server": {
          "usages": [
            "signing",
            "digital signing",
            "key encipherment",
            "server auth"
          ],
          "expiry": "8760h"
        },
        "client": {
          "usages": [
            "signing",
            "digital signature",
            "key encipherment", 
            "client auth"
          ],
          "expiry": "8760h"
        }
      }
    }
  }

#ca.json
{
    "CN": "Joseph0M API CA",
    "key": {
      "algo": "rsa",
      "size": 2048
    },
    "CA": {
        "expiry": "8766h",
        "pathlen": 2
    },
    "names": [
    {
      "C": "GB",
      "L": "London",
      "O": "Joseph0M",
      "E": "email",
      "ST": "England"
    }
   ]
  }

#intermediate_ca.json
{
    "CN": "Joseph0M API Intermediate CA",
    "key": {
      "algo": "rsa",
      "size": 2048
    },
    "names": [
      {
        "C":  "GB",
        "L":  "London",
        "O":  "Joseph0M",
        "E": "email",
        "ST": "England"
      }
    ],
    "ca": {
      "expiry": "42720h"
    }
  }
  
#host.json
{
    "CN": "my domain",
    "key": {
      "algo": "rsa",
      "size": 2048
    },
    "names": [
    {
      "C": "GB",
      "L": "London",
      "O": "Joseph0M",
      "E": "email",
      "ST": "England"
    }
    ],
    "hosts": [
      "domain name",
      "localhost",
      "ip address of server"
    ]
  }

Any help would be greatly appreciated, thank you!

Answer 1

So, I managed to get it working finally, for some reason the client certs that were signed by the intermediate ca just didn't work, so I signed them with the root, and added the ca.pem to ssl_ca_certs. Thanks Maarten for your help, its really appreciated! For those that might be in a similar situation, the following was quite the help too: https://mtls.dev/