sphinx-build with -blinkcheck and custom CA

Question

We have our own company wide certificate authority which we use to signa SSL certificates. Mostly, this is working fine as long as you have your OS (CentOS 7 in our case) register that authority. It is stored here:

/etc/pki/ca-trust/source/anchors/company_ca.pem

This allows Firefox/chrome to trust the SSL certificates that were signed via it.

I am using sphinx-build -W -blinkcheck […] to check that the links in my Python project are still valid as link rot sucks in documentation. This is fine for all external links.

However, when linking to our own SSL version of mantis (a bug tracker), I get a

SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)'),)))

error. Mantis, in our set up, only runs on https.

How do I tell sphinx to add the company-wide authority?

I run this generally via tox like thus:

The tox fragement which runs this:

[testenv:docs]
basepython=python2.7
deps=-r{toxinidir}/requirements/requirements.txt
commands=./check_docs.bash

The bash script:

#!/bin/bash
set -eux
sphinx-apidoc --force --separate --private --module-first -o docs src/ '*/*test*'
cd docs
pytest --maxfail=1 \
    --tb=line \
    -v \
    --junitxml=junit_sphinx.xml \
    --exitfirst \
    --failed-first \
    --full-trace \
    -ra \
    --capture=no \
    check_sphinx.py

And the pythons script:

import subprocess


def test_linkcheck(tmpdir):
    doctrees = tmpdir.join("doctrees")
    htmldir = tmpdir.join("html")
    subprocess.check_call([
        "sphinx-build", "-W", "-blinkcheck", "-d",
        str(doctrees), ".",
        str(htmldir)
    ])


def test_build_docs(tmpdir):
    doctrees = tmpdir.join("doctrees")
    htmldir = tmpdir.join("html")
    subprocess.check_call([
        "sphinx-build", "-W", "-bhtml", "-d",
        str(doctrees), ".",
        str(htmldir)
    ])

Answer 1

Sphinx uses requests which uses certifi -- thanks to sraw who kindly pointed this out in a comment. You can modify the certifi.where() to include your own certificate authority.

Because you might run tox or re-build your virtual environement, doing so manually is tedious and error prone. A fixture makes this much easier to deal with.

The Python script changes to the following.

# -*- coding: utf-8 -*-
import subprocess
import certifi
import requests
import pytest

CA = '/etc/pki/ca-trust/source/anchors/company_ca.pem'


@pytest.fixture
def certificate_authority(scope="module"):
    try:
        # Checking connection to Mantis…
        requests.get('https://mantisbt.example.com')
        # Connection to Mantis OK, thus CA should work fine.
    except requests.exceptions.SSLError:
        # SSL Error. Adding custom certs to Certifi store…
        cafile = certifi.where()
        with open(CA, 'rb') as infile:
            customca = infile.read()
        with open(cafile, 'ab') as outfile:
            outfile.write(customca)
        # That might have worked.


def test_linkcheck(certificate_authority, tmpdir):
    doctrees = tmpdir.join("doctrees")
    htmldir = tmpdir.join("html")
    subprocess.check_call([
        "sphinx-build", "-W", "-blinkcheck", "-d",
        str(doctrees), ".",
        str(htmldir)
    ])


def test_build_docs(certificate_authority, tmpdir):
    doctrees = tmpdir.join("doctrees")
    htmldir = tmpdir.join("html")
    subprocess.check_call([
        "sphinx-build", "-W", "-bhtml", "-d",
        str(doctrees), ".",
        str(htmldir)
    ])