How to properly write cross-references to external documentation with intersphinx?

Question

I'm trying to add cross-references to external API into my documentation but I'm facing three different behaviors.

I am using sphinx(1.3.1) with Python(2.7.3) and my intersphinx mapping is configured as:

{
'python': ('https://docs.python.org/2.7', None),
'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'cv2' : ('http://docs.opencv.org/2.4/', None),
'h5py' : ('http://docs.h5py.org/en/latest/', None)
}

I have no trouble writing a cross-reference to numpy API with :class:`numpy.ndarray` or :func:`numpy.array` which gives me, as expected, something like numpy.ndarray.

However, with h5py, the only way I can have a link generated is if I omit the module name. For example, :class:`Group` (or :class:`h5py:Group`) gives me Group but :class:`h5py.Group` fails to generate a link.

Finally, I cannot find a way to write a working cross-reference to OpenCV API, none of these seems to work:

:func:`cv2.convertScaleAbs`
:func:`cv2:cv2.convertScaleAbs`
:func:`cv2:convertScaleAbs`
:func:`convertScaleAbs`

How to properly write cross-references to external API, or configure intersphinx, to have a generated link as in the numpy case?

Answer 1

I gave another try on trying to understand the content of an objects.inv file and hopefully this time I inspected numpy and h5py instead of only OpenCV's one.

How to read an intersphinx inventory file

Despite the fact that I couldn't find anything useful about reading the content of an object.inv file, it is actually very simple with the intersphinx module.

from sphinx.ext import intersphinx
import warnings


def fetch_inventory(uri):
    """Read a Sphinx inventory file into a dictionary."""
    class MockConfig(object):
        intersphinx_timeout = None  # type: int
        tls_verify = False

    class MockApp(object):
        srcdir = ''
        config = MockConfig()

        def warn(self, msg):
            warnings.warn(msg)

    return intersphinx.fetch_inventory(MockApp(), '', uri)


uri = 'http://docs.python.org/2.7/objects.inv'

# Read inventory into a dictionary
inv = fetch_inventory(uri)
# Or just print it
intersphinx.debug(['', uri])

File structure (numpy)

After inspecting numpy's one, you can see that keys are domains:

[u'np-c:function',
 u'std:label',
 u'c:member',
 u'np:classmethod',
 u'np:data',
 u'py:class',
 u'np-c:member',
 u'c:var',
 u'np:class',
 u'np:function',
 u'py:module',
 u'np-c:macro',
 u'np:exception',
 u'py:method',
 u'np:method',
 u'np-c:var',
 u'py:exception',
 u'np:staticmethod',
 u'py:staticmethod',
 u'c:type',
 u'np-c:type',
 u'c:macro',
 u'c:function',
 u'np:module',
 u'py:data',
 u'np:attribute',
 u'std:term',
 u'py:function',
 u'py:classmethod',
 u'py:attribute']

You can see how you can write your cross-reference when you look at the content of a specific domain. For example, py:class:

{u'numpy.DataSource': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.DataSource.html#numpy.DataSource',
  u'-'),
 u'numpy.MachAr': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.MachAr.html#numpy.MachAr',
  u'-'),
 u'numpy.broadcast': (u'NumPy',
  u'1.9',
  u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html#numpy.broadcast',
  u'-'),
  ...}

So here, :class:`numpy.DataSource` will work as expected.

h5py

In the case of h5py, the domains are:

[u'py:attribute', u'std:label', u'py:method', u'py:function', u'py:class']

and if you look at the py:class domain:

{u'AttributeManager': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/attr.html#AttributeManager',
  u'-'),
 u'Dataset': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/dataset.html#Dataset',
  u'-'),
 u'ExternalLink': (u'h5py',
  u'2.5',
  u'http://docs.h5py.org/en/latest/high/group.html#ExternalLink',
  u'-'),
 ...}

That's why I couldn't make it work as numpy references. So a good way to format them would be :class:`h5py:Dataset`.

OpenCV

OpenCV's inventory object seems malformed. Where I would expect to find domains there is actually 902 function signatures:

[u':',
 u'AdjusterAdapter::create(const',
 u'AdjusterAdapter::good()',
 u'AdjusterAdapter::tooFew(int',
 u'AdjusterAdapter::tooMany(int',
 u'Algorithm::create(const',
 u'Algorithm::getList(vector<string>&',
 u'Algorithm::name()',
 u'Algorithm::read(const',
 u'Algorithm::set(const'
 ...]

and if we take the first one's value:

{u'Ptr<AdjusterAdapter>': (u'OpenCV',
  u'2.4',
  u'http://docs.opencv.org/2.4/detectorType)',
  u'ocv:function 1 modules/features2d/doc/common_interfaces_of_feature_detectors.html#$ -')}

I'm pretty sure it is then impossible to write OpenCV cross-references with this file...

Conclusion

I thought intersphinx generated the objects.inv based on the content of the documentation project in an standard way, which seems not to be the case. As a result, it seems that the proper way to write cross-references is API dependent and one should inspect a specific inventory object to actually see what's available.

Answer 2

For convenience, I made a small extension for aliasing intersphinx cross references. This is useful as sometimes the object inventory gets confused when an object from a submodule is imported from a package's __init__.py.

See also https://github.com/sphinx-doc/sphinx/issues/5603

###
# Workaround of
# Intersphinx references to objects imported at package level can"t be mapped.
#
# See https://github.com/sphinx-doc/sphinx/issues/5603

intersphinx_aliases = {
    ("py:class", "click.core.Group"):
        ("py:class", "click.Group"),
    ("py:class", "click.core.Command"):
        ("py:class", "click.Command"),
}


def add_intersphinx_aliases_to_inv(app):
    from sphinx.ext.intersphinx import InventoryAdapter
    inventories = InventoryAdapter(app.builder.env)

    for alias, target in app.config.intersphinx_aliases.items():
        alias_domain, alias_name = alias
        target_domain, target_name = target
        try:
            found = inventories.main_inventory[target_domain][target_name]
            try:
                inventories.main_inventory[alias_domain][alias_name] = found
            except KeyError:
                print("could not add to inv")
                continue
        except KeyError:
            print("missed :(")
            continue


def setup(app):
    app.add_config_value("intersphinx_aliases", {}, "env")
    app.connect("builder-inited", add_intersphinx_aliases_to_inv)

To use this, I paste the above code in my conf.py and add aliases to the intersphinx_aliases dictionary.

Answer 3

How to use OpenCV 2.4 (cv2) intersphinx

Inspired by @Gall's answer, I wanted to compare the contents of the OpenCV & numpy inventory files. I couldn't get sphinx.ext.intersphinx.fetch_inventory to work from ipython, but the following does work:

curl http://docs.opencv.org/2.4/objects.inv | tail -n +5 | zlib-flate -uncompress > cv2.inv
curl https://docs.scipy.org/doc/numpy/objects.inv | tail -n +5 | zlib-flate -uncompress > numpy.inv

numpy.inv has lines like this:

numpy.ndarray py:class 1 reference/generated/numpy.ndarray.html#$ -

whereas cv2.inv has lines like this:

cv2.imread ocv:pyfunction 1 modules/highgui/doc/reading_and_writing_images_and_video.html#$ -

So presumably you'd link to the OpenCV docs with :ocv:pyfunction:`cv2.imread` instead of :py:function:`cv2.imread`. Sphinx doesn't like it though:

WARNING: Unknown interpreted text role "ocv:pyfunction".

A bit of Googling revealed that the OpenCV project has its own "ocv" sphinx domain: https://github.com/opencv/opencv/blob/2.4/doc/ocv.py -- presumably because they need to document C, C++ and Python APIs all at the same time.

To use it, save ocv.py next to your Sphinx conf.py, and modify your conf.py:

sys.path.insert(0, os.path.abspath('.'))
import ocv
extensions = [
    'ocv',
]
intersphinx_mapping = {
    'cv2': ('http://docs.opencv.org/2.4/', None),
}

In your rst files you need to say :ocv:pyfunc:`cv2.imread` (not :ocv:pyfunction:).

Sphinx prints some warnings (unparseable C++ definition: u'cv2.imread') but the generated html documentation actually looks ok with a link to http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#cv2.imread. You can edit ocv.py and remove the line that prints that warning.

Answer 4

Stubborn fool that I am, I used 2to3 and the Sphinx deprecated APIs chart to revive @david-röthlisberger's ocv.py-based answer so it'll work with Sphinx 2.3 on Python 3.5.

The fixed-up version is here:

https://gist.github.com/ssokolow/a230b27b7ea4a31f7fb40621e6461f9a

...and the quick version of what I did was:

1. Run 2to3 -w ocv.py && rm ocv.py.bak
2. Cycle back and forth between running Sphinx and renaming functions to their replacements in the chart. I believe these were the only changes I had to make on this step:
   1. Directive now has to be imported from docutils.parsers.rst
   2. Replace calls to l_(...) with calls to _(...) and remove the l_ import.
3. Replace calls to env.warn with calls to log.warn where log = sphinx.util.logging.getLogger(__name__).

Then, you just pair it with this intersphinx definition and you get something still new enough to be relevant for most use cases:

'cv2': ('https://docs.opencv.org/3.0-last-rst/', None)

Answer 5

In addition to the detailed answer from @gall, I've discovered that intersphinx can also be run as a module:

python -m sphinx.ext.intersphinx 'http://python-eve.org/objects.inv'

This outputs nicely formatted info. For reference: https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/intersphinx.py#L390

Answer 6

The accepted answer no longer works in the new version (1.5.x) ...

import requests
import posixpath
from sphinx.ext.intersphinx import read_inventory

uri = 'http://docs.python.org/2.7/'

r = requests.get(uri + 'objects.inv', stream=True)
r.raise_for_status()

inv = read_inventory(r.raw, uri, posixpath.join)

Answer 7

An additional way to inspect the objects.inv file is with the sphobjinv module.

You can search local or even remote inventory files (with fuzzy matching). For instance with scipy:

$ sphobjinv suggest -t 90 -u https://docs.scipy.org/doc/scipy/reference/objects.inv "signal.convolve2d"

Remote inventory found.

:py:function:`scipy.signal.convolve2d`
:std:doc:`generated/scipy.signal.convolve2d`

Note that you may need to use :py:func: and not :py:function: (I'd be happy to know why).