Pytest Cov Report Missing Mock Exception Returns

Question

I'm a Network Engineer who is trying to write some Python, so very much still learning each day. I have an issue with Unit Testing and Pytest coverage reports. I think I understand the concept of unit tests and pytest.

I have a function that does a DNS lookup using socket

def get_ip(d):
    """Returns the first IPv4 address that resolves from 'd'

    Args:
        d (string): string that needs to be resolved in DNS.

    Returns:
        string: IPv4 address if found
    """
    logger.info("Returns IP address from hostname: " + d)
    try:
        data = socket.gethostbyname(d)
        ip_addr = repr(data).strip("'")
        logger.debug("IP Address Resolved to: " + ip_addr)
        return ip_addr
    except socket.gaierror:
        return False

I have written a unit test which passes fine. I'm using pytest-mock to handle the mocking of the socket for the DNS lookup. Side Effect seems to be mocking the Exception and I've set a return_value as False and I assuming I have asserted that False is returned, the test passes ok which is why I'm assuming that my test is ok.

import pytest
import pytest_mock
import socket
from utils import network_utils

@pytest.fixture
def test_setup_network_utils():
    return network_utils

def test_get_ip__unhappy(mocker, test_setup_network_utils):
    mocker.patch.object(network_utils, 'socket')
    test_setup_network_utils.socket.gethostbyname.side_effect = socket.gaierror
    with pytest.raises(Exception):
        d = 'xxxxxx'
        test_setup_network_utils.socket.gethostbyname.return_value = False
        test_setup_network_utils.get_ip(d)
    assert not test_setup_network_utils.socket.gethostbyname.return_value
    test_setup_network_utils.socket.gethostbyname.assert_called_once()
    test_setup_network_utils.socket.gethostbyname.assert_called_with(d)

The pytest-cov report shows the return False line as not being covered.

pytest --cov-report term-missing:skip-covered --cov=utils unit_tests

network_utils.py     126      7    94%   69

Line 69 is the below line of code in function

except socket.gaierror:
    return False <<<<<<<< This is missing from the report

Any pointers would be appreciated as to why the return of False isn't being declared as covered. Like I said above I'm pretty new to Python and coding and this is my 1st question on Stackoverflow. Hopefully I've explained what my issue and have provided enough information for some guidance.

Answer 1

When you declare

mocker.patch.object(network_utils, 'socket')

you are replacing the whole socket module with a single mock, so everything in the socket module becomes a mock too, including socket.gaierror. Thus, when trying to catch socket.gaierror while running the test, Python will complain that it is not an exception (does not subclass from BaseException) and fail. Therefore, the desired return line is not executed.

Overall, your test looks overengineered and contains a lot of unnecessary code. What do you need the test_setup_network_utils for? Why are you catching an arbitrary exception in test? A revisited test_get_ip__unhappy that covers the complete code in the gaierror case:

def test_get_ip__unhappy(mocker):
    # test setup: patch socket.gethostbyname so it always raises
    mocker.patch('spam.socket.gethostbyname')
    spam.socket.gethostbyname.side_effect = socket.gaierror

    # run test
    d = 'xxxxxx'
    result = spam.get_ip(d)

    # perform checks
    assert result is False
    spam.socket.gethostbyname.assert_called_once()
    spam.socket.gethostbyname.assert_called_with(d)

Of course, spam is just an example; replace it with your actual import.