Output log calls captured in 'pytest' for successful tests

1k views Asked by At

I have some pytest tests that trigger a request to an AWS Lambda function. So that I can inspect the test logs as necessary, I wish to output the request ID of each invocation.

I would like the output to be formatted in a similar way to captured logs, for example -

----------------------------------- Captured log call -----------------------------------
INFO     root:test_lambda OK 62bcb138-9f35-4f04-9768-d512769a3dcc
INFO     root:test_lambda InvalidOrgIdException 428360e7-568c-45e2-833e-6eaa55d97914

In the first instance I was using the caplog fixture, thinking that it was everything that I needed.

@pytest.mark.parametrize(argnames, argvalues, ids=ids, indirect=indirect)
def test(self, event, expected, log_file) -> None:
    response = self.lambda_client.invoke(
        FunctionName=self.function_name,
        Payload=event
    )

    name = expected['errorType'] if 'errorType' in expected else 'OK'
    request_id = response['ResponseMetadata']['HTTPHeaders']['x-amzn-requestid']
    logging.info(f'{name} {request_id}')

    payload = json.loads(response['Payload'].read())
    body = payload['body']

    assert json.loads(body) == expected

However, when using the caplog fixture, log calls are only emitted on failure, not success. Apparently this is a feature, but I find it unfathomable that there isn't (at least not that I can find) an option to emit logs on success.

I next attempted to create a session fixture that could read from caplog on teardown and output my results, but this was a non-starter do to a scope mismatch.

ScopeMismatch: You tried to access the function scoped fixture caplog with a session scoped request object

The only working solution I have at the moment is to log request ID's to a file, and then to read them in the teardown_class method. However this way of doing things seems inelegant as it relies on the -s CLI option, and I suspect there is a better way to do it.

python -B -m pytest tests/integration -s --log-file=request_id.log

How can I output captured log calls on success and display them in a similar manner to how they would be displayed in the event of a failure?


Working but inelegant solution

class TestApplicationLoadBalancer:

    # Removed for brevity

    def teardown_class(self):
        log_file = logging.getLoggerClass().root.handlers[1].baseFilename
        with open(log_file, 'r') as f:
            print(f.read())


    @pytest.mark.parametrize(argnames, argvalues, ids=ids, indirect=indirect)
    def test(self, event, expected, log_file) -> None:
        response = self.lambda_client.invoke(
            FunctionName=self.function_name,
            Payload=event
        )

        name = expected_response.get('errorType', 'OK')
        request_id = response.get('ResponseMetadata', {}) \
            .get('HTTPHeaders', {}) \
            .get('x-amzn-requestid', None)
        logging.info(f'{name: <25}\t{request_id}')

        payload = json.loads(expected['Payload'].read())
        body = payload['body']

        assert json.loads(body) == expected
1

There are 1 answers

0
Peter Dowdy On

I'm struggling with this too - could you use the --capture argument to capture writes to stdout / stderr / certain file handles? I'm also trying to capture a detailed output of what the actual assertions were and attach them to the test as evidence, and pytest makes this frustratingly hard. However, anything that is covered by --capture goes into the tests[].call object in the report json, whether or fails or passes, so it might give you what you want.