Pytest run only tests with specific marker attribute

Question

I'm using @pytest.mark to uniquely identify specific tests, therefore I created my custom marker.

@pytest.mark.key

I'm using it as such:

@pytest.mark.key("test-001")
def test_simple(self):
    self.passing_step()
    self.passing_step()
    self.passing_step()
    self.passing_step()
    assert True

Now from the console I would like to run all tests with the marked key "test-001". How can I achieve this?

What i'm looking for is something like this:

pypi.org/project/pytest-jira/0.3.6

where a test can be mapped to a Jira key. I looked at the source code for the link but i'm unsure how to achieve it in order for me to run specific tests. Say I only wanna run the test with the key "test-001".

Answer 1

Pytest does not provide this out of the box. You can filter by marker names using the -m option, but not by marker attributes.
You can add your own option to filter by keys, however. Here is an example:

conftest.py

def pytest_configure(config):
    # register your new marker to avoid warnings
    config.addinivalue_line(
        "markers",
        "key: specify a test key"
    )


def pytest_addoption(parser):
    # add your new filter option (you can name it whatever you want)
    parser.addoption('--key', action='store')


def pytest_collection_modifyitems(config, items):
    # check if you got an option like --key=test-001
    filter = config.getoption("--key")
    if filter:
        new_items = []
        for item in items:
            mark = item.get_closest_marker("key")
            if mark and mark.args and mark.args[0] == filter:
                # collect all items that have a key marker with that value
                new_items.append(item)
        items[:] = new_items

Now you run something like

pytest --key=test-001

to only run the tests with that marker attribute.

Note that this will still show the overall number of tests as collected, but run only the filtered ones. Here is an example:

test_key.py

import pytest

@pytest.mark.key("test-001")
def test_simple1():
    pass

@pytest.mark.key("test-002")
def test_simple2():
    pass


@pytest.mark.key("test-001")
def test_simple3():
    pass

def test_simple4():
    pass

$ python -m pytest -v --key=test-001 test_key.py

...
collected 4 items

test_key.py::test_simple1 PASSED
test_key.py::test_simple3 PASSED
================================================== 2 passed in 0.26s ==================================================

Answer 2

you can run pytest with -m option c

check below command:

pytest -m 'test-001' <your test file>

Answer 3

You can create custom markers. *You can see Working with custom markers.

First, register orange, apple and pineapple markers to [pytest] in pytest.ini as shown below. *You can still use custom markers without registration but there are warnings so registration is needed:

# "pytest.ini"

[pytest]
markers =
    orange: Orange marker
    apple: Apple marker
    pineapple: Pineapple marker

Then, use them as shown below:

import pytest

@pytest.mark.orange
def test1():
    assert True

@pytest.mark.apple
def test2():
    assert True

@pytest.mark.pineapple
def test3():
    assert True

Then, if you run pytest with -k apple as shown below:

pytest -k apple

Then, apple and pineapple are run as shown below. *-k can run the markers which contain the given string:

import pytest

@pytest.mark.orange # Not run
def test1():
    assert True

@pytest.mark.apple # Run
def test2():
    assert True

@pytest.mark.pineapple # Run
def test3():
    assert True

And, if you run pytest with -m apple as shown below:

pytest -m apple

Then, only apple is run as shown below. *-m can run the markers which is exactly same as the given string:

import pytest

@pytest.mark.orange # Not run
def test1():
    assert True

@pytest.mark.apple # Run
def test2():
    assert True

@pytest.mark.pineapple # Not run
def test3():
    assert True

Lastly, you can check the markers with --markers as shown below:

$ pytest --markers
@pytest.mark.orange: Orange marker

@pytest.mark.apple: Apple marker

@pytest.mark.pineapple: Pineapple marker
...