Right now, I have a Python package (let's call it
mypackage) with a bunch of tests that I run with pytest. One particular feature can have many possible implementations, so I have used the funcarg mechanism to run these tests with a reference implementation.
# In mypackage/tests/conftest.py def pytest_funcarg__Feature(request): return mypackage.ReferenceImplementation # In mypackage/tests/test_stuff.py def test_something(Feature): assert Feature(1).works
Now, I am creating a separate Python package with a fancier implementation (
fancypackage). <strong>Is it possible to run all of the tests in
mypackage that contain the
Feature funcarg, only with different implementations?</strong>
I would like to avoid having to change
fancypackage if I add new tests in
mypackage, so explicit imports aren't ideal. I know that I can run all of the tests with
pytest.main(), but since I have several implementations of my feature, I don't want to call
pytest.main() multiple times. Ideally, it would look like something like this:
# In fancypackage/tests/test_impl1.py def pytest_funcarg__Feature(request): return fancypackage.Implementation1 ## XXX: Do pytest collection on mypackage.tests, but don't run them # In fancypackage/tests/test_impl2.py def pytest_funcarg__Feature(request): return fancypackage.Implementation2 ## XXX: Do pytest collection on mypackage.tests, but don't run them
Then, when I run pytest in
fancypackage, it would collect each of the
mypackage.tests tests twice, once for each feature implementation. I have tried doing this with explicit imports, and it seems to work fine, but I don't want to explicitly import everything.
An additional nice bonus would be to only collect those tests that contain the
Feature funcarg. Is that possible?
Before switching to py.test, I did this with the standard library's
unittest. The function for that is the following:
def mypackage_test_suite(Feature): loader = unittest.TestLoader() suite = unittest.TestSuite() mypackage_tests = loader.discover('mypackage.tests') for test in all_testcases(mypackage_tests): if hasattr(test, 'Feature'): test.Feature = Feature suite.addTest(test) return suite def all_testcases(test_suite_or_case): try: suite = iter(test_suite_or_case) except TypeError: yield test_suite_or_case else: for test in suite: for subtest in all_testcases(test): yield subtest
Obviously things are different now because we're dealing with test functions and classes instead of just classes, but it seems like there should be some equivalent in py.test that builds the test suite and allows you to iterate through it.Answer1:
You could parameterise your
@pytest.fixture(params=['ref', 'fancy']) def Feature(request): if request.param == 'ref': return mypackage.ReferenceImplementation else: return fancypackage.Implementation1
Now if you run py.test it will test both.
Selecting tests on the fixture they use is not possible AFAIK, you could probably cobble something together using