56358

Running pytest tests in another Python package

Question:

Right now, I have a Python package (let's call it mypackage) with a bunch of tests that I run with pytest. One particular feature can have many possible implementations, so I have used the funcarg mechanism to run these tests with a reference implementation.

# In mypackage/tests/conftest.py def pytest_funcarg__Feature(request): return mypackage.ReferenceImplementation # In mypackage/tests/test_stuff.py def test_something(Feature): assert Feature(1).works

Now, I am creating a separate Python package with a fancier implementation (fancypackage). <strong>Is it possible to run all of the tests in mypackage that contain the Feature funcarg, only with different implementations?</strong>

I would like to avoid having to change fancypackage if I add new tests in mypackage, so explicit imports aren't ideal. I know that I can run all of the tests with pytest.main(), but since I have several implementations of my feature, I don't want to call pytest.main() multiple times. Ideally, it would look like something like this:

# In fancypackage/tests/test_impl1.py def pytest_funcarg__Feature(request): return fancypackage.Implementation1 ## XXX: Do pytest collection on mypackage.tests, but don't run them # In fancypackage/tests/test_impl2.py def pytest_funcarg__Feature(request): return fancypackage.Implementation2 ## XXX: Do pytest collection on mypackage.tests, but don't run them

Then, when I run pytest in fancypackage, it would collect each of the mypackage.tests tests twice, once for each feature implementation. I have tried doing this with explicit imports, and it seems to work fine, but I don't want to explicitly import everything.

<h3>Bonus</h3>

An additional nice bonus would be to only collect those tests that contain the Feature funcarg. Is that possible?

<h2>Example with unittest</h2>

Before switching to py.test, I did this with the standard library's unittest. The function for that is the following:

def mypackage_test_suite(Feature): loader = unittest.TestLoader() suite = unittest.TestSuite() mypackage_tests = loader.discover('mypackage.tests') for test in all_testcases(mypackage_tests): if hasattr(test, 'Feature'): test.Feature = Feature suite.addTest(test) return suite def all_testcases(test_suite_or_case): try: suite = iter(test_suite_or_case) except TypeError: yield test_suite_or_case else: for test in suite: for subtest in all_testcases(test): yield subtest

Obviously things are different now because we're dealing with test functions and classes instead of just classes, but it seems like there should be some equivalent in py.test that builds the test suite and allows you to iterate through it.

Answer1:

You could parameterise your Feature fixture:

@pytest.fixture(params=['ref', 'fancy']) def Feature(request): if request.param == 'ref': return mypackage.ReferenceImplementation else: return fancypackage.Implementation1

Now if you run py.test it will test both.

Selecting tests on the fixture they use is not possible AFAIK, you could probably cobble something together using request.applymarker() and -m. however.

Recommend

  • Custom delete method on queryset
  • Elasticsearch match list against field
  • How do I make MFC checkbox read-only but keep text enabled?
  • The android java class cannot recognize the R file
  • WPF DataGrid lost focus after row delete
  • How do I control the soft menu button in Honeycomb?
  • Hudson dependencies
  • how to avoid the dependencies hell with unit test in angular 2+
  • Perl function name clash
  • JSF NumberFormatException with f:setPropertyActionListener
  • MySQL performance when updating row with FK
  • How to explicitly/implicitly implemented interface members in C++/CLI?
  • uniform generation of points on 3D box
  • Yii2: Config params vs. const/define
  • Python CGI os.system causing malformed header
  • Repeat a vertical line on every page in Report Builder / SSRS
  • Sails.js/waterline: Executing waterline queries in toJSON function of a model?
  • Android screen density dpi vs ppi
  • NetLogo BehaviorSpace - Measure runs using reporters
  • Fetching methods from BroadcastReceiver to update UI
  • Bug in WPF DataGrid
  • JavaScriptCore crash on iOS9
  • recyclerView does not call the onBindViewHolder when scroll in the view
  • output of program is not same as passed argument
  • Does CUDA 5 support STL or THRUST inside the device code?
  • WinForms: two way TextBox problem
  • Akka Routing: Reply's send to router ends up as dead letters
  • Calling of Constructors in a Java
  • Traverse Array and Display in markup
  • Transpose CSV data with awk (pivot transformation)
  • Proper folder structure for lots of source files
  • python regex in pyparsing
  • Why can't I rebase on to an ancestor of source changesets if on a different branch?
  • Angular 2 constructor injection vs direct access
  • Java static initializers and reflection
  • Android Google Maps API OnLocationChanged only called once
  • unknown Exception android
  • Checking variable from a different class in C#
  • UserPrincipal.Current returns apppool on IIS
  • java string with new operator and a literal