-
-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Description
When adding a new test, you don't have a baseline image, and probably expect to copy this out of the result set. On master, the result is never created if the baseline image doesn't exist. It appears that copy_baseline will raise an expected failure before the result is saved.
Relatedly, pytest does not have an easy way of seeing this message that the baseline image is missing. Using pytest -v, it just marks the test as xfail instead of x. With the old tests.py -v, it will print out a much more useful KNOWNFAIL: Do not have baseline image .../result_images/test_mplot3d/proj3d_lines_dists-expected.png because this file does not exist: .../mpl_toolkits/tests/baseline_images/test_mplot3d/proj3d_lines_dists.png.
From what I've seen, a missing baseline image has been marked as xfail since before pytest, but I'm not sure that makes sense; I think it should be an error.
I think this is related to some pytest re-factorings; cc @Kojoley.