X Tutup
Skip to content

New tests without baseline images never produce a result #7240

@QuLogic

Description

@QuLogic

When adding a new test, you don't have a baseline image, and probably expect to copy this out of the result set. On master, the result is never created if the baseline image doesn't exist. It appears that copy_baseline will raise an expected failure before the result is saved.

Relatedly, pytest does not have an easy way of seeing this message that the baseline image is missing. Using pytest -v, it just marks the test as xfail instead of x. With the old tests.py -v, it will print out a much more useful KNOWNFAIL: Do not have baseline image .../result_images/test_mplot3d/proj3d_lines_dists-expected.png because this file does not exist: .../mpl_toolkits/tests/baseline_images/test_mplot3d/proj3d_lines_dists.png.

From what I've seen, a missing baseline image has been marked as xfail since before pytest, but I'm not sure that makes sense; I think it should be an error.

I think this is related to some pytest re-factorings; cc @Kojoley.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    X Tutup