X Tutup
Skip to content

Serialize comparison of multiple baseline images.#7674

Merged
tacaswell merged 1 commit intomatplotlib:masterfrom
QuLogic:serialize-baseline-compare
Dec 23, 2016
Merged

Serialize comparison of multiple baseline images.#7674
tacaswell merged 1 commit intomatplotlib:masterfrom
QuLogic:serialize-baseline-compare

Conversation

@QuLogic
Copy link
Copy Markdown
Member

@QuLogic QuLogic commented Dec 23, 2016

This prevents the results from stomping on one another, though it doesn't fix the case of multiple tests using the same baseline image.

See #7673 for an explanation. Since there are so many tests with multiple copies of the same baseline (it's not a huge amount, but enough, apparently), this seems like a reasonable workaround.

@QuLogic
Serialize comparison of multiple baseline images.
dfe0f63 
This prevents the results from stomping on one another, though it
doesn't fix the case of multiple tests using the same baseline image.

See matplotlib#7673 for an explanation.
@QuLogic QuLogic added this to the 2.1 (next point release) milestone Dec 23, 2016
@QuLogic
Copy link
Copy Markdown
Member Author

QuLogic commented Dec 23, 2016

And it died on something else; not sure if that's proof it's working or just bad luck.

@tacaswell
Copy link
Copy Markdown
Member

tacaswell commented Dec 23, 2016 

=================================== FAILURES ===================================
____________________ test_mixedsubplots[0-mixedsubplot-svg] ____________________
[gw0] linux -- Python 3.5.2 /home/travis/build/matplotlib/matplotlib/venv/bin/python
expected = '/home/travis/build/matplotlib/matplotlib/result_images/test_mplot3d/mixedsubplot-expected.svg'
actual = '/home/travis/build/matplotlib/matplotlib/result_images/test_mplot3d/mixedsubplot.svg'
tol = 0
    def raise_on_image_difference(expected, actual, tol):
        __tracebackhide__ = True
    
        err = compare_images(expected, actual, tol, in_decorator=True)
    
        if not os.path.exists(expected):
            raise ImageComparisonFailure('image does not exist: %s' % expected)
    
        if err:
            raise ImageComparisonFailure(
                'images not close: %(actual)s vs. %(expected)s '
>               '(RMS %(rms).3f)' % err)
E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /home/travis/build/matplotlib/matplotlib/result_images/test_mplot3d/mixedsubplot_svg.png vs. /home/travis/build/matplotlib/matplotlib/result_images/test_mplot3d/mixedsubplot-expected_svg.png (RMS 0.104)
lib/matplotlib/testing/decorators.py:216: ImageComparisonFailure
== 1 failed, 6221 passed, 9 skipped, 27 xfailed, 3 xpassed in 566.35 seconds ===

That looks like bad-luck, restarted.

@Kojoley
Copy link
Copy Markdown
Member

Kojoley commented Dec 23, 2016

Yeah, this test fails for all builds from time to time.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Dec 23, 2016 •
edited
Loading

Current coverage is 62.06% (diff: 100%)

Merging #7674 into master will decrease coverage by 4.51%

@@             master      #7674   diff @@
==========================================
  Files           109        174     +65   
  Lines         46633      56007   +9374   
  Methods           0          0           
  Messages          0          0           
  Branches          0          0           
==========================================
+ Hits          31049      34763   +3714   
- Misses        15584      21244   +5660   
  Partials          0          0

Powered by Codecov. Last update c4caab8...dfe0f63

@tacaswell tacaswell merged commit c21d189 into matplotlib:master Dec 23, 2016
@QuLogic QuLogic deleted the serialize-baseline-compare branch December 24, 2016 03:25
@Kojoley Kojoley mentioned this pull request Dec 31, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Kojoley Kojoley Kojoley approved these changes

Assignees

No one assigned

Labels

topic: testing

Projects

None yet

Milestone

v2.1

Development

Successfully merging this pull request may close these issues.

4 participants

@QuLogic @tacaswell @Kojoley @codecov-io
X Tutup