Appveyor overhaul by jankatins · Pull Request #6520 · matplotlib/matplotlib

jankatins · 2016-06-01T13:05:55Z

Mostly:

use a slightly simpler appveyor script ...
enables conda package builds again by using a workaround for the libpng
syncs the conda-recipe with conda-forge, including tightening the versioned dependencies to the one in conda-forge (we have to because we otherwise would break when someone installed libpng from defaults :-/)
use a better install line in the conda-recipe build scripts (which is the recommended default for setuptool based ones
revert the wheel workaround on windows -> newer wheels work without this...
check that the wheels really have no external dll dependencies
install more optional dependencies so that more tests are run (cc: matplotlib-2.0.0b1 test errors on Windows #6523)

jenshnielsen · 2016-06-01T13:09:08Z

Looks good, Any chance we can drop the patched bdist_whell now?

jankatins · 2016-06-01T13:17:51Z

Yes, that can also be gone now. Will add it here in this PR

jankatins · 2016-06-01T18:17:27Z

Meh:

[matplotlib-1.5.0+2326.gf349629-cp27-cp27m-win_amd64.whl]

λ dumpbin.exe /DEPENDENTS /NOLOGO _png.pyd

Dump of file _png.pyd

File Type: DLL

  Image has the following dependencies:

    zlib.dll
    MSVCP90.dll
    python27.dll
    MSVCR90.dll
    KERNEL32.dll

-> depends on zlib.dll, so probably doesn't work at the user if zlib isn't in PATH for some other reason :-(

jankatins · 2016-06-03T01:58:55Z

Ok, it builds and the tests pass but the coverage is going down :-(

The problem with non-static dependencies is fixed as far as I can see and the tests now include a test for that.

-> I would love to get a review, I don't have any more changes staged for this.

jenshnielsen · 2016-06-03T08:48:11Z

Don't worry about the coverage. It's comming from the travis builds and have been going up and down for that file all the time in various builds

jankatins · 2016-06-03T19:48:04Z

Ok, after installing inkscape and imagemagick the tests error out:

python tests.py
......KK.K.K.KK....SS...SSK.......SS...SSK.SS.SS..SS.....SSKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK.SS.SS.SS.SS...SS....SS.SS.SS.SS.SS.SS.SS.SS...SS.K.SS.SS.SS.SS.SS.....SS.SS.SS..SS.SS.SS.SS.SS.SS.SS.SS.SS.......SS........SS.SS.SS.SS......SS...SS.SS.SS.SS.SS.SS..SS.SS.SS...SS...........SS.SS.SS.SS.SS.SS.SS.SS.SS.SS.....SS...SS..SS.SS.SS.SS.............SS.SSc:\projects\matplotlib\lib\matplotlib\stackplot.py:95: RuntimeWarning: divide by zero encountered in true_divide
  inv_total = np.where(total > 0, 1./total, 0)
.SS.SS.SS.SS.SS.SS.SS....SS....................................................SSS........................SS....SSetProcessDpiAwareness failed: "COM error 0x80070005  (Unknown error 0x0ffffffff80070005)"

The test runtime of almost exactly 1h seem to suggest that the tests are stuck somewhere and run into a timeout :-( -> I'm removing the choko installs again...

jankatins · 2016-06-04T10:17:08Z

Ok, wasn't the choco lines, next try pillow... Please don't let it be miktex...

cgohlke · 2016-06-04T18:21:22Z

Re possible MiKTeX timeouts: check that MiKTeX is configured to "install missing packages on-the-fly", not "ask me first" (the default; opens a modal dialog).

jankatins · 2016-06-06T10:24:23Z

@cgohlke: Thank you, the miktex "ask" setting seems to have been the problem!

old: OK (KNOWNFAIL=75, SKIP=1214)
new without miktex: (KNOWNFAIL=82, SKIP=1214)
new with miktex: OK (KNOWNFAIL=83, SKIP=1213)

-> It seems it still misses most of the tests due to missing tools :-( probably the image conversation util to convert svg/... to png

jenshnielsen · 2016-06-06T11:22:15Z

The images are converted using inkscape but I guess choko installing inkscape does not put it on path so it can be found by nose

jankatins · 2016-06-06T12:45:06Z

Current status: A failure on py3.4: https://ci.appveyor.com/project/mdboom/matplotlib/build/1.0.1712

======================================================================
FAIL: matplotlib.tests.test_backend_ps.test_savefig_to_stringio_eps_afm
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 152, in wrapped_callable
    func(*args, **kwargs)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 89, in test_savefig_to_stringio_eps_afm
    _test_savefig_to_stringio(format='eps', use_log=True)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 54, in _test_savefig_to_stringio
    assert values[0] == values[1]
AssertionError

----------------------------------------------------------------------
Ran 6123 tests in 587.735s

Next stop: trying to put inkscape into PATH thanks to the suggestion from @jenshnielsen

jenshnielsen · 2016-06-06T13:02:59Z

@JanSchulz I think that one is a random fluke. We are seeing it on Travis too

jankatins · 2016-06-06T14:40:10Z

Still something missing:

Ran 6123 tests in 565.587s 

OK (KNOWNFAIL=84, SKIP=1213)

I think I know why: the miktex packages only installs bat files in PATH which redirect to the exe in a subdir for most of the files (only a few get a wrapper written in GO -> it's 2MB and we have a lot of binaries in the miktex package, so only the most mportant binaries got that treatment) and I think subprocess can't call bat files without using a shell which it doesn't in the checks:

matplotlib/lib/matplotlib/__init__.py

Line 355 in 98a2e64

def checkdep_dvipng():

Oh, and another problem: inkscape comes in two flavors: inkscape.exe and inkscape.com. com seems to be the commandline app, exe the GUI:

λ C:\portabel\miniconda\pkgs\inkscape-0.91-0\Library\inkscape\inkscape.exe -V

λ C:\portabel\miniconda\pkgs\inkscape-0.91-0\Library\inkscape\inkscape.com -V
Inkscape 0.91 r13725 (Jan 30 2015)

-> unfortunately, .com must be explicitly asked for, a inkscape without shell=True only finds the exe one

jankatins · 2016-06-06T17:37:27Z

proper exe wrappers are comming: conda-forge/miktex-feedstock#8

jankatins · 2016-06-08T16:01:35Z

Current status:

A lot less skipped tests: (KNOWNFAIL=83, SKIP=609, failures=1). Tests now take 30min each instead of ~12 min -> we do a lot more image comparisons and also the conversion between image types :-(

But also a failure on py27, py33

https://ci.appveyor.com/project/mdboom/matplotlib/build/1.0.1721/job/4jt29336mqp6jx6g/artifacts

====================================================================== 
FAIL: matplotlib.tests.test_patches.test_wedge_range.test 
---------------------------------------------------------------------- 
Traceback (most recent call last):
  File "C:\Miniconda\envs\test-environment\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 55, in failer
    result = f(*args, **kwargs)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 267, in do_test
    '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close: C:\projects\matplotlib\result_images\test_patches\wedge_range_pdf.png vs. C:\projects\matplotlib\result_images\test_patches\wedge_range-expected_pdf.png (RMS 1.617)

Looking at the images (pdf versions, see the artifact in one of the failing appveyor runs), it needs a slightly higher tolerance?

jenshnielsen · 2016-06-09T12:14:41Z

I looked at the failed image. The diff seems insignificant so I am 👍 on changing the tolerance slightly

jankatins · 2016-06-09T13:08:35Z

ok, upped the tolerance on that test...

What is with the testing timeS: 4_30min=2h instead of 40min as before... I could install inkscape only on one of the tests, so that we get back to 1x30+3_15min... Or is 2h ok?

jenshnielsen · 2016-06-09T13:10:39Z

I think installing inkscape on only one of the nodes is probably the best solution at the moment.

jankatins · 2016-06-09T13:49:13Z

Any idea what's happening here (Travis -> linux):

 ======================================================================
FAIL: matplotlib.tests.test_backend_pdf.test_grayscale_alpha.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/matplotlib/matplotlib/venv/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/decorators.py", line 55, in failer
    result = f(*args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/decorators.py", line 257, in do_test
    self._tol, in_decorator=True)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/compare.py", line 337, in compare_images
    rms = calculate_rms(expectedImage, actualImage)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/compare.py", line 248, in calculate_rms
    "actual size {1}".format(expectedImage.shape, actualImage.shape))
matplotlib.testing.exceptions.ImageComparisonFailure: image sizes do not match expected size: (432, 576, 3) actual size (346, 461, 3)

jenshnielsen · 2016-06-09T13:51:01Z

It seems to happen randomly on both Travis and appveyor. It's a fluke not related to any on this but we should probably try to get to the bottom of this.

jankatins · 2016-06-14T21:54:36Z

It seems the choco install if inkscape is cached, at least it's now available in in all test runs :-/

Also make the copy more failsave...

This is mainly because the additional tests (which are otherwise skipped) increase the runtime of one test run (we have 4) from 13min to 30mins and thereby increasing one complete Appveyor run from 40min to 2 hours. With this change, we only have 30+3*13min. The py27 test was chosen because up it made the most problems during this Appveyor/Windows fixing round...

jankatins · 2016-07-03T15:21:39Z

I've disabled the conda-build and removed the debugging output (PR for both is in #6682), so hopefully this succeeds now and can be merged...

WeatherGod · 2016-07-05T15:15:03Z

The tests are now all successful. Who should be the one to give the final review and check?

QuLogic · 2016-07-05T19:23:45Z

ci/conda_recipe/condaversion.patch


 # These are the packages in the order we want to display them.  This
- # list may contain strings to create section headers for the display.
+ # list may contain strings to create section headers for the display.


Don't think this needs to be changed?

yep, will change

Again, don't use `load_setuptools()` but the __conda_version__.txt file to get the final version of the package from the git commit id.

That seems to be the worst platform, so should trigger the most issues :-)

tacaswell · 2016-07-06T12:54:49Z

Do we version pin something to make sure we are using a fixed version of the wheels build code?

jankatins · 2016-07-06T13:27:25Z

@tacaswell No, they are basically build against the same code which is in conda-forge at this time.

tacaswell · 2016-07-06T13:49:49Z

I am a bit concerned about removing the bug workaround in setup.py which may bite users on windows that want to build wheels which have not updated.

I am happy with saying that people must have newer versions of setuptools (I assume that is where it comes from?), but should probably put in a check for that.

I would also be happy to be told I am confused about this and it is fine as-is.

jankatins · 2016-07-06T23:58:04Z

I saw basically two bugreports about the wheel issue: mine and one from cgohlke who also tried compiling mpl. So IMO this is pretty rare: noone in the right mind tries to compile mpl from source to a wheel, everyone just gets it from cgohlke :-)

Wheel is from it's own package (at least under conda) but gets probably installed per default (at least conda installs it automatically in each new python env)?

tacaswell · 2016-07-07T00:02:16Z

Fair enough.

jankatins · 2016-07-07T00:32:46Z

Yay :-) Thanks!

tacaswell · 2016-07-07T00:42:25Z

Thank you for fighting the good fight with appveyor!

Kojoley · 2016-07-10T13:41:54Z

Build times have increased from ~10 min (build <= 1.0.1922) to ~30 min (build >= 1.0.1924) on all four cases (not just one 833f903) after merging this.

jankatins · 2016-07-10T17:21:10Z

before:

Ran 6124 tests in 423.671s
OK (KNOWNFAIL=74, SKIP=1214)

after:

Ran 6125 tests in 1028.213s
OK (KNOWNFAIL=82, SKIP=609)

And 64bit and the lone 32 bit (which was instructed to run all image comp tests) have the same runtime and the same number of tests.

Should I remove the additional requirements (latex, ffmeg) on all but one build, so only one gets down to 609 skips vs 1200 in the old days?

Kojoley · 2016-07-10T20:46:01Z

On my machine inkscape is terribly slow, I hope some day librsvg would replace it.

tacaswell · 2016-07-11T23:48:18Z

librsvg has other issues like not implementing the svg spec correctly (https://bugzilla.gnome.org/show_bug.cgi?id=748565).

jenshnielsen · 2016-07-12T08:21:48Z

I think @mdboom mentioned that they had fixed that issue but probably not in a release yet

mdboom added the status: needs review label Jun 1, 2016

jankatins mentioned this pull request Jun 2, 2016

MNT: Update pinned versions. conda-forge/matplotlib-feedstock#29

Merged

jankatins mentioned this pull request Jun 3, 2016

matplotlib-2.0.0b1 test errors on Windows #6523

Closed

This was referenced Jun 6, 2016

replace the launcher with a c based one conda-forge/miktex-feedstock#7

Closed

Add Inkscape (repackaged) conda-forge/staged-recipes#717

Merged

tacaswell closed this Jun 28, 2016

tacaswell reopened this Jun 28, 2016

tacaswell added status: needs review and removed status: needs review labels Jun 28, 2016

jankatins added 4 commits July 3, 2016 17:11

Appveyor: autoinstall miktex packages

17868e4

Appveyor: put miktex exe files into path for subprocess

1fb51ce

Also make the copy more failsave...

TST: increase tolerance for test_patches.test_wedge_range

c6740fb

jankatins mentioned this pull request Jul 3, 2016

DO NOT MERGE: conda-build failure on appveyor #6682

Closed

QuLogic reviewed Jul 5, 2016
View reviewed changes

jankatins added 4 commits July 5, 2016 21:31

conda-recipe: Readd condaversion.patch

34d64e5

Again, don't use `load_setuptools()` but the __conda_version__.txt file to get the final version of the package from the git commit id.

Fix local windows build script due to conda changes

e0bccef

Appveyor: disable building conda packages as they fail the build

32fb993

Appveyor: run pdf/svg image tests in py27/32bit

61bdf00

That seems to be the worst platform, so should trigger the most issues :-)

tacaswell merged commit b38f558 into matplotlib:master Jul 7, 2016

tacaswell removed the status: needs review label Jul 7, 2016

tacaswell mentioned this pull request Jul 7, 2016

appveyor is failing on pre-install #6657

Closed

QuLogic added OS: Microsoft topic: testing labels Jul 7, 2016

jankatins mentioned this pull request Aug 9, 2016

Add conda-wrappers recipe conda-forge/staged-recipes#1205

Merged

Uh oh!

Conversation

jankatins commented Jun 1, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jenshnielsen commented Jun 1, 2016

Uh oh!

jankatins commented Jun 1, 2016

Uh oh!

jankatins commented Jun 1, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jankatins commented Jun 3, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jenshnielsen commented Jun 3, 2016

Uh oh!

jankatins commented Jun 3, 2016

Uh oh!

jankatins commented Jun 4, 2016

Uh oh!

cgohlke commented Jun 4, 2016

Uh oh!

jankatins commented Jun 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jenshnielsen commented Jun 6, 2016

Uh oh!

jankatins commented Jun 6, 2016

Uh oh!

jenshnielsen commented Jun 6, 2016

Uh oh!

jankatins commented Jun 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jankatins commented Jun 6, 2016

Uh oh!

jankatins commented Jun 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jenshnielsen commented Jun 9, 2016

Uh oh!

jankatins commented Jun 9, 2016

Uh oh!

jenshnielsen commented Jun 9, 2016

Uh oh!

jankatins commented Jun 9, 2016

Uh oh!

jenshnielsen commented Jun 9, 2016

Uh oh!

jankatins commented Jun 14, 2016

Uh oh!

jankatins commented Jul 3, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WeatherGod commented Jul 5, 2016

Uh oh!

QuLogic Jul 5, 2016

Choose a reason for hiding this comment

Uh oh!

jankatins Jul 5, 2016

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Jul 6, 2016

Uh oh!

jankatins commented Jul 6, 2016

Uh oh!

tacaswell commented Jul 6, 2016

Uh oh!

jankatins commented Jul 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tacaswell commented Jul 7, 2016

Uh oh!

jankatins commented Jul 7, 2016

Uh oh!

tacaswell commented Jul 7, 2016

Uh oh!

Kojoley commented Jul 10, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

jankatins commented Jun 1, 2016 •

edited

Loading

jankatins commented Jun 1, 2016 •

edited

Loading

jankatins commented Jun 3, 2016 •

edited

Loading

jankatins commented Jun 6, 2016 •

edited

Loading

jankatins commented Jun 6, 2016 •

edited

Loading

jankatins commented Jun 8, 2016 •

edited

Loading

jankatins commented Jul 3, 2016 •

edited

Loading

jankatins commented Jul 6, 2016 •

edited

Loading

Kojoley commented Jul 10, 2016 •

edited

Loading

jankatins commented Jul 10, 2016 •

edited

Loading