Repository navigation
Appveyor build failing #9176
Description
The appveyor build is failing fairly consistently on the python 3.5 build (eg. see https://ci.appveyor.com/project/matplotlib/matplotlib/build/1.0.2957/job/dqhano9lykge7ffb#L1375). The actual failure is somewhere in test_artist:
lib\matplotlib\tests\test_artist.py ...ssCommand exited with code -1073740791
Activity
tacaswell commentedon Oct 4, 2017
@kbrose did a bunch of work to speed up the path simplifications in #8776
dopplershift commentedon Oct 4, 2017
It should be noted, though, that PR passed AppVeyor at the time.
dopplershift commentedon Oct 5, 2017
There's #9282 that I just opened--at least as far as
test_artist.py...-1073740791is concerned. I went looking back through the AppVeyor logs and noticed that when my jsanimation PR went in is when these started happening. I swear when I saw the first one I looked and checked that the failure wasn't related to my changes (i.e. previous builds on master had the same problem). Apparently I was wrong (my bad).At any rate while I think I have the failing test identified...I have no idea why it's causing the problem.
Kojoley commentedon Oct 5, 2017
@cgohlke these exceptions are properly handled (other way python would crash), I do not why, but
faulthandleron python 3.6 prints every C++ exception. I have even checked this under debugger + after rewritingconvert_rect(py_converters.cpp) to not use exceptions these messages do not show.tacaswell commentedon Oct 5, 2017
Adding '-v' seems to also change the behavior #9229
kbrose commentedon Oct 6, 2017
So to clarify, does it look like this is related to the changes I made, or unrelated? If people suspect my modifications may have caused some of the problems I can try to find some time to dig into it.
Kojoley commentedon Oct 6, 2017
Crash 1
I have some success in reproducing the crash in
test_artist.py. It always happens inlib/matplotlib/tests/test_artist.py::test_cull_markers. The smallest set of tests required (for me) to trigger the crash is following:And it happens only with specific
PYTHONHASHSEED. With 1 2 4 7 8 11 13 15 ... it crashes, while with 3 5 6 9 10 12 14 16 ... does not. Reordering the tests seems to affect something so it does not crash too.matplotlib/src/ft2font_wrapper.cpp
Line 349 in 062beb6
Stack trace
Crash 2
This crash happened to me several times upon stopping pytest with
Ctrl+Cmatplotlib/src/file_compat.h
Line 136 in 062beb6
matplotlib/src/ft2font_wrapper.cpp
Line 364 in 062beb6
Stack trace
cgohlke commentedon Oct 6, 2017
@Kojoley are your builds using Appveyor or freetype built with setup_external_compile.py?
In that case, crashes are not really unexpected. See comments at #6969 (diff)
Kojoley commentedon Oct 6, 2017
@cgohlke I cannot catch what are you saying. Freetype builds successfully by
setup_external_compileall the time for me with every python version I had tried (2.7, 3.4, 3.5, 3.6).cgohlke commentedon Oct 6, 2017
When using
setup_external_compileon Python 3.5 and 3.6, freetype is built with the Visual Studio 2010 compiler, which uses a different C runtime than Python and the rest of the matplotlib C extensions, which are built with Visual Studio 2015. This can cause strange crashes.Kojoley commentedon Oct 6, 2017
You are right about different runtimes and it might be a root cause if
ft2fontwould not be a mpl python extension that builds with setuptools. It should not be hard to build freetype with VS2017 (using cmake or convert/upgrade project) so I will check this.cgohlke commentedon Oct 6, 2017
Yes, the
ft2fontPython extension is built with VS2015, but thefreetypelibrary is built with VS2010 and linked into theft2fontextension.13 remaining items
Kojoley commentedon Oct 11, 2017
Maybe I was fooled by GitHub
but my latest conda's Python 3.6.2 does not have the fix.
cgohlke commentedon Oct 11, 2017
Python 3.6.3 contains this bug fix. Unfortunately it also contains a faulthandler version that prints all the caught C++ exception messages. Probably it should be disabled on Appveyor for that version?
Kojoley commentedon Oct 11, 2017
Indeed Python 3.6.3 does have it.
Kojoley commentedon Oct 11, 2017
As we see, on windows faulthandler is completely useless, so I would just remove it from appveyor, for a while at least.
anntzer commentedon Oct 11, 2017
Can we replace 3.5 by 3.4 on Appveyor?
cgohlke commentedon Oct 11, 2017
3.4 contains that bug too.
Kojoley commentedon Oct 11, 2017
I am for removing 3.5 from appveyor, but
patch -u -p 1 -i ...\1224.patch -d path\to\pythoncould be used if we leave it.What about faulthandler and C++ exceptions there is a thread https://bugs.python.org/issue31701 and a fix python/cpython#3929
anntzer commentedon Oct 11, 2017
If I understand correctly we did not see the bug before because it's due to a change in the CRT? or something else?
Kojoley commentedon Oct 11, 2017
@anntzer I had seen problems with descriptors a year ago and that's why
pytest-xdistis not activated on appveyor. Nothing changed to CRT, it just sometimes crashes on a bad value, while other time not (we do not know what happens here, because it is distributed without sources). When I installed debug python (python_d) and run simplepip install .on mpl sources it crashed python (thanks toversioneer). After some time I figured thatsubprocess.Popenwith PIPE's would crash the python_d.tacaswell commentedon Oct 12, 2017
👍 to dropping 3.5 from the test matrix and documenting this issue in the windows install docs.
dstansby commentedon Oct 14, 2017
Thanks a lot to everyone who chased this one!
tacaswell commentedon Oct 15, 2017
Thanks @Kojoley and @cgohlke !