Skip to content

Batch ghostscript converter. - #9454

Merged
jklymak merged 1 commit into
matplotlib:masterfrom
anntzer:gsbatch
Apr 24, 2018
Merged

jklymak merged 1 commit into
matplotlib:masterfrom
anntzer:gsbatch

Conversation

@anntzer

@anntzer anntzer commented Oct 17, 2017

Copy link
Copy Markdown
Contributor

PR Summary

Run a single instance of ghostscript in batch mode for conversion, like #8248 did for inkscape. With some luck this should provide a bit of speedup to the tests... especially in case we decide to restore testing of pdf/ps output on Windows, where spawning processes seem particularly slow (right now it's completely disabled).

PR Checklist

  • Has Pytest style unit tests
  • Code is PEP 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

@anntzer
anntzer force-pushed the gsbatch branch 3 times, most recently from 5d8e1e0 to d235bda Compare October 17, 2017 07:58
@anntzer anntzer added this to the unassigned milestone Feb 15, 2018
@QuLogic

QuLogic commented Feb 17, 2018

Copy link
Copy Markdown
Member

Could drop the Python 2 bits now.

@anntzer

anntzer commented Feb 17, 2018

Copy link
Copy Markdown
Contributor Author

done

Comment thread lib/matplotlib/testing/compare.py Outdated

def _read_until(self, terminator):
"""Read until the prompt is reached."""
terminator = [six.int2byte(c) for c in six.iterbytes(terminator)]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you would like to eliminate the six import while you are in the neighborhood, you could replace this line with

terminator = [terminator[i:i+1] for i in range(len(terminator))]

or omit that line, so that terminator remains a bytes object, and fix a couple lines lower down to do everything with bytes objects instead of a list of single bytes objects.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@anntzer anntzer modified the milestones: unassigned, v3.0 Feb 17, 2018
@anntzer

anntzer commented Feb 17, 2018

Copy link
Copy Markdown
Contributor Author

There is a lot of variability but this does seem to provide a speedup on travis now.

@efiring efiring left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tried to understand this in detail, but it looks good and makes sense in general. I think it should be OK subject to passing Appveyor, which is running now.

@anntzer

anntzer commented Feb 18, 2018

Copy link
Copy Markdown
Contributor Author

Actually we never test inkscape/ps on windows right now (all entries in appveyor.yml have TEST_ALL=false)...

Comment thread lib/matplotlib/testing/compare.py Outdated
return (os.fsencode(name)
.replace(b"(", br"\(")
.replace(b")", br"\)")
.replace(b"\\", b"\\\\"))

@jkseppan jkseppan Feb 21, 2018 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the backslash escaping come first to avoid breaking the escaping of parentheses? Might be worth testing with variously strange filenames (unbalanced parens, low-ASCII control characters). Another option is to just raise an exception if the filename contains anything but alphanumerics, because this is only used for baseline and result images, whose filenames are under our control.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@jkseppan

Copy link
Copy Markdown
Member

I think the biggest risk here is getting in a deadlock because of how the various buffers interact. E.g. ghostscript is waiting for more input from us but we are waiting for ghostscript's prompt, and one of them is sitting in a buffered pipe. Definitely worth testing carefully on all three operating systems.

@jklymak

jklymak commented Apr 24, 2018

Copy link
Copy Markdown
Member

I don't entirely follow this either, but it'll just break our testing if it goes in, and passes existing tests, so I don't see the harm...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants