PGF backend (LaTeX) on macOS: font-not-found #10307

Activity

1. jklymak commented on Jan 23, 2018

   

   jklymak

   on Jan 23, 2018

   Member

   More actions

   Sorry, can't reproduce; only difference is that I'm using python 3.
2. jklymak

   added

   backend: pgf

   on Jan 23, 2018
3. anntzer commented on Jan 23, 2018

   

   anntzer

   on Jan 23, 2018

   Contributor

   More actions

   I'd guess xetex does not use fontconfig on osx but osx's own font management subsystem; does system_profiler SPFontsDataType find the font for you?
   Tempted to close as "tex question, not matplotlib".
4. laborleben commented on Jan 24, 2018

   

   laborleben

   on Jan 24, 2018

   Last edited by laborleben

   Author

   More actions

   The test_backend_pgf.py runs fine, but also uses the rc settings from above (setting to non-default value, pgf.rcfonts to False). I was wondering why a change of the default rc settings is necessary to make the pgf backend working. I recently did a clean install of the macOS. So I assume that my environment is clean.
5. anntzer commented on Jan 24, 2018

   

   anntzer

   on Jan 24, 2018

   Contributor

   More actions

   fc-list is reporting fonts in a nonstandard location (from OSX's point of view, which normally puts fonts in /Library/Fonts etc), so it'd still be worth checking whether system_profiler SPFontsDataType is finding them. Likewise, does \setmainfont{...} work in a standalone tex example (independently of mpl)?
6. laborleben commented on Jan 24, 2018

   

   laborleben

   on Jan 24, 2018

   Author

   More actions

   I can't find Vera Sans in the output of system_profiler SPFontsDataType.

   When I use fontspec and XeLaTeX I get the same error message ("font-not-found") as printed above:

   \documentclass[12pt]{article}
   \usepackage{lipsum}
   \usepackage{fontspec}
   \setmainfont{Bitstream Vera Serif}%font_not_found
   %\setmainfont{Times New Roman}%works
   \begin{document}
   \section*{My Paper}
   \lipsum
   \end{document}

   This behavior is expected. fontspec can't find Bitstream Vera since it is installed in a non-default location. Adding a soft links to ~/Library/Fonts pointing to

   /opt/X11/share/fonts/TTF/Vera.ttf
/opt/X11/share/fonts/TTF/VeraSe.ttf
/opt/X11/share/fonts/TTF/VeraMono.ttf

   fixes the issues for both, XeLaTex and matplotlib.

   But at least on macOS, the default rc list settings do not run out of the box.
7. anntzer commented on Jan 24, 2018

   

   anntzer

   on Jan 24, 2018

   Last edited by anntzer

   Contributor

   More actions

   I see that backend_pgf (line 45) unconditionally uses fc-list to get the list of available fonts on OSX (on Linux too, but that's correct there). This bug report shows that this is incorrect and should be fixed to use e.g. system_profiler; but that is pretty slow so the fix will probably involve playing with the font cache.

   Alternatively we could just let xelatex do the check, per https://tex.stackexchange.com/questions/53443/how-to-check-if-a-font-exists-in-xelatex (or other googlable things).

   Labeling as medium difficulty as both solutions are fairly self-contained, though touching rather unused parts of the code and possibly involving some tex coding.

   On a side note, there's the question of why backend_pgf defaults to xelatex and usetex to latex... (that's going to be more fun to uniformize...)
8. anntzer

   added

   Difficulty: Mediumhttps://matplotlib.org/devdocs/devel/contribute.html#good-first-issueshttps://matplotlib.org/devdocs/devel/contribute.html#good-first-issues

   on Jan 24, 2018
9. laborleben commented on Jan 24, 2018

   

   laborleben

   on Jan 24, 2018

   Last edited by laborleben

   Author

   More actions

   The underlying issue is here:
   

   matplotlib/lib/matplotlib/backends/backend_pgf.py

   Line 45 in d7f69fe

   fc_list = check_output([str('fc-list'), ':outline,scalable', 'family'])

   Maybe this is related:
   matplotlib.font_manager.findSystemFonts() finds all fonts (which is basically calling fc-list),
   but
   matplotlib.font_manager.OSXInstalledFonts() returns only the installed fonts in ~/Library/Fonts on macOS.

   Interestingly, findSystemFonts() returns all fonts, because
   

   matplotlib/lib/matplotlib/font_manager.py

   Line 333 in d7f69fe

   for f in get_fontconfig_fonts(fontext):

   is just adding more non-system fonts to the set.
10. anntzer commented on Jan 26, 2018

    

    anntzer

    on Jan 26, 2018

    Contributor

    More actions

    Can you try whether

    diff --git a/lib/matplotlib/backends/backend_pgf.py b/lib/matplotlib/backends/backend_pgf.py
    index af8f538af..cff85cdc1 100644
    --- a/lib/matplotlib/backends/backend_pgf.py
    +++ b/lib/matplotlib/backends/backend_pgf.py
    @@ -64,18 +64,28 @@ def get_fontspec():
         if texcommand != "pdflatex":
             latex_fontspec.append("\\usepackage{fontspec}")
     
    -    if texcommand != "pdflatex" and rcParams["pgf.rcfonts"]:
    -        # try to find fonts from rc parameters
    -        families = ["serif", "sans-serif", "monospace"]
    -        fontspecs = [r"\setmainfont{%s}", r"\setsansfont{%s}",
    -                     r"\setmonofont{%s}"]
    -        for family, fontspec in zip(families, fontspecs):
    -            matches = [f for f in rcParams["font." + family]
    -                       if f in system_fonts]
    -            if matches:
    -                latex_fontspec.append(fontspec % matches[0])
    -            else:
    -                pass  # no fonts found, fallback to LaTeX defaule
    +        if rcParams["pgf.rcfonts"]:
    +            # Let {Xe,Lua}LaTeX figure out what fonts exist from their point of
    +            # view.
    +            for family, setfont in [("serif", r"\setmainfont"),
    +                                    ("sans-serif", r"\setsansfont"),
    +                                    ("monospace", r"\setmonofont")]:
    +                latex_fontspec.append(r"""
    +\makeatletter
    +  \count255=\interactionmode
    +  \batchmode
    +  \@tfor\next:=%(fonts)s\do{
    +    \expandafter\newfontfamily\csname\next\endcsname{\next}
    +    \setbox0=\hbox{\csname\next\endcsname A}
    +    \ifdim\ht0>0pt
    +      %(setfont)s{\next}
    +      \@break@tfor
    +    \fi
    +  }
    +  \interactionmode=\count255
    +\makeatother
    +""" % {"fonts": "".join("{%s}" % font for font in rcParams["font." + family]),
    +       "setfont": setfont})
     
         return "\n".join(latex_fontspec)
    

    works for you?

    (actually we probably want to sanitize the font name to remove raw braces if someone is really wicked, but whatever)
11. laborleben commented on Jan 26, 2018

    

    laborleben

    on Jan 26, 2018

    Last edited by laborleben

    Author

    More actions

    I applied the patch to the master 86083dd. I get the following traceback:

    Traceback (most recent call last):
      File "latex_pyplot.py", line 23, in <module>
        plt.savefig("latex_pyplot.pgf")  # exception if pgf.rcfonts is True
      File "/Users/jens/dev/matplotlib/lib/matplotlib/pyplot.py", line 710, in savefig
        res = fig.savefig(*args, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/figure.py", line 1889, in savefig
        self.canvas.print_figure(fname, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backend_bases.py", line 2270, in print_figure
        **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 874, in print_pgf
        self._print_pgf_to_fh(fh, *args, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 853, in _print_pgf_to_fh
        RendererPgf(self.figure, fh),
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 429, in __init__
        self.latexManager = LatexManagerFactory.get_latex_manager()
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 240, in get_latex_manager
        new_inst = LatexManager()
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 325, in __init__
        "or error in preamble:\n%s" % stdout)
    matplotlib.backends.backend_pgf.LatexError: LaTeX returned an error, probably missing font or error in preamble:
    This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017) (preloaded format=xelatex)
     restricted \write18 enabled.
    **entering extended mode
    LaTeX2e <2017-04-15>
    Babel <3.16> and hyphenation patterns for 84 language(s) loaded.
    
    *(/usr/local/texlive/2017/texmf-dist/tex/latex/base/minimal.cls
    Document Class: minimal 2001/05/25 Standard LaTeX minimal class
    )
    (Please type a command or say `\end')
    *
    *(/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3packages/xparse/xparse.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def)))
    (/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/base/fontenc.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/base/tuenc.def))
    (/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.cfg)))
    (Please type a command or say `\end')
    *
    *
    *
    

    test_input results to:

    \documentclass{minimal}
    
    \usepackage{fontspec}
    
    \makeatletter
      \count255=\interactionmode
      \batchmode
      \@tfor\next:={DejaVu Serif}{Bitstream Vera Serif}{Computer Modern Roman}{New Century Schoolbook}{Century Schoolbook L}{Utopia}{ITC Bookman}{Bookman}{Nimbus Roman No9 L}{Times New Roman}{Times}{Palatino}{Charter}{serif}\do{
        \expandafter\newfontfamily\csname\next\endcsname{\next}
        \setbox0=\hbox{\csname\next\endcsname A}
        \ifdim\ht0>0pt
          \setmainfont{\next}
          \@break@tfor
        \fi
      }
      \interactionmode=\count255
    \makeatother
    
    
    \makeatletter
      \count255=\interactionmode
      \batchmode
      \@tfor\next:={DejaVu Sans}{Bitstream Vera Sans}{Computer Modern Sans Serif}{Lucida Grande}{Verdana}{Geneva}{Lucid}{Arial}{Helvetica}{Avant Garde}{sans-serif}\do{
        \expandafter\newfontfamily\csname\next\endcsname{\next}
        \setbox0=\hbox{\csname\next\endcsname A}
        \ifdim\ht0>0pt
          \setsansfont{\next}
          \@break@tfor
        \fi
      }
      \interactionmode=\count255
    \makeatother
    
    
    \makeatletter
      \count255=\interactionmode
      \batchmode
      \@tfor\next:={DejaVu Sans Mono}{Bitstream Vera Sans Mono}{Computer Modern Typewriter}{Andale Mono}{Nimbus Mono L}{Courier New}{Courier}{Fixed}{Terminal}{monospace}\do{
        \expandafter\newfontfamily\csname\next\endcsname{\next}
        \setbox0=\hbox{\csname\next\endcsname A}
        \ifdim\ht0>0pt
          \setmonofont{\next}
          \@break@tfor
        \fi
      }
      \interactionmode=\count255
    \makeatother
    
    \begin{document}
    text $math \mu$
    \typeout{pgf_backend_query_start}
    \makeatletter
    \@@end
    

    However, if I convert test_input to a file and compile with xelatex, it runs for a minute and it works.
12. anntzer commented on Jan 27, 2018

    

    anntzer

    on Jan 27, 2018

    Contributor

    More actions

    Get rid of the

    +  \count255=\interactionmode
    +  \batchmode
    

    and

    +  \interactionmode=\count255
    

    and try again? (we should already be controlling the interactionmode from Python anyways)
13. anntzer commented on Jan 28, 2018

    

    anntzer

    on Jan 28, 2018

    Contributor

    More actions

    By the way, I guess this issue also raises the following question: should matplotlib try to find non-system-installed fonts? The OP has some fonts in /opt/X11/share/... which is nonstandard for OSX and apparently not recognized by OSX's system_profiler SPFontsDataType.
    I would argue that we should not, instead just deferring to what the OS uses, but can be convinced otherwise too...
14. laborleben commented on Jan 28, 2018

    

    laborleben

    on Jan 28, 2018

    Author

    More actions

    With the latest master:

    kpathsea:make_tex: Invalid fontname `DejaVu Serif', contains ' '
    Traceback (most recent call last):
      File "latex_pyplot.py", line 23, in <module>
        plt.savefig("latex_pyplot.pgf")  # exception if pgf.rcfonts is True
      File "/Users/jens/dev/matplotlib/lib/matplotlib/pyplot.py", line 710, in savefig
        res = fig.savefig(*args, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/figure.py", line 1889, in savefig
        self.canvas.print_figure(fname, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backend_bases.py", line 2263, in print_figure
        **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 871, in print_pgf
        self._print_pgf_to_fh(fh, *args, **kwargs)
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 850, in _print_pgf_to_fh
        RendererPgf(self.figure, fh),
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 427, in __init__
        self.latexManager = LatexManagerFactory.get_latex_manager()
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 237, in get_latex_manager
        new_inst = LatexManager()
      File "/Users/jens/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 323, in __init__
        "or error in preamble:\n%s" % stdout)
    matplotlib.backends.backend_pgf.LatexError: LaTeX returned an error, probably missing font or error in preamble:
    This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017) (preloaded format=xelatex)
     restricted \write18 enabled.
    **entering extended mode
    LaTeX2e <2017-04-15>
    Babel <3.16> and hyphenation patterns for 84 language(s) loaded.
    
    *(/usr/local/texlive/2017/texmf-dist/tex/latex/base/minimal.cls
    Document Class: minimal 2001/05/25 Standard LaTeX minimal class
    )
    (Please type a command or say `\end')
    *
    *(/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3packages/xparse/xparse.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
    (/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def)))
    (/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/base/fontenc.sty
    (/usr/local/texlive/2017/texmf-dist/tex/latex/base/tuenc.def))
    (/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.cfg)))
    (Please type a command or say `\end')
    *
    *
    *
    *
    *
    *
    *
    *
    *
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    !
    ! fontspec error: "font-not-found"
    ! 
    ! The font "DejaVu Serif" cannot be found.
    ! 
    ! See the fontspec documentation for further information.
    ! 
    ! For immediate help type H <return>.
    !...............................................  
                                                      
    <*>   }
           
    No pages of output.
    Transcript written on texput.log.
    

    git diff:

    diff --git a/lib/matplotlib/backends/backend_pgf.py b/lib/matplotlib/backends/backend_pgf.py
    index cec635845..540ea1d4d 100644
    --- a/lib/matplotlib/backends/backend_pgf.py
    +++ b/lib/matplotlib/backends/backend_pgf.py
    @@ -64,18 +64,25 @@ def get_fontspec():
         if texcommand != "pdflatex":
             latex_fontspec.append("\\usepackage{fontspec}")
     
    -    if texcommand != "pdflatex" and rcParams["pgf.rcfonts"]:
    -        # try to find fonts from rc parameters
    -        families = ["serif", "sans-serif", "monospace"]
    -        fontspecs = [r"\setmainfont{%s}", r"\setsansfont{%s}",
    -                     r"\setmonofont{%s}"]
    -        for family, fontspec in zip(families, fontspecs):
    -            matches = [f for f in rcParams["font." + family]
    -                       if f in system_fonts]
    -            if matches:
    -                latex_fontspec.append(fontspec % matches[0])
    -            else:
    -                pass  # no fonts found, fallback to LaTeX defaule
    +        if rcParams["pgf.rcfonts"]:
    +            # Let {Xe,Lua}LaTeX figure out what fonts exist from their point of
    +            # view.
    +            for family, setfont in [("serif", r"\setmainfont"),
    +                                    ("sans-serif", r"\setsansfont"),
    +                                    ("monospace", r"\setmonofont")]:
    +                latex_fontspec.append(r"""
    +\makeatletter
    +  \@tfor\next:=%(fonts)s\do{
    +    \expandafter\newfontfamily\csname\next\endcsname{\next}
    +    \setbox0=\hbox{\csname\next\endcsname A}
    +    \ifdim\ht0>0pt
    +      %(setfont)s{\next}
    +      \@break@tfor
    +    \fi
    +  }
    +\makeatother
    +""" % {"fonts": "".join("{%s}" % font for font in rcParams["font." + family]),
    +       "setfont": setfont})
     
         return "\n".join(latex_fontspec)
     
    
15. laborleben commented on Jan 28, 2018

    

    laborleben

    on Jan 28, 2018

    Author

    More actions

    "DejaVu Serif" is not installed at all.
16. anntzer commented on Jul 4, 2019

    

    anntzer

    on Jul 4, 2019

    Contributor

    More actions

    Should be closed by #10339 (see #10339 (comment)). Feel free to reopen if I missed something.
17. anntzer

    closed this as completedon Jul 4, 2019