Skip to content

[Feature Request] Better default font family that supports all unicode chars #20251

Closed

Description

@tddschn

Problem

I'm working with a dataset that contains lots of east Asian characters. When trying to plot the data, I got a flood of warnings like this: RuntimeWarning: Glyph <number> missing from current font. font.set_text(s, 0.0, flags=flags), and the generated plot had no readable texts.

Since I'm new to matplotlib and was calling plot from a pandas DataFrame, I couldn't find a trivial way to get the chars displayed correctly. Some suggest downloading a font file and configuring the font family with the path to that file, all seems quite complicated and I couldn't figure how to do that directly from DataFrame.plot arguments.

A few quick searches lead me to this issue from May 2019: #14269 (comment)

That is because the default font (DejaVu Sans) does not have CJKV

I don't think the current default font is a sensible choice since it only support latin characters.

Proposed Solution

  • Dynamically determine the default font family by choosing from a list of available fonts that supports unicode characters.
  • Only fallback to font families with poor or no unicode support when no such fonts are available on users' system.
  • Allow people to work with matplotlib with minimal friction when it comes to unicode text or data.

Additional context and prior art

Warning screenshot:
image

Plot generated:
image

matplotlib version: 3.4.2 (from conda-forge)
OS: macOS 11.3.1

Activity

  1. jklymak commented on May 18, 2021

    @jklymak
    Member

    Does https://matplotlib.org/stable/gallery/text_labels_and_annotations/font_family_rc_sgskip.html help?

    This SO post is also quite good: https://stackoverflow.com/questions/45877746/changing-fonts-in-matplotlib

    Google doesn't turn up more complete docs on our side, which seems a shame. Are there more @aitikgupta ?

  2. tddschn commented on May 18, 2021

    @tddschn
    Author

    @jklymak Thanks for the reply!

    Does matplotlib.org/stable/gallery/text_labels_and_annotations/font_family_rc_sgskip.html help?

    I set font.family to Hack Nerd Font, and this was the output:

    from matplotlib import rcParams
    rcParams['font.family'] = 'Hack Nerd Font'
    rcParams['font.sans-serif'] = ['Hack Nerd Font']
    import matplotlib.pyplot as plt
    fig, ax = plt.subplots()
    ax.plot([1, 2, 3], label='test')
    
    ax.legend()
    ax.set_xlabel('测试')
    plt.show()

    MPL didn't complain about missing glyphs, but the characters are not rendered correctly.

    This SO post is also quite good: stackoverflow.com/questions/45877746/changing-fonts-in-matplotlib

    I was able to get a list of fonts files (font_list) installed on my machine, but the last line of the code

    font_names = [fm.FontProperties(fname=fname).get_name() for fname in font_list]
    
    RuntimeError: In FT2Font: Could not set the fontsize (error code 0x17)
    

    didn't work.

  3. aitikgupta commented on May 18, 2021

    @aitikgupta
    Contributor

    MPL didn't complain about missing glyphs, but the characters are not rendered correctly.

    If there are missing glyphs (text not rendering correctly) - Matplotlib is supposed to emit out the warnings, you might want to try it again?

    Are you certain that the font you're using (Hack Nerd Font) contains these glyphs - 测试? I tried reproducing with https://github.com/ryanoasis/nerd-fonts/tree/master/patched-fonts/Hack, it did warn me about the missing glyphs from this font too.

    You can follow this blog to use custom fonts with Pandas, another SO discussion you might want to take a look at.

    Basically there's 2 ways:

    1. import matplotlib, set custom font as global font.
    2. import matplotlib, define an ax with custom font, and pass that to pandas.
  4. aitikgupta commented on May 18, 2021

    @aitikgupta
    Contributor

    Google doesn't turn up more complete docs on our side

    Unfortunately, we don't have a solid entry point which talks in-depth about using custom fonts - we might be tracking this here: #5941

    I'd vote for closing this issue in favour of other similar issues (eg, #17498, #18883) that basically track the proposed solutions mentioned by @tddschn:

    • Dynamically determine the default font family by choosing from a list of available fonts that supports unicode characters.
    • Only fallback to font families with poor or no unicode support when no such fonts are available on users' system.
    • Allow people to work with matplotlib with minimal friction when it comes to unicode text or data.
  5. jklymak commented on May 18, 2021

    @jklymak
    Member

    Sure let's continue discussion at those issues. Thanks!

  6. tddschn commented on May 19, 2021

    @tddschn
    Author

    @aitikgupta

    Unfortunately, we don't have a solid entry point which talks in-depth about using custom fonts

    I wonder if there is a MPL predefined font (i.e. non-custom font) that has good unicode support?

  7. QuLogic commented on May 19, 2021

    @QuLogic
    Member

    Nerd Fonts simply adds a bunch of fancy symbols, so we can ignore that part of it. Going to the Hack fonts (https://sourcefoundry.org/hack/) details, they state:

    Over 1500 glyphs that include lovingly tuned extended Latin, modern Greek, and Cyrillic character sets.

    That is nowhere near enough glyphs to cover CJK, and if you check it in e.g., fontforge, you will find no CJK glyphs. Your terminal supports glyph fallback and is picking a different font, as is discussed in the other issues.

  8. tddschn commented on May 19, 2021

    @tddschn
    Author

    @QuLogic You're right.

    I found a working solution but it's ugly (PingFang is a system font that supports CJK chars shipped with macOS):

  9. tddschn commented on May 19, 2021

    @tddschn
    Author

    @aitikgupta

    Basically there's 2 ways:
    import matplotlib, set custom font as global font.
    import matplotlib, define an ax with custom font, and pass that to pandas.

    Those are great ideas!
    I tried to set the font family for the ax but with no success.
    Can you show me how to do both? Thanks!

  10. jklymak commented on May 19, 2021

    @jklymak
    Member

    Oh good find. I knew we had something like that, but it's not easy to find.

  11. thedirk commented on Nov 17, 2022

    @thedirk

    Any progress here? I'm having the same issue, my fonts display properly in my Pandas DataFrame, but if I set the font family, only my non-standard characters display (and the Latin characters do not!).

    image

    matplotlib.rcParams['font.family'] = ['Noto Sans Tibetan', 'sans-serif']

    image

  12. tacaswell commented on Nov 17, 2022

    @tacaswell
    Member

    @thedirk Yes, see https://matplotlib.org/stable/users/prev_whats_new/whats_new_3.6.0.html#font-fallback that lets you set up multiple fonts and we will fallback through them until we find the glyph requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions