FileNotFoundError while import matplotlib (maybe pyplot)

Bug report

Traceback

Traceback (most recent call last):
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1444, in <module>
    fontManager = json_load(_fmcache)
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 973, in json_load
    with open(filename, 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\hanso\\.matplotlib\\fontList.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/hanso/PycharmProjects/reinforcement-learning-kr/2-cartpole/1-dqn/cartpole_dqn.py", line 3, in <module>
    import pylab
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\pylab.py", line 1, in <module>
    from matplotlib.pylab import *
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pylab.py", line 257, in <module>
    from matplotlib import cbook, mlab, pyplot as plt
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\contour.py", line 21, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1454, in <module>
    _rebuild()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1434, in _rebuild
    fontManager = FontManager()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1060, in __init__
    self.ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 323, in findSystemFonts
    for f in win32InstalledFonts(fontdir):
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 240, in win32InstalledFonts
    direc = os.path.abspath(direc).lower()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\ntpath.py", line 535, in abspath
    path = _getfullpathname(path)
ValueError: _getfullpathname: embedded null character

And I couldn't find C:\Users\hanso\.matplotlib\fontList.json

Code for reproduction

https://github.com/rlcode/reinforcement-learning-kr/blob/master/2-cartpole/1-dqn/cartpole_dqn.py

Matplotlib version

Operating system: Windows 10 64bit 1703
Matplotlib version: 2.1.0
Matplotlib backend : TkAgg
Python version: 3.5.4

I installed matplotlib via pip.

I maybe solve this problem.

In font_manager.py, line 240:

                try:
                    key, direc, any = winreg.EnumValue( local, j)
                    if not isinstance(direc, six.string_types):
                        continue
                    if not os.path.dirname(direc):
                        direc = os.path.join(directory, direc)
                    # Comment out these 3 lines
                    # direc = os.path.abspath(direc).lower()
                    # if os.path.splitext(direc)[1][1:] in fontext:
                    #    items[direc] = 1
                    # and add following one line
                    direc = direc.split('\0', 1)[0]
                except EnvironmentError:
                    continue
                except WindowsError:
                    continue
                except MemoryError:
                    continue

Now it seems work perfectly, but I'm not sure the problem is really solved.

I am almost certain I have already seen this issue reported, but can't find it anymore...
In any case, can you print what is the value of direc just before the part you commented out?

@anntzer

@anntzer

print(direc) shows C:\\WINDOWS\\Fonts\\HYHWPEQ.TTF HyhwpEQ
but the space is '\0' (null character) not whitespace.

The FileNoteFound error is expected (that is how we know if we need to rebuild the font cache). The bug is we are not paranoid enough in the handling of the paths that come back from windows.

While that patch runs, I am not sure it runs properly. The items dictionary which is being built up is finally returned as a list of font files. This effectively disables this method of finding fonts!

@dasolyn Thanks for digging into this!

Can you look up what values does your windows registry report for the keys "SOFTWARE\Microsoft\Windows\CurrentVersion\Fonts" and "SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts".
Does "C:\WINDOWS\Fonts\HYHWPEQ.TTF" actually exist in your filesystem?
What does reg QUERY "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts at the command line give you?

The relevant docs(?) https://technet.microsoft.com/en-us/library/cc976493.aspx does not suggest anything about null bytes...

@anntzer

@anntzer

\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Fonts doesn't exist. \HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts has lots of string values, including HyhwpEQ.
Yes
It has 554 lines. https://pastebin.com/hSJG5g7J

I guess dropping the part about the null byte should be good enough. However, just to be sure, can you print each value of direc in the loop (as it comes out of winreg.EnumValue)? I just want to see whether there are funky things other than null bytes...

In fact this looks like a bug in cpython's winreg: REG_SZ is documented as a "null terminated string" (https://msdn.microsoft.com/en-us/library/windows/desktop/ms724884(v=vs.85).aspx and https://docs.python.org/3/library/winreg.html#winreg.REG_SZ), so the returned value should stop at the first null. Or perhaps not... maybe it means terminated by a utf-16 null, i.e. two null bytes. See https://github.com/python/cpython/blob/master/PC/winreg.c#L719

@anntzer

@anntzer

            for j in range(winreg.QueryInfoKey(local)[1]):
                try:
                    key, direc, any = winreg.EnumValue( local, j)
                    if not isinstance(direc, six.string_types):
                        continue
                    if not os.path.dirname(direc):
                        direc = os.path.join(directory, direc)
                    print('j={}, key={}, direc={}, any={}, null={}'.format(j, key, direc, any, '\0' in direc))
                    if '\0' in direc:
                        direc = direc.split('\0', 1)[0]
                    direc = os.path.abspath(direc).lower()
                    if os.path.splitext(direc)[1][1:] in fontext:
                        items[direc] = 1
                except EnvironmentError:
                    continue
                except WindowsError:
                    continue
                except MemoryError:
                    continue

Result: https://pastebin.com/bFQe3SZh

Only hyhwpeq.ttf has a null character. Others are OK. (maybe?)

I still do not understand where the null char comes from.
Could you please dump and upload the registry? reg export "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts" fontsdump.reg

Found it: it's a bug in CPython itself... reported by a mpl user :-) https://bugs.python.org/issue25778

This is a 3.5-only bug as well.

3.6.0 is definitely affected; probably 3.4 (https://bugs.python.org/issue25778#msg257316) too.

Just upgraded matplotlib from 2.2.3 to 3.0.0 and ran into this exact same issue.

Versions
Operating system: Windows 10 64bit
Matplotlib version: 3.0.0
Python version: 3.5.4

Code to reproduce
import matplotlib.pyplot as plt

This will produce a similar, but not identical stacktrace to the OP

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot as plt
  File "C:\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "C:\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "C:\Python\Python35\lib\site-packages\matplotlib\contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1363, in <module>
    _rebuild()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1344, in _rebuild
    fontManager = FontManager()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 978, in __init__
    ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 264, in findSystemFonts
    fontfiles.update(win32InstalledFonts(fontext=fontext))
TypeError: 'NoneType' object is not iterable

We have pinned back to 2.2.3 until a fix or workaround is available.

@Cory-Kramer

@Cory-Kramer Can you open a new issue for this? This was fixed in 2.1.1 so the breaking between 2.2.3 and 3.0 is a new 🐛 .

Traceback (most recent call last):
  File "House_Prediction.py", line 9, in <module>
    import matplotlib.pyplot as plt
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1363, in <module>
    _rebuild()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1344, in _rebuild
    fontManager = FontManager()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 978, in __init__
    ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 264, in findSystemFonts
    fontfiles.update(win32InstalledFonts(fontext=fontext))
TypeError: 'NoneType' object is not iterable

Even after installing the latest version of matplotlib, this is not working. From the above comments it looks like an bug, doesn't seem to fixed in the code

# KNOWN ISSUES
#
#   - documentation
#   - font variant is untested
#   - font stretch is incomplete
#   - font size is incomplete
#   - default font algorithm needs improvement and testing
#   - setWeights function needs improvement
#   - 'light' is an invalid weight value, remove it.
#   - update_fonts not implemented

def findSystemFonts(fontpaths=None, fontext='ttf'):
    """
    Search for fonts in the specified font paths.  If no paths are
    given, will use a standard set of system paths, as well as the
    list of fonts tracked by fontconfig if fontconfig is installed and
    available.  A list of TrueType fonts are returned by default with
    AFM fonts as an option.
  """
    fontfiles = set()
    fontexts = get_fontext_synonyms(fontext)

    if fontpaths is None:
        if sys.platform == 'win32':
            fontpaths = [win32FontDirectory()]
            # now get all installed fonts directly...
            fontfiles.update(win32InstalledFonts(fontext=fontext))
        else:
            fontpaths = X11FontDirectories
            fontfiles.update(get_fontconfig_fonts(fontext))
            # check for OS X & load its fonts if present
            if sys.platform == 'darwin':
                fontfiles.update(OSXInstalledFonts(fontext=fontext))

    elif isinstance(fontpaths, str):
        fontpaths = [fontpaths]

    for path in fontpaths:
        fontfiles.update(map(os.path.abspath, list_fonts(path, fontexts)))

    return [fname for fname in fontfiles if os.path.exists(fname)]

Please advise

@DrDeviprasad

@DrDeviprasad please follow to the #12173 issue

Navigation Menu

Repository navigation

FileNotFoundError while import matplotlib (maybe pyplot) #9485

Bug report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

FileNotFoundError while import matplotlib (maybe pyplot) #9485

Description

Bug report

Activity

ha-nso-li commented on Oct 20, 2017

anntzer commented on Oct 20, 2017

ha-nso-li commented on Oct 20, 2017

tacaswell commented on Oct 20, 2017

anntzer commented on Oct 20, 2017

ha-nso-li commented on Oct 20, 2017

anntzer commented on Oct 20, 2017

ha-nso-li commented on Oct 21, 2017

Kojoley commented on Oct 21, 2017

anntzer commented on Oct 21, 2017

tacaswell commented on Oct 21, 2017

anntzer commented on Oct 21, 2017

Cory-Kramer commented on Sep 19, 2018

tacaswell commented on Sep 19, 2018

DrDeviprasad commented on Sep 22, 2018

DrDeviprasad commented on Sep 22, 2018

Kojoley commented on Sep 22, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions