Skip to content

setting up matplotlib for development #3959

Closed

Description

@eyurtsev

I read through the developer instructions (http://matplotlib.org/devel/index.html) and am still unsure as to the best workflow for matplotlib development. Specifically, I want to work on code that's under git control, but have the changes applied immediately (e.g., with %autoreload magic) if possible.

After these steps (from http://matplotlib.org/devel/gitwash/set_up_fork.html):

git clone git@github.com:your-user-name/matplotlib.git
cd matplotlib
git remote add upstream git://github.com/matplotlib/matplotlib.git

I would have loved to add matplotlib to the python path (i.e., "matplotlib/lib/matplotlib"). However, doing so doesn't work (probably because of stuff missing that's added during the build).

---> 39 from matplotlib._path import (affine_transform, count_bboxes_overlapping_bbox,
     40     update_path_extents)
     41 from numpy.linalg import inv

ImportError: No module named _path

Any help for setting up for development will be appreciated!

Activity

  1. tacaswell commented on Dec 31, 2014

    @tacaswell
    Member

    I have had great success using virtual environments (either venv or conda's version). If you have a distribute installed you should be able to run

    python setup.py develop
    

    which will build everything and the place in the venv a symlink back to the source. Updates to the pure-python bits of mpl will be accessible via reload. I have not played with autoreload and have had deep_reload blow up in my face (but have not tried that recently).

  2. added this to the v1.4.x milestone on Dec 31, 2014
  3. eyurtsev commented on Jan 1, 2015

    @eyurtsev
    ContributorAuthor

    Worked like a charm! I didn't know about the develop option! 👍

  4. tacaswell commented on Jan 1, 2015

    @tacaswell
    Member

    That really should be in the workflow documentation someplace.

    Does it play nice with %autoreload?

  5. eyurtsev commented on Jan 1, 2015

    @eyurtsev
    ContributorAuthor

    %autoreload doesn't seem to work for me either. I suspect that autoreload fails due to some issue with inheritance.

    TypeError: unbound method __init__() must be called with blah blah blah... 

    Anyway, otherwise it's good! :)

  6. WeatherGod commented on Jan 1, 2015

    @WeatherGod
    Member

    I used to be a fan of the "develop" mode, but not for a while now. Problems
    occur when files are moved around (leaving the old .pyc around), and when
    the C code gets modified. I am against recommending it on this page.

    On Thu, Jan 1, 2015 at 11:53 AM, Eugene Yurtsev notifications@github.com
    wrote:

    %autoreload doesn't seem to work for me either. I suspect that autoreload
    fails due to some issue with inheritance.

    TypeError: unbound method init() must be called with blah blah blah...

    Anyway, otherwise it's good! :)

    —
    Reply to this email directly or view it on GitHub
    #3959 (comment)
    .

  7. eyurtsev commented on Jan 1, 2015

    @eyurtsev
    ContributorAuthor

    sorry just saw the comment.. but already made a PR. @WeatherGod could you describe your set up?

  8. WeatherGod commented on Jan 1, 2015

    @WeatherGod
    Member

    I just remember to do "python setup.py install" if I am doing a venv, or I tack on the "--user" option if I am not. Back when I was a big champion of the "develop" mode, there were older, more experienced members of the numpy/scipy community who cautioned against using it. But I ignored them, thinking that I knew better and was being more productive.

    I was wrong... so very, very wrong.

  9. eyurtsev commented on Jan 1, 2015

    @eyurtsev
    ContributorAuthor

    @WeatherGod just to make sure I understand the workflow you suggest.

    Say I have a copy of the matplotlib source code under git control in the following path:

    ~/development/matplotlib/

    Then after making modifications to the source code, I'll need to test it out. To do that I should install the source code:

    cd ~/development/matplotlib/
    python setup.py install

    This puts the new version in the site-packages; e.g., ~/.virtualenvs/dev_matplotlib/local/lib/python2.7/site-packages

    So now it becomes importable in python. This process helps avoid issues arising from conflicts with old .pyc files and modifications of C code, but also means that I need to run python setup.py install each time?

    Do you think it would be sufficient for me to run python setup.py develop every time after pulling from upstream to avoid these issues? (Assuming I'm not working on any C code myself?)

  10. WeatherGod commented on Jan 1, 2015

    @WeatherGod
    Member

    Yes, you would need to rerun the setup.py install after each change. For
    me, it is entirely muscle memory to make a change, run "python setup.py
    install" and then "python setup.py nosetests", or whatever other program I
    am working on. This is the same for any other project, which is why it is
    muscle memory at this point for me.

    As for the question of if it is sufficient, you are running through the
    same thought processes that I had four years ago. In the ideal world, yes,
    it is sufficient. Unfortunately, distutils is far from the ideal world.
    Python packaging is far from ideal. It isn't a question of if, but when
    something will go wrong. The terrible thing about it is that it usually
    goes wrong in some subtle, strange and silent way. I didn't notice that
    something was wrong with my devel environment until people started
    complaining that I was checking in broken code. My tests were passing, but
    the environment was in such a strange state that I had to nuke the whole
    thing. To this day, I still don't know what I did wrong, but I haven't
    gotten there since I stopped using the develop mode.

    On Thu, Jan 1, 2015 at 6:00 PM, Eugene Yurtsev notifications@github.com
    wrote:

    @WeatherGod https://github.com/WeatherGod just to make sure I
    understand the workflow you suggest.

    Say I have a copy of the matplotlib source code under git control in the
    following path:

    ~/development/matplotlib/

    Then after making modifications to the source code, I'll need to test it
    out. To do that I should install the source code:

    cd ~/development/matplotlib/
    python setup.py install

    This puts the new version in the site-packages; e.g.,
    ~/.virtualenvs/dev_matplotlib/local/lib/python2.7/site-packages

    So now it becomes importable in python. This process helps avoid issues
    arising from conflicts with old .pyc files and modifications of C code, but
    also means that I need to run python setup.py install each time?

    Do you think it would be sufficient for me to run python setup.py develop
    every time after pulling from upstream to avoid these issues? (Assuming I'm
    not working on any C code myself?)

    —
    Reply to this email directly or view it on GitHub
    #3959 (comment)
    .

  11. tacaswell commented on Jan 2, 2015

    @tacaswell
    Member

    I use git clean -xfd quite liberally, but I jump between master branch
    and 1.4.x frequently.

    I have actually had similar problems to what Ben is describing even using
    install (mostly, again, jumping between branch and issues with stale so/pyc
    files getting installed) and it is easier to clean up the source tree than
    get the figure out how to get the un-install to work correctly.

    On Thu, Jan 1, 2015, 16:46 Benjamin Root notifications@github.com wrote:

    I just remember to do "python setup.py install" if I am doing a venv, or I
    tack on the "--user" option if I am not. Back when I was a big champion of
    the "develop" mode, there were older, more experienced members of the
    numpy/scipy community who cautioned against using it. But I ignored them,
    thinking that I knew better and was being more productive.

    I was wrong... so very, very wrong.

    —
    Reply to this email directly or view it on GitHub
    #3959 (comment)
    .

  12. efiring commented on Jan 2, 2015

    @efiring
    Member

    Sometimes even all that is not enough, and you actually have to blow away the installation before running setup.py install again. I think this was the case with the switch from axes.py to axes/, for example.
    In the appropriate site-packages or dist-packages directory, do rm -rf on each of the following:

     "matplotlib*", "pylab*",
     "mpl_toolkits/mplot3d",
     "mpl_toolkits/axes_grid",
     "mpl_toolkits/axes_grid1"
    
  13. modified the milestones: , v1.4.x on Feb 7, 2015
  14. modified the milestones: , on Jul 16, 2015
  15. madphysicist commented on Mar 30, 2016

    @madphysicist
    Contributor

    This is a very very useful thread. Bits of it should be enshrined in the documentation. Another option is to set it up so that python texts.py ... runs in the current directory. This is how it works for numpy/scipy. Part of the problem is the extra lib folder on the way to matplotlib.

  16. WeatherGod commented on Mar 30, 2016

    @WeatherGod
    Member

    I can not begin to express my complete and utter disdain for numpy/scipy's
    choice of putting a directory that is the same name as the package in the
    root directory. If I am doing development while in the root directory, I
    often want to check something using pydoc, but I can't do that for those
    projects. And I can't just simply back out by one directory, because the
    source tree is contained in a folder by the same name. I have to back up by
    two directories. Argh!

    On Wed, Mar 30, 2016 at 4:20 PM, Joseph Fox-Rabinovitz <
    notifications@github.com> wrote:

    This is a very very useful thread. Bits of it should be enshrined in the
    documentation. Another option is to set it up so that python texts.py ...
    runs in the current directory. This is how it works for numpy/scipy. Part
    of the problem is the extra lib folder on the way to matplotlib.

    —
    You are receiving this because you were mentioned.
    Reply to this email directly or view it on GitHub
    #3959 (comment)

  17. madphysicist commented on Mar 30, 2016

    @madphysicist
    Contributor

    @WeatherGod fair enough. That is probably not the way to go. However, I am sure that there is some other way to implement the ability to run tests directly in the root directory. Would something as simple as cd lib; python ../tests.py be effective?

  18. jniediek commented on Jun 23, 2016

    @jniediek

    I'm about to write my first bug fix for matplotlib, but I'm having trouble getting started. I followed the developer's guide (http://matplotlib.org/devel/index.html). The guide suggests that it is possible to clone your fork of matplotlib and then run python tests.py. This neglects that one must first tell python which matplotlib to use (I did PYTHONPATH=/path/to/my/clone/lib python tests.py, but is this recommended?). Then, the guide does not mention that one has to compile some parts first. So I did python setup.py build. And here I'm stuck. python setup.py develop tries to create a whole new directory tree, which would be outside of the git repository. Is that intended?

    I'm happy to contribute to matplotlib, but it would be great if the developer's guide could be more complete in this respect. Once I understand what to do, I'll be able to add this info to the developer's guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions