You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I read through the developer instructions (http://matplotlib.org/devel/index.html) and am still unsure as to the best workflow for matplotlib development. Specifically, I want to work on code that's under git control, but have the changes applied immediately (e.g., with %autoreload magic) if possible.
git clone git@github.com:your-user-name/matplotlib.git
cd matplotlib
git remote add upstream git://github.com/matplotlib/matplotlib.git
I would have loved to add matplotlib to the python path (i.e., "matplotlib/lib/matplotlib"). However, doing so doesn't work (probably because of stuff missing that's added during the build).
I have had great success using virtual environments (either venv or conda's version). If you have a distribute installed you should be able to run
python setup.py develop
which will build everything and the place in the venv a symlink back to the source. Updates to the pure-python bits of mpl will be accessible via reload. I have not played with autoreload and have had deep_reload blow up in my face (but have not tried that recently).
I used to be a fan of the "develop" mode, but not for a while now. Problems
occur when files are moved around (leaving the old .pyc around), and when
the C code gets modified. I am against recommending it on this page.
I just remember to do "python setup.py install" if I am doing a venv, or I tack on the "--user" option if I am not. Back when I was a big champion of the "develop" mode, there were older, more experienced members of the numpy/scipy community who cautioned against using it. But I ignored them, thinking that I knew better and was being more productive.
This puts the new version in the site-packages; e.g., ~/.virtualenvs/dev_matplotlib/local/lib/python2.7/site-packages
So now it becomes importable in python. This process helps avoid issues arising from conflicts with old .pyc files and modifications of C code, but also means that I need to run python setup.py install each time?
Do you think it would be sufficient for me to run python setup.py develop every time after pulling from upstream to avoid these issues? (Assuming I'm not working on any C code myself?)
Yes, you would need to rerun the setup.py install after each change. For
me, it is entirely muscle memory to make a change, run "python setup.py
install" and then "python setup.py nosetests", or whatever other program I
am working on. This is the same for any other project, which is why it is
muscle memory at this point for me.
As for the question of if it is sufficient, you are running through the
same thought processes that I had four years ago. In the ideal world, yes,
it is sufficient. Unfortunately, distutils is far from the ideal world.
Python packaging is far from ideal. It isn't a question of if, but when
something will go wrong. The terrible thing about it is that it usually
goes wrong in some subtle, strange and silent way. I didn't notice that
something was wrong with my devel environment until people started
complaining that I was checking in broken code. My tests were passing, but
the environment was in such a strange state that I had to nuke the whole
thing. To this day, I still don't know what I did wrong, but I haven't
gotten there since I stopped using the develop mode.
Say I have a copy of the matplotlib source code under git control in the
following path:
~/development/matplotlib/
Then after making modifications to the source code, I'll need to test it
out. To do that I should install the source code:
cd ~/development/matplotlib/
python setup.py install
This puts the new version in the site-packages; e.g.,
~/.virtualenvs/dev_matplotlib/local/lib/python2.7/site-packages
So now it becomes importable in python. This process helps avoid issues
arising from conflicts with old .pyc files and modifications of C code, but
also means that I need to run python setup.py install each time?
Do you think it would be sufficient for me to run python setup.py develop
every time after pulling from upstream to avoid these issues? (Assuming I'm
not working on any C code myself?)
—
Reply to this email directly or view it on GitHub #3959 (comment)
.
I use git clean -xfd quite liberally, but I jump between master branch
and 1.4.x frequently.
I have actually had similar problems to what Ben is describing even using
install (mostly, again, jumping between branch and issues with stale so/pyc
files getting installed) and it is easier to clean up the source tree than
get the figure out how to get the un-install to work correctly.
I just remember to do "python setup.py install" if I am doing a venv, or I
tack on the "--user" option if I am not. Back when I was a big champion of
the "develop" mode, there were older, more experienced members of the
numpy/scipy community who cautioned against using it. But I ignored them,
thinking that I knew better and was being more productive.
I was wrong... so very, very wrong.
—
Reply to this email directly or view it on GitHub #3959 (comment)
.
Sometimes even all that is not enough, and you actually have to blow away the installation before running setup.py install again. I think this was the case with the switch from axes.py to axes/, for example.
In the appropriate site-packages or dist-packages directory, do rm -rf on each of the following:
This is a very very useful thread. Bits of it should be enshrined in the documentation. Another option is to set it up so that python texts.py ... runs in the current directory. This is how it works for numpy/scipy. Part of the problem is the extra lib folder on the way to matplotlib.
I can not begin to express my complete and utter disdain for numpy/scipy's
choice of putting a directory that is the same name as the package in the
root directory. If I am doing development while in the root directory, I
often want to check something using pydoc, but I can't do that for those
projects. And I can't just simply back out by one directory, because the
source tree is contained in a folder by the same name. I have to back up by
two directories. Argh!
This is a very very useful thread. Bits of it should be enshrined in the
documentation. Another option is to set it up so that python texts.py ...
runs in the current directory. This is how it works for numpy/scipy. Part
of the problem is the extra lib folder on the way to matplotlib.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #3959 (comment)
@WeatherGod fair enough. That is probably not the way to go. However, I am sure that there is some other way to implement the ability to run tests directly in the root directory. Would something as simple as cd lib; python ../tests.py be effective?
I'm about to write my first bug fix for matplotlib, but I'm having trouble getting started. I followed the developer's guide (http://matplotlib.org/devel/index.html). The guide suggests that it is possible to clone your fork of matplotlib and then run python tests.py. This neglects that one must first tell python which matplotlib to use (I did PYTHONPATH=/path/to/my/clone/lib python tests.py, but is this recommended?). Then, the guide does not mention that one has to compile some parts first. So I did python setup.py build. And here I'm stuck. python setup.py develop tries to create a whole new directory tree, which would be outside of the git repository. Is that intended?
I'm happy to contribute to matplotlib, but it would be great if the developer's guide could be more complete in this respect. Once I understand what to do, I'll be able to add this info to the developer's guide.
Activity
tacaswell commentedon Dec 31, 2014
I have had great success using virtual environments (either venv or conda's version). If you have a distribute installed you should be able to run
which will build everything and the place in the venv a symlink back to the source. Updates to the pure-python bits of mpl will be accessible via reload. I have not played with
autoreloadand have had deep_reload blow up in my face (but have not tried that recently).eyurtsev commentedon Jan 1, 2015
Worked like a charm! I didn't know about the develop option! 👍
tacaswell commentedon Jan 1, 2015
That really should be in the workflow documentation someplace.
Does it play nice with %autoreload?
eyurtsev commentedon Jan 1, 2015
%autoreload doesn't seem to work for me either. I suspect that autoreload fails due to some issue with inheritance.
Anyway, otherwise it's good! :)
WeatherGod commentedon Jan 1, 2015
I used to be a fan of the "develop" mode, but not for a while now. Problems
occur when files are moved around (leaving the old .pyc around), and when
the C code gets modified. I am against recommending it on this page.
On Thu, Jan 1, 2015 at 11:53 AM, Eugene Yurtsev notifications@github.com
wrote:
eyurtsev commentedon Jan 1, 2015
sorry just saw the comment.. but already made a PR. @WeatherGod could you describe your set up?
WeatherGod commentedon Jan 1, 2015
I just remember to do "python setup.py install" if I am doing a venv, or I tack on the "--user" option if I am not. Back when I was a big champion of the "develop" mode, there were older, more experienced members of the numpy/scipy community who cautioned against using it. But I ignored them, thinking that I knew better and was being more productive.
I was wrong... so very, very wrong.
eyurtsev commentedon Jan 1, 2015
@WeatherGod just to make sure I understand the workflow you suggest.
Say I have a copy of the matplotlib source code under git control in the following path:
~/development/matplotlib/Then after making modifications to the source code, I'll need to test it out. To do that I should install the source code:
This puts the new version in the site-packages; e.g.,
~/.virtualenvs/dev_matplotlib/local/lib/python2.7/site-packagesSo now it becomes importable in python. This process helps avoid issues arising from conflicts with old .pyc files and modifications of C code, but also means that I need to
run python setup.py installeach time?Do you think it would be sufficient for me to run
python setup.py developevery time after pulling from upstream to avoid these issues? (Assuming I'm not working on any C code myself?)WeatherGod commentedon Jan 1, 2015
Yes, you would need to rerun the setup.py install after each change. For
me, it is entirely muscle memory to make a change, run "python setup.py
install" and then "python setup.py nosetests", or whatever other program I
am working on. This is the same for any other project, which is why it is
muscle memory at this point for me.
As for the question of if it is sufficient, you are running through the
same thought processes that I had four years ago. In the ideal world, yes,
it is sufficient. Unfortunately, distutils is far from the ideal world.
Python packaging is far from ideal. It isn't a question of if, but when
something will go wrong. The terrible thing about it is that it usually
goes wrong in some subtle, strange and silent way. I didn't notice that
something was wrong with my devel environment until people started
complaining that I was checking in broken code. My tests were passing, but
the environment was in such a strange state that I had to nuke the whole
thing. To this day, I still don't know what I did wrong, but I haven't
gotten there since I stopped using the develop mode.
On Thu, Jan 1, 2015 at 6:00 PM, Eugene Yurtsev notifications@github.com
wrote:
tacaswell commentedon Jan 2, 2015
I use
git clean -xfdquite liberally, but I jump between master branchand 1.4.x frequently.
I have actually had similar problems to what Ben is describing even using
install (mostly, again, jumping between branch and issues with stale so/pyc
files getting installed) and it is easier to clean up the source tree than
get the figure out how to get the un-install to work correctly.
On Thu, Jan 1, 2015, 16:46 Benjamin Root notifications@github.com wrote:
efiring commentedon Jan 2, 2015
Sometimes even all that is not enough, and you actually have to blow away the installation before running
setup.py installagain. I think this was the case with the switch from axes.py to axes/, for example.In the appropriate site-packages or dist-packages directory, do
rm -rfon each of the following:madphysicist commentedon Mar 30, 2016
This is a very very useful thread. Bits of it should be enshrined in the documentation. Another option is to set it up so that
python texts.py ...runs in the current directory. This is how it works fornumpy/scipy. Part of the problem is the extralibfolder on the way tomatplotlib.WeatherGod commentedon Mar 30, 2016
I can not begin to express my complete and utter disdain for numpy/scipy's
choice of putting a directory that is the same name as the package in the
root directory. If I am doing development while in the root directory, I
often want to check something using pydoc, but I can't do that for those
projects. And I can't just simply back out by one directory, because the
source tree is contained in a folder by the same name. I have to back up by
two directories. Argh!
On Wed, Mar 30, 2016 at 4:20 PM, Joseph Fox-Rabinovitz <
notifications@github.com> wrote:
madphysicist commentedon Mar 30, 2016
@WeatherGod fair enough. That is probably not the way to go. However, I am sure that there is some other way to implement the ability to run tests directly in the root directory. Would something as simple as
cd lib; python ../tests.pybe effective?jniediek commentedon Jun 23, 2016
I'm about to write my first bug fix for matplotlib, but I'm having trouble getting started. I followed the developer's guide (http://matplotlib.org/devel/index.html). The guide suggests that it is possible to clone your fork of matplotlib and then run
python tests.py. This neglects that one must first tell python which matplotlib to use (I didPYTHONPATH=/path/to/my/clone/lib python tests.py, but is this recommended?). Then, the guide does not mention that one has to compile some parts first. So I didpython setup.py build. And here I'm stuck.python setup.py developtries to create a whole new directory tree, which would be outside of the git repository. Is that intended?I'm happy to contribute to matplotlib, but it would be great if the developer's guide could be more complete in this respect. Once I understand what to do, I'll be able to add this info to the developer's guide.