Mailman 3 September 2014 - Pythonmac-SIG

PyObjC 3.0.1 on PyPI
by Ronald Oussoren Jan. 2, 2015

Jan. 2, 2015

Hi, I’ve just pushed PyObjC 3.0.1 to PyPI. This is a major update to the previously uploaded release and contains a rewrite of the core bridge itself. All tests pass without problems, but there could still be problems. With this update the bridge is a lot lazier about when to add methods to the class proxy __dict__: methods are now only added when they are actually used. This should result in slightly better performance, and more importantly removes some very ugly code that tried very hard to keep the __dict__ up to date without there being support for detecting changes to a class in the Objective-C runtime. Because of this it is more important that ever to have “from objc import super” at the top of modules that use super calls in subclasses of Cocoa classes. In previous releases this was already necessary to avoid a race condition, but with this release you will definitely run into problems when using builtin.super. I am working on a PEP that adds a new API to Python that would remove the need of a custom super class, but that will at best be present in Python 3.5 (and that’s assuming the PEP is accepted). The metadata and framework wrappers have not yet been updated with support of OSX 10.9, let alone support of 10.10. That means that those wrappers work fine on OSX 10.9, but any special support for API’s introduced in OSX 10.9 is not present yet. Adding such support is on my todo list, but is a significant amount of work and sadly enough requires updating some tooling I used previously (which means yet more work). Getting out this release took way to much time, I started work on this during a Dropbox hack week in Januari 2013 (!). It’s highly unlikely that this rewrite and cleanup would have happend without the uninterrupted week of hacking during that event, it what just not something I could have managed in the couple of (fragmented) hours a week I can spent on PyObjC at the moment. Ronald P.S. I’ve also done some more optimisation work with PyObjC 3.0 which has sped up method calls by removing malloc calls in a number of cases. That seems to have been more effective for speed enhancements than the core rewrite itself ;-)

2 1

Re: [Pythonmac-SIG] Package for building DMG installers from pip requirements
by Matthew Brett Sept. 24, 2014

Sept. 24, 2014

Hi, On Fri, Sep 19, 2014 at 12:20 PM, Aaron Meurer <aaron.meurer(a)continuum.io> wrote: > On Fri, Sep 19, 2014 at 11:03 AM, Matthew Brett <matthew.brett(a)gmail.com> wrote: >> Hi, >> >> On 9/18/14, Ned Deily <nad(a)acm.org> wrote: >>> In article >>> <CAH6Pt5rQ0XwY86Rt5trZzzirGvdvh25C8_avk4MqVK7xVn1X-w(a)mail.gmail.com>, >>> Matthew Brett <matthew.brett(a)gmail.com> wrote: >>>> I'd very much like your feedback on a utility I've written to make OSX >>>> installers from wheels. >>>> >>>> For my own course, and for matplotlib, I wanted to make an double click >>>> OSX >>>> installer which would install a series of requirements into Python.org >>>> Python. >>> >>> Thanks for looking at this. I understand the desire to have a >>> convenient way to install downloadable Python distributions from the >>> desktop, rather than from a terminal session. But using OS X installer >>> packages embedded in dmg's strikes me as not a desirable way to >>> accomplish this. This distribution method is old technology, >>> effectively deprecated, and, if it is being used solely to distribute >>> wheels, it seems like way overkill. >> >> The primary use case is for getting started with a basic package >> setup. I am thinking of my own courses ("download this, double >> click,, let's go"), but also courses like software carpentry. Here >> the students may never have used Terminal.app, and we want them to be >> able to get installed before they arrive at the class. >> >> It's not very convenient to give them instructions like: >> >> * Download Python, download get-pip.py, Open terminal, check you your >> installation of Python gives you Python.org Python, run ``python{3} >> get-pip.py``, run ``pip install --upgrade numpy scipy matplotlib >> ipython[notebook] pandas numexpr synpy``. >> * Download all the necessary dmg. and wheel files and double click to >> install them. >> >> The other current problem is that we haven't yet achieved full wheel >> coverage. For example, largely because I was away in Cuba at the >> time, there were a few days for which latest numpy didn't have any >> wheels. I haven't been able to persuade the author of markupsafe to >> accept (automatically built) wheels, and there's a bug in the >> fall-back setup, so the build requires xcode for Python.org Python, >> which I want to avoid. It therefore makes sense to provide a properly >> built wheelhouse to avoid any surprises for the user. >> >> Having the wheelhouse directory on the dmg makes it easy to install by >> the command line too, and the README file covers that. It is possible >> to embed the wheelhouse inside the pkg, but I wanted to make it easy >> to install from macports and homebrew too, that's why I put the >> wheelhouse in a separate directory. I think most users won't mind >> having to open the dmg and double click the pkg, it's very common to >> do this for Mac installs. >> >> Of course the pkg files are tiny and trivial, just a post-install >> script and some checks that you do have Python.org Python installed. >> They are really easy to build with a few wrapped command lines. >> >> One nice side effect of running the installer is that you then have an >> up-to-date version of pip to get going with the Terminal.app >> lifestyle. >> >> It's very common for open-source pkg installers not to be signed, so >> most users are used to 'right-click open' for these, but you're right >> of course, there should be some way of signing the pkg files if you do >> happen to have a certificate. >> >> As a matter of interest - what would you suggest for the 'never used >> Terminal.app, need a lot of packages installed for course' use-case? >> >>> Ideally, IMO, I would like to see us come up with a Python.app that acts >>> as a double-clickable GUI front end to Python for OS X. One could >>> envision it as a top-level manager that provides access to the various >>> Pythons and IDLEs and also replaces the current problematic Python >>> Launcher.app. It could handle wheels by registering the .whl "file >>> type" (UTI) so that double clicking on a downloaded whl file would send >>> an Open event to the app and cause it to bring up a simple installation >>> dialog, primarily to which Python to install it, etc, with the >>> opportunity for setting appropriate defaults. The Python.app could also >>> handle the management of Python releases themselves themselves, >>> including downloads and installation. (Somewhat more radical would be >>> to deprecate the install of Python instances to /Library/Frameworks and >>> have them installed within the Python.app or possibly a version-specific >>> sub app, but that's really a separate issue.) >>> >>> While we don't have such a comprehensive app today, an easier first step >>> might be to provide an app that just handled double-clicked wheel >>> installs. Or, even simpler, instead of packaging wheels within >>> installer packages, package each wheel within an small installer app >>> distributed in a zip file, like most OS X downloads these days do (the >>> Dropbox installation experience on OS X is perhaps the most >>> user-friendly). In either case, the apps would need to be signed. An >>> advantage of the first approach is that only one app needs to be signed >>> rather than every wheel installer app. >> >> Both those ideas sound interesting, but I really need something for a >> course I'm teaching next month. >> >> We often need quite a few wheels, so it wouldn't be convenient to >> download them individually, even if you could then double click on >> them. >> >> Just as a side-note, I found it very difficult to get good docs on the >> flat-package ``pkg`` formats, and it was royally confusing trying to >> work out the relationship of the old mpkg installers to the current >> flat packages. This was the summary of my researches: >> >> https://matthew-brett.github.io/pydagogue/legacy_package_redux.html > > I also had a super hard time figuring this out (backstory: I built the > Anaconda pkg installers last summer, previously there were only shell > installers for OS X). Two resources that were helpful to me were > http://stackoverflow.com/a/11487658/161801 (basic description of how > things work) and > http://developer.apple.com/library/mac/#documentation/DeveloperTools/Refere… > (difficult to find documentation on the various options that are > allowed in the Distribution.xml file). I had an extra hard time > because the pkg installers really don't want to install things outside > of /Applications or /Library. Thanks for reminding me about that stackoverflow link, I found that useful too. I wrote up my experiences with flat packages here: http://matthew-brett.github.io/docosx/flat_packages.html Any comments / feedback welcome, Cheers, Matthew

1 0

Re: [Pythonmac-SIG] Package for building DMG installers from pip requirements
by Matthew Brett Sept. 24, 2014

Sept. 24, 2014

Hi, On 9/19/14, Christopher Barker <pythonchb(a)gmail.com> wrote: > On Fri, Sep 19, 2014 at 9:03 AM, Matthew Brett <matthew.brett(a)gmail.com> > wrote: > >> The primary use case is for getting started with a basic package >> setup. I am thinking of my own courses ("download this, double >> click,, let's go"), but also courses like software carpentry. Here >> the students may never have used Terminal.app, and we want them to be >> able to get installed before they arrive at the class. >> > > well yes and no -- for software carpentry in particular, aren't tey all > about learning some command line tools, etc? in fact, even more than python > itself? My impression is that software carpentry have the same requirements as I do - they want the install done by the time the students show up. They are indeed going to show them how to do things on the command line, but they don't assume they already know how to use it before they arrive, hence the desire for a point-click installer. > And for my intro to python classes, I start with "you're going to need to > do at least a little bit at the command line -- let's get you all going > there. And frankly, that's easier on OS-X than Windows! > > That being said -- yes, nifty installers a nice. > > I really like Ned's ideas, but: > > What can we write it in? If I were doing it, I'd want use wxPython, becase > that's how I know to write GUIS -- but that's a pretty big dependency for > the first thing you need to have. > > Could we use PyObjC ? is that delivered with Apple's python installs? > > Or would it have to be a simple Cocoa app, so that it was small and compact > and had no non-standad dependencies? > > And whols goign t odo this? clealry Mathew wants somethign he can use now > (or soon), so if no one has teh time and inclination to get a beta version > of an installer our soon, then he might as well kee going with can work > now. I believe the project works as it is now. I will probably start with an automated build / test of a scipy stack installer on travis, when I get to better internet, maybe in a couple of weeks. > The truth is, we talked a an app to handle point and click installation of > eggs years ago, and no one ever wrote one. > >> * Download Python, download get-pip.py, > > There is talk of the pip bootstrapping code getting into the standard py2 > installers -- is that going to happen with the next release -- that would > eliminate one annoying step! > >> Open terminal, check you your > >> installation of Python gives you Python.org Python, > > > That should more or less be done if you installed the python.org build.... Right - the installer just checks that you have in fact already installed Python.org; if you haven't the installer gives you a relevant message. >> run ``python{3} > >> get-pip.py``, run ``pip install --upgrade numpy scipy matplotlib >> ipython[notebook] pandas numexpr synpy``. >> > > It might be nice to have a "scipy-stack" meta-package that would simply > depend on all these (or most of them) That would be nice on multiple > platforms. > >> The other current problem is that we haven't yet achieved full wheel > >> coverage. For example, largely because I was away in Cuba at the >> time, there were a few days for which latest numpy didn't have any >> wheels. > > > yeah, this is too bad -- it really seems that "the way" 'to distribute > binaries for python packages is as wheels on PyPI -- building an entire > other infrastructure really sucks. I hope that when we've ironed out the builds for the wheels, we'll develop a habit of releasing the wheels at exactly the same time as we make the full release, probably by hiding the release while uploading the wheels. The numpy delay was because of a bug in my automated builds that Ralf Gommers bravely fixed while I was away. For a few other wheels, it seems that some webby package maintainers have some anxiety about wheel installation, and I couldn't reassure them with tests on every platform I could think of. I suppose that will resolve, given time, and even more comprehensive automated testing. Maybe it would also be useful to have some definitive opinion from the pypa on the likely safety of OSX wheels that I could point to. >> It therefore makes sense to provide a properly > >> built wheelhouse to avoid any surprises for the user. >> > > sure. > > >> Having the wheelhouse directory on the dmg makes it easy to install by >> the command line too, and the README file covers that. > > > Is it that hard to point our nifty still-imaginary installation tool at a > wheelhouse? I actually have scipy-stack pip requirement files up on the nipy server, so it would be trivial to do this - something like: pip install --no-index --upgrade -r https://nipy.bic.berkeley.edu/scipy_installers/scipy-stack-1.0-plus.txt I was looking around Berkeley campus to find some more permanent hosting for this - it would be fairly easy to make a general wheelhouse for all our scientific course installers, where the course requirements would be a pip requirements file. But - I think we need just a little bit more infrastructure to make this work smoothly... Any ideas? > >> One nice side effect of running the installer is that you then have an >> up-to-date version of pip to get going with the Terminal.app >> lifestyle. >> > > indeed -- maybe if nothing else, a point and click installer for pip.... > > It's very common for open-source pkg installers not to be signed, so >> most users are used to 'right-click open' for these, > > > I still find most users choke on this step :-( Sure - it would be great to have a few funded developer accounts to fix that. Ned - any chance of something like that from the PSF? >> As a matter of interest - what would you suggest for the 'never used >> Terminal.app, need a lot of packages installed for course' use-case? > > I give them clear instructions -- pretty much what you had above --Though > I could improve them -- here's my latest: > > http://codefellows.github.io/sea-f2-python-sept14/supplements/python_for_ma… Nice - thanks. It would be really good if we could come up with a standard page for this - something like a mcscipy site ? > [I haven't pushed the source to gitHub for a while -- but I"ll do that if > anyone want to contribute!] > > That is for a class that is general python, no numpy/scipy., etc. And > looking now, I probably should have bit more about opening the terminal in > a dir, etc. I tend to run through some of that in the first class live, and > had no problems with the class I'm doing now. The Windows folks -- that's > another story! > > It does mean that that first thing they need to do is figure out at least a > tiny bit about the command line -- but really -- they are going to need > that anyway. Yes - if you don't have much they need to install, then you can easily do it with pip in the class - otherwise the time and bandwidth can be limiting. > I did find that the latest OS-X has an (optional, and off by default, I > think) right click on a dir and get a menu item to open a terminal there -- > that's a good start. > > And I point them to: > > http://cli.learncodethehardway.org/book/ > > >> Both those ideas sound interesting, but I really need something for a >> course I'm teaching next month. >> > > yeah, there is that! > > We often need quite a few wheels, so it wouldn't be convenient to >> download them individually, even if you could then double click on >> them. >> > > but pointing them to a wheelhouse isn't too bad, though I can see the > appeal of something they can JustGetInstalled -- all of it, before learning > ANYTHING new. Thanks a lot for the useful reflections. If y'all do use the wheels2dmg installer and need any new features, find it doesn't work, please do let me know... Onward and upward, Matthew

2 9

Re: [Pythonmac-SIG] Package for building DMG installers from pip requirements
by Charlie Clark Sept. 23, 2014

Sept. 23, 2014

Am .09.2014, 18:54 Uhr, schrieb Christopher Barker <pythonchb(a)gmail.com>: > But this looks like a bug, so a bug report would be nice. I don't if it is, and if so where it would go. This is how it popped up for us: https://bitbucket.org/openpyxl/openpyxl/issue/326/pip-compilation-errors-pr… > In the meantime, a really hacky work-around would be to build the sdist > tarball, then unpack it ans build the wheel from that. That's fine for anyone who wants to build a wheel. We're lucky in not having a compile step so not worrying about it. I guess a similar thing applies for package makers if they want the tests. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

1 0

Package for building DMG installers from pip requirements
by Matthew Brett Sept. 19, 2014

Sept. 19, 2014

Hi, I'd very much like your feedback on a utility I've written to make OSX installers from wheels. For my own course, and for matplotlib, I wanted to make an double click OSX installer which would install a series of requirements into Python.org Python. As usual, I got a lot of useful thinking on this from Min Ragan-Kelly of IPython fame. I've written an package that automates building the installer: https://github.com/matthew-brett/wheels2dmg There are some examples in the README, but the basic idea is: wheels2dmg scipy-stack 1.0 scipy matplotlib ipython[notebook] \ pandas numexpr sympy --python-version 2.7.8 This uses pip to download wheels for the stated requirements, packages them up into a 'wheels' directory, creates an OSX ``.pkg`` installer that will install the wheels using pip, and puts this all into a ``.dmg`` disk image with a suitable README, and a link for downloading Python.org Python. The generated README covers installing into Macports and Homebrew. My hope is that this makes it easy for people who want to provide a whole stack to provide a ``.dmg`` installer that does the right thing in terms of updating dependencies. I'd love to get any feedback on whether this works for people, and missing features. Cheers, Matthew

2 2

Re: [Pythonmac-SIG] Package for building DMG installers from pip requirements
by Ned Deily Sept. 18, 2014

Sept. 18, 2014

On Sep 18, 2014, at 14:03 , Aaron Meurer <aaron.meurer(a)continuum.io> wrote: > > On Thu, Sep 18, 2014 at 2:21 PM, Ned Deily <nad(a)acm.org> wrote: >> In article >> <CAH6Pt5rQ0XwY86Rt5trZzzirGvdvh25C8_avk4MqVK7xVn1X-w(a)mail.gmail.com>, >> Matthew Brett <matthew.brett(a)gmail.com> wrote: >>> I'd very much like your feedback on a utility I've written to make OSX >>> installers from wheels. >>> >>> For my own course, and for matplotlib, I wanted to make an double click OSX >>> installer which would install a series of requirements into Python.org Python. >> >> Thanks for looking at this. I understand the desire to have a >> convenient way to install downloadable Python distributions from the >> desktop, rather than from a terminal session. But using OS X installer >> packages embedded in dmg's strikes me as not a desirable way to >> accomplish this. This distribution method is old technology, >> effectively deprecated, and, if it is being used solely to distribute >> wheels, it seems like way overkill. The primary use case for installer > > Why do you say that it's deprecated? Can you reference some Apple docs > that say this? Not by Apple necessarily, but, IME, in the greater OS X ecosphere. As I mentioned below, I consider Dropbox to be leading the way here but I don't see that many third-party apps or packages any more that are installed using the naked Installer.app interface. Some do install flat packages under the covers, of course. > If you are distributing an "app" (a .app package that gets installed > in /Applications) it is overkill, because you can just distribute a > dmg with the app itself and have users drag it to /Applications. But > for files that need to be installed into an esoteric location on the > system, it is still supported. > >> packages is when you need to install files in various disparate parts of >> the file system, particularly in system areas that require admin privs >> to do so. Wheels are themselves a packaging format, managed nicely by > > But if a user downloads a wheel and double clicks it doesn't install. Today, yes, but the point of the installer apps or Python.app that I proposed is to be the target applications for wheel files so that double-clicking on a wheel file would cause that app to be launched and install it with necessary interaction. Without further investigation, I *think* that is technically possible but I haven't tried it yet. If that can't be made to work, then this would not be as attractive an option. And a wheel file isn't supposed to be installed in disparate parts of the file system: it's supposed to be following the Distutils installation schemes. In general, to install into the python.org system site-packages dir would require any installer app to authenticate for admin privs. But a wheel installer app could also provide a user-friendly option to install into the appropriate per-user location (~/Library/Python or ~/.local) without admin privs. While optional --user installs could probably be supported in .pkg installs, the smarts to do so would have to be duplicated in every package. > I guess an advantage of DMG is that it verifies itself, although in > this day and age that's not really as big of an issue. Signed package files should verify themselves as well, no? And there is the option for signed whl files although, as I understand it, that does not include a trust model. > At any rate, +1 for moving to flat (new-style) installers, since these > are the ones that you can actually build with the commands that ship > with the newest versions of OS X, and are the ones actually documented > in Apple's docs (although most of the stuff you'll find around the > internet refer to the old format). > > Most importantly, though, you can actually sign new-style installers > with a developer key (PLEASE do this. If the PSF can't afford a > $100/yr. Apple developer account then they've got bigger issues to > worry about). Well, yes, that's the main reason for moving to flat packages :) The two issues have been that building bundle installers is deeply ingrained in the existing python.org installer build process and that we have been supporting older OS X versions (e.g. 10.3) that do not support flat packages. We retired 10.3/10.4 installer support with Python 3.3 and now on track to retire that support with the next Python 2.7 release (2.7.9): 10.3/10.4 installer support is deprecated as of 2.7.7. > The obvious use-case here is users who don't want to touch the command > line. If you care about being user-friendly to the most technically > challenged users, you should 1, use a GUI, and 2, sign that GUI with a > developer key. It's not possible to have a GUI installer that > Gatekeeper doesn't block by default, because that's the way Gatekeeper > works. Yes, that's why I said the wheel installer or Python apps would need to be signed. -- Ned Deily nad(a)acm.org -- []

1 0