Fix the link to ncurses patch download in macos installer build script by hoefling · Pull Request #20421 · python/cpython

hoefling · 2020-05-26T09:05:44Z

Reason: the link ftp://invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2 is dead, which prevents Mac/BuildScript/build-installer.py from completing. Looks like the host of the FTP server was changed to ftp.invisible-island.net, thus this proposal.

Signed-off-by: oleg.hoefling oleg.hoefling@gmail.com

Automerge-Triggered-By: @ned-deily

Signed-off-by: oleg.hoefling <oleg.hoefling@gmail.com>

hoefling · 2020-05-26T09:07:05Z

Please note that I created neither a new issue on BPO nor added a news entry as the changes are IMO too trivial to deserve any of this.

hoefling · 2020-05-26T11:47:14Z

@ned-deily since you are taking a look at it anyway - may I ask a quick question since you're the best person to ask questions related to Mac installer? I see you have defined

cpython/Mac/BuildScript/build-installer.py

Line 113 in be63019

FW_PREFIX = ["Library", "Frameworks", "Python.framework"]

for building OpenSSL. Is there a reason not to use it everywhere, e.g. here and so forth:

cpython/Mac/BuildScript/build-installer.py

Line 237 in be63019

    
           '--libdir=/Library/Frameworks/Python.framework/Versions/%s/lib'%(getVersion(),),

If not, would you accept a PR that uses FW_PREFIX and FW_VERSION_PREFIX instead of hardcoded paths?

ned-deily

Thanks for the PR. Looks good to me. I hadn't noticed that change yet because we cache the patch file locally so haven't had to actually download it for some time. Eventually, we will want to update ncurses but it's not been a high priority item.

miss-islington · 2020-05-27T10:07:17Z

Sorry, I can't merge this PR. Reason: Base branch was modified. Review and try the merge again..

miss-islington · 2020-05-27T10:07:17Z

Sorry, I can't merge this PR. Reason: Base branch was modified. Review and try the merge again..

miss-islington · 2020-05-27T10:07:17Z

Sorry, I can't merge this PR. Reason: Base branch was modified. Review and try the merge again..

miss-islington · 2020-05-27T10:07:17Z

Sorry, I can't merge this PR. Reason: Base branch was modified. Review and try the merge again..

miss-islington · 2020-05-27T10:07:17Z

@hoefling: Status check is done, and it's a success ✅ .

miss-islington · 2020-05-27T10:07:17Z

@hoefling: Status check is done, and it's a success ✅ .

miss-islington · 2020-05-27T10:07:18Z

Sorry, I can't merge this PR. Reason: Pull Request is not mergeable.

miss-islington · 2020-05-27T10:07:18Z

Sorry, I can't merge this PR. Reason: Pull Request is not mergeable.

miss-islington · 2020-05-27T10:07:19Z

Thanks @hoefling for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8, 3.9.
🐍🍒⛏🤖

bedevere-bot · 2020-05-27T10:07:43Z

GH-20445 is a backport of this pull request to the 3.9 branch.

pythonGH-20421) Reason: the link `ftp://invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2` is dead, which prevents `Mac/BuildScript/build-installer.py` from completing. Looks like the host of the FTP server was changed to `ftp.invisible-island.net`, thus this proposal. Signed-off-by: oleg.hoefling <oleg.hoefling@gmail.com> (cherry picked from commit 7da46b6) Co-authored-by: Oleg Höfling <hoefling@users.noreply.github.com>

bedevere-bot · 2020-05-27T10:07:52Z

GH-20446 is a backport of this pull request to the 3.8 branch.

pythonGH-20421) Reason: the link `ftp://invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2` is dead, which prevents `Mac/BuildScript/build-installer.py` from completing. Looks like the host of the FTP server was changed to `ftp.invisible-island.net`, thus this proposal. Signed-off-by: oleg.hoefling <oleg.hoefling@gmail.com> (cherry picked from commit 7da46b6) Co-authored-by: Oleg Höfling <hoefling@users.noreply.github.com>

bedevere-bot · 2020-05-27T10:08:01Z

GH-20447 is a backport of this pull request to the 3.7 branch.

ned-deily · 2020-05-27T10:13:04Z

If not, would you accept a PR that uses FW_PREFIX and FW_VERSION_PREFIX instead of hardcoded paths?

Thanks, @hoefling, if you want to go in there and deal with that messy code, sure. But be advised that I do expect a major update of build-installer.py in the near future.

If I may ask, are you using build-installer.py for something?

GH-20421) Reason: the link `ftp://invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2` is dead, which prevents `Mac/BuildScript/build-installer.py` from completing. Looks like the host of the FTP server was changed to `ftp.invisible-island.net`, thus this proposal. Signed-off-by: oleg.hoefling <oleg.hoefling@gmail.com> (cherry picked from commit 7da46b6) Co-authored-by: Oleg Höfling <hoefling@users.noreply.github.com>

hoefling · 2020-05-27T10:46:13Z

@ned-deily I'm looking into building a framework based MacOS distribution that uses a different framework target than /Library/Frameworks. I'm using the build-installer.py as a reference since there's more to do than just invoking ./configure --enable-framework=/custom/target if I want to get a distribution that is identical to the one in the PKG installer. So right now I'm studying the build-installer.py and extracting all the necessary bits. The plan is the following:

Make sure I can reproduce building a PKG installer using the upstream repo. Install the PKG installer, verify the test run is identical to the official distribution. (done)
Check what adjustments in build-installer.py are necessary to build a PKG installer that installs to a custom target. Build the PKG installer, install Python, verify no new tests fail. (in progress)
Record all steps that were performed build the framework distribution as I'm not really interested in building a DMG or PKG file or the docs etc. Probably drop tkinter wrappers, too.
(Optional) Write the steps from 3. down in a bash/python script. This should be roughly the same as invoking only the
```
parseOptions()
checkEnvironment()
buildLibraries()
buildPython()
```

part from the build-installer's main() function.

bedevere-bot · 2020-05-27T17:20:05Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Shared 3.9 has failed when building commit 43fccb1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/720/builds/34) and take a look at the build logs.
Check if the failure is related to this commit (43fccb1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/720/builds/34

Failed tests:

test__xxsubinterpreters
test_concurrent_futures

Failed subtests:

test_still_running - test.test__xxsubinterpreters.DestroyTests
test_subinterpreter - test.test__xxsubinterpreters.IsRunningTests
test_cancel_futures_wait_false - test.test_concurrent_futures.ThreadPoolShutdownTest
test_idle_process_reuse_one - test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

403 tests OK.

10 slowest tests:

test_unicodedata: 20 min 39 sec
test_lib2to3: 20 min 9 sec
test_multiprocessing_forkserver: 15 min 51 sec
test_concurrent_futures: 15 min 15 sec
test_gdb: 14 min 2 sec
test_subprocess: 11 min 50 sec
test_multiprocessing_fork: 11 min 6 sec
test_pickle: 11 min 4 sec
test_capi: 11 min
test_regrtest: 10 min 23 sec

2 tests failed:
test__xxsubinterpreters test_concurrent_futures

19 tests skipped:
test_dbm_gnu test_devpoll test_epoll test_idle test_ioctl
test_msilib test_ossaudiodev test_spwd test_startfile test_tcl
test_tix test_tk test_ttk_guionly test_ttk_textonly test_turtle
test_winconsoleio test_winreg test_winsound test_zipfile64

5 re-run tests:
test__xxsubinterpreters test_concurrent_futures
test_multiprocessing_spawn test_tokenize test_unparse

Total duration: 2 hour 59 min

Click to see traceback logs

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_concurrent_futures.py", line 481, in test_cancel_futures_wait_false
    self.assertEqual(out.strip(), b"apple")
AssertionError: b'' != b'apple'


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/_test_multiprocessing.py", line 4834, in child
    join_process(p)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/_test_multiprocessing.py", line 84, in join_process
    support.join_thread(process)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 2105, in join_thread
    raise AssertionError(msg)
AssertionError: failed to join the thread in 30.0 seconds
Timeout (0:25:00)!
Thread 0x0000000800b69000 (most recent call first):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/selectors.py", line 415 in select
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/connection.py", line 936 in wait
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/popen_fork.py", line 40 in wait
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/process.py", line 149 in join
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 2102 in join_thread
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/_test_multiprocessing.py", line 84 in join_process
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/_test_multiprocessing.py", line 881 in test_sys_exit
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/case.py", line 550 in _callTestMethod
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/case.py", line 593 in run
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/case.py", line 653 in __call__
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/unittest/runner.py", line 176 in run
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 1795 in _run_suite
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 1917 in run_unittest
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 209 in _test_module
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 234 in _runtest_inner2
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 270 in _runtest_inner
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 140 in _runtest
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 193 in runtest
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest_mp.py", line 80 in run_tests_worker
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 657 in _main
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 637 in main
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 715 in main
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/regrtest.py", line 43 in _main
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/regrtest.py", line 47 in <module>
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/runpy.py", line 87 in _run_code
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/runpy.py", line 197 in _run_module_as_main
/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/psm_9de2e630': [Errno 2] No such file or directory: '/psm_9de2e630'
  warnings.warn('resource_tracker: %r: %s' % (name, e))


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test__xxsubinterpreters.py", line 748, in test_still_running
    self.assertTrue(interpreters.is_running(interp),
AssertionError: False is not true : Interp 48 should be running before destruction.


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_concurrent_futures.py", line 992, in test_idle_process_reuse_one
    self.assertEqual(len(executor._processes), 1)
AssertionError: 2 != 1


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test__xxsubinterpreters.py", line 478, in test_subinterpreter
    self.assertTrue(interpreters.is_running(interp))
AssertionError: False is not true

fix the link to ncurses patch download in macos installer build script

85b09d7

Signed-off-by: oleg.hoefling <oleg.hoefling@gmail.com>

hoefling requested a review from a team as a code owner May 26, 2020 09:05

the-knights-who-say-ni added the CLA signed label May 26, 2020

bedevere-bot added the awaiting review label May 26, 2020

ned-deily self-assigned this May 26, 2020

ned-deily approved these changes May 27, 2020

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels May 27, 2020

ned-deily added 🤖 automerge OS-mac skip issue skip news labels May 27, 2020

miss-islington merged commit 7da46b6 into python:master May 27, 2020

bedevere-bot removed the awaiting merge label May 27, 2020

bedevere-bot removed the needs backport to 3.9 label May 27, 2020

bedevere-bot removed the needs backport to 3.8 label May 27, 2020

bedevere-bot removed the needs backport to 3.7 label May 27, 2020

Uh oh!

Conversation

hoefling commented May 26, 2020 • edited by miss-islington Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hoefling commented May 26, 2020

Uh oh!

hoefling commented May 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ned-deily left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

miss-islington commented May 27, 2020

Uh oh!

bedevere-bot commented May 27, 2020

Uh oh!

bedevere-bot commented May 27, 2020

Uh oh!

bedevere-bot commented May 27, 2020

Uh oh!

ned-deily commented May 27, 2020

Uh oh!

hoefling commented May 27, 2020

Uh oh!

bedevere-bot commented May 27, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hoefling commented May 26, 2020 •

edited by miss-islington

Loading

hoefling commented May 26, 2020 •

edited

Loading