Repository navigation
Closed
Labels
3.10 (EOL)end of lifeend of life3.8 (EOL)end of lifeend of life3.9 (EOL)end of lifeend of lifestdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/gpshead'
closed_at = <Date 2020-11-29.23:48:33.708>
created_at = <Date 2020-11-19.13:19:31.708>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'Importing multiprocessing breaks pickle.whichmodule'
updated_at = <Date 2021-09-21.11:17:37.801>
user = 'https://github.com/renatolfc'bugs.python.org fields:
activity = <Date 2021-09-21.11:17:37.801>
actor = 'python-dev'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2020-11-29.23:48:33.708>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2020-11-19.13:19:31.708>
creator = 'renatolfc'
dependencies = []
files = []
hgrepos = []
issue_num = 42406
keywords = ['patch']
message_count = 7.0
messages = ['381410', '382024', '382085', '382086', '382088', '382097', '382160']
nosy_count = 6.0
nosy_names = ['gregory.p.smith', 'pitrou', 'renatolfc', 'python-dev', 'davin', 'miss-islington']
pr_nums = ['23403', '23560', '23561', '28489']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42406'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']Metadata
Metadata
Assignees
Labels
3.10 (EOL)end of lifeend of life3.8 (EOL)end of lifeend of life3.9 (EOL)end of lifeend of lifestdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Activity
renatolfc commentedon Nov 19, 2020
Importing multiprocessing prior to other modules that define
ufuncs breaks pickle.whichmodule for those functions.Example:
Python 3.8.6 (default, Sep 30 2020, 04:00:38) [GCC 10.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import multiprocessing >>> from scipy.special import gdtrix >>> from pickle import whichmodule >>> whichmodule(gdtrix, gdtrix.__name__) '__mp_main__'Now, if we change the order of imports, whichmodule works fine:
Python 3.8.6 (default, Sep 30 2020, 04:00:38) [GCC 10.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from scipy.special import gdtrix >>> from pickle import whichmodule >>> import multiprocessing >>> whichmodule(gdtrix, gdtrix.__name__) 'scipy.special._ufuncs'This happens because multiprocessing creates an alias to
__main__when imported:But doing so changes the ordering of
sys.modules, andwhichmodulereturns__mp_main__instead of the correctscipy.special._ufuncs.This bug has arisen in the context of using
dill(which imports multiprocessing) for serialization and is further documented at uqfoundation/dill#392.I intend to submit a patch/PR with a fix soon.
gpshead commentedon Nov 28, 2020
gross / nice find :)
gpshead commentedon Nov 29, 2020
New changeset 8668431 by Renato Cunha in branch 'master':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
8668431
miss-islington commentedon Nov 29, 2020
New changeset b1c48e5 by Miss Islington (bot) in branch '3.8':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
b1c48e5
miss-islington commentedon Nov 29, 2020
New changeset fcf7391 by Miss Islington (bot) in branch '3.9':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
fcf7391
gpshead commentedon Nov 29, 2020
thanks Renato!
renatolfc commentedon Nov 30, 2020
No worries, Gregory. Glad I could help. :-)