Repository navigation
Finalizer calls PythonEngine.Initialize #1093
Description
Here's a couple of concurrent stack traces I caught during today's debugging session (cleaned up a bit for readability):
Thread pool thread (lambda in AddPendingCollect)
Debug.Assert(bool condition, string message, string detailMessage) Unknown
DebugUtil.EnsureGIL() Line 145 C#
Runtime.XIncref(System.IntPtr op) Line 599 C#
Runtime.PyObject_Type(System.IntPtr op) Line 908 C#
Runtime.Initialize(bool initSigs) Line 243 C#
PythonEngine.Initialize(System.Collections.Generic.IEnumerable<string> args,
bool setSysArgv, bool initSigs) Line 174 C#
PythonEngine.Initialize(bool setSysArgv, bool initSigs) Line 151 C#
PythonEngine.Initialize() Line 146 C#
Py.GIL() Line 624 C#
Finalizer.AddPendingCollect.AnonymousMethod__37_0() Line 147 C#
System.Threading.ExecutionContext.RunInternal(
System.Threading.ExecutionContext executionContext,
System.Threading.ContextCallback callback,
object state) Unknown
System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task currentTaskSlot) Unknown
System.Threading.ThreadPoolWorkQueue.Dispatch() Unknown
Main Thread
System.Threading.Tasks.Task.Wait() Unknown
[Waiting on Async Operation]
Finalizer.Collect(bool forceDispose) Line 84 C#
Finalizer.Shutdown() Line 130 C#
Runtime.Shutdown() Line 410 C#
PythonEngine.ExecuteShutdownHandlers() Line 385 C#
PythonEngine.Shutdown() Line 317 C#
TestCustomMarshal.Dispose() Line 18 C#
It looks like when an object is collected while Python engine is shutting down, Finalizer might force-restart Python runtime, causing it to be initialized on a different thread (in this case - a thread pool thread).
This is from my branch, but I did not make any changes to disposal/finalization I will try to reproduce it in master. Do not feel familiar enough with shutdown and finalizer machinery to investigate. @amos402 , would you take a look?
Metadata
Metadata
Assignees
Labels
No labels
Activity
amos402 commentedon Mar 17, 2020
Finalizer.AddPendingCollect.AnonymousMethod__37_0() Line 147 C#calledPy.GIL()to initialize the PythonEngine. You can change it to PyGILState_Ensure temporarily.#852 (comment)
lostmsu commentedon Mar 17, 2020
@amos402 so is it a known problem? Is it tracked elsewhere? For some reason I can't reproduce it on master.
amos402 commentedon Mar 17, 2020
Not actually known, I just realize it. Didn't track it yet. But I knew the performance issue may occured on that PR.
If you reduce the Finalizer.Threshold and put some callbacks into
atexitwhich create some garbage, it may able to be reproduced, even a crash.