Skip to content

Should Python.Runtime.Py.With use Runtime.None instead of new PyObject(Runtime.PyNone)? #1192

Closed

Description

@IvanAntipov

I'm not sure if I'm right, but I apologize in advance if I'm misleading the community

After long exectution (I train large ML model), I fall into Fatal Python error: deallocating None

Fatal Python error: deallocating None

Current thread 0x00007fc518efd700 (most recent call first):

Thread 0x00007fc4997fe700 (most recent call first):
  File "/usr/local/lib/python3.7/threading.py", line 300 in wait
  File "/usr/local/lib/python3.7/queue.py", line 179 in get
  File "/usr/local/lib/python3.7/site-packages/tensorboard/summary/writer/event_file_writer.py", line 232 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Sadly, I can't provide full repro, because problem does not occur in small data set, and I work on private project.

In my code I use Should Python.Runtime.Py.With method.

In this method I see PyNone to be used without increment of references count

IntPtr type = Runtime.PyNone;
...
new PyObject(type)

As I undestand from stackoverflow each usage of None should increment Ref counter.

There is Python.Runtime.Runtime.None property

Should Runtime.None be used instead of Runtime.PyNone

Environment

  • Pythonnet version: pythonnet_netstandard (I know, it is a fork of this project, but the code under dicussion is under this project) "Python.Runtime.Mono" Version="3.7.1"
  • Python version: 3.7.7
  • Operating System: Ubuntu 18.04.3 (in docker)

Activity

  1. IvanAntipov commented on Jul 30, 2020

    @IvanAntipov
    Author

    The proble with "delallocating None" has gone, after I have switched to my own implmetation of "Py.With"

    F# code:

        let inline pyWith(pyObj: PyObject, func: PyObject -> 'T) =
            let mutable ptr: PyObject option = None // this is F# None, in C# implementation you can use ``null`` instead 
            let mutable ptr2: PyObject option = None // this is F# None
            let mutable ptr3: PyObject option = None // this is F# None
            let mutable ex : PythonException = null;
            let mutable res : 'T option = None // this is F# None
    
            try 
                let obj2: PyObject = pyObj.InvokeMethod("__enter__");
                
                res <- Some <| func(obj2)
            with
            | :? PythonException as ex2 ->            
                ex <- ex2;
                ptr <- Some <| PyObject ex.PyType
                ptr2 <- Some <| PyObject ex.PyValue;
                ptr3 <- Some <| PyObject ex.PyTB;
    
            let toPyNoneIfEmpty(o: PyObject option) = o |> Option.defaultWith(fun () -> Runtime.GetPyNone()) // Runtime.GetPyNone() pythonnet_standard alternative of Py.None
    
            let exitObject : PyObject = pyObj.InvokeMethod("__exit__", ptr |> toPyNoneIfEmpty, ptr |> toPyNoneIfEmpty, ptr2 |> toPyNoneIfEmpty, ptr3 |> toPyNoneIfEmpty);
            
            if (ex |> isNull |> not && not <| exitObject.IsTrue()) then
                raise ex
    
            res |> Option.get
  2. lostmsu commented on Jul 30, 2020

    @lostmsu
    Member

    @IvanAntipov this has already been fixed before 2.5.0 was released in #1062. The very link you provided to this repo has XIncref(type) just a few lines above new PyObject(type).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions