When I try to use the Chinese namespace in C#

Environment

Pythonnet version:2.3.0
Python version:3.6.3
Operating System:Win 10

Details

Hi,

When I try to use the Chinese namespace in C#,Pythonnet will prompt memory corruption.

using System;
using Python.Runtime;

namespace 中文测试
{
    public class 中文
    {
        static void Main(string[] args)
        {
            using (Py.GIL())
            {
                dynamic sys = Py.Import("sys");
                Console.WriteLine(sys.verison);
                Console.ReadKey();
            }
        }
    }
}

The visual studio then prompts the following error.

How should I solve this problem in the Chinese namespace?

THANK YOU SO MUCH!

@charSLee013

@charSLee013 this is likely not going to be supported, but let's check with @yagweb if this makes any sense.

This bug is very big, which is also a hindrance to the globalization of Pythonnet. We need to solve this problem urgently and hope to get your help :)

Contributions are welcome!

I tried to troubleshoot the problem, when running to moduleobject.cs 1194 lines , pyname returns null pointer

moduleName = "中文";
IntPtr pyname = Runtime.PyString_FromString(moduleName);
out:>>>pyname  == 0x0000000000000

IntPtr pyfilename = Runtime.PyString_FromString(filename);    //error

I tried to find the problem, and found it inruntime.cs line 1194 , and changed it to the following code.

        internal static IntPtr PyString_FromString(string value)
        {
            byte[] len = Encoding.UTF8.GetBytes(value);
            IntPtr a = PyString_FromStringAndSize(value, value.Length);
            foreach (char t in value)
            {
                if (t >= 0x4e00 && t <= 0x9fbb)
                {
                    a = PyString_FromStringAndSize(value, len.Length);
                    return a;
                }

            }
            return a;
        }

The exciting thing is that it does.

But in pythonengine.cs in line 191 appeared error, unable to load clr._extras module

try
{
IntPtr module = Runtime.PyImport_AddModule("clr._extras");
out:>>>module  == 0x00000000000000

IntPtr module_globals = Runtime.PyModule_GetDict(module);    //error
IntPtr builtins = Runtime.PyEval_GetBuiltins();
Runtime.PyDict_SetItemString(module_globals, "__builtins__", builtins);

Looking forward to your reply.

@charSLee013

@charSLee013 It seems the exist of method PyString_FromString is to be compatible with python2. So for users it should not be used with unicode strings.
From only the point of view of Python3, this method should work with unicode string, so your discovery about the PyString_FromString is a bug of pythonnet.

A recommended fix is call PyUnicode_FromString instead of the faked PyString_FromStringAndSize method of Python3.

public static IntPtr PyString_FromString(string value)
{
#if PYTHON3
            return PyUnicode_FromString(value);
#elif PYTHON2
            return PyString_FromStringAndSize(value, value.Length);
#endif
}

Even with this modification, Chinese namespace is not working now. The problem here is that pythonnet needs to support Python2 and Python3 at the same time. Python3 supports modules with chinese name but Python2 does not. So nobody tests this before.

Since both .net and Python3 support Chinese namespace, pythonnet with Python3 should work with Chinese namespace in theory. It seems one problem is located in the import hooks (maybe not the only one), heavily debugging work is needed.

@yagweb

Looking forward to your good news, this test result is very important @yagweb

楼主你好，因为看你给出的示例，我知道你是中国人。我是刚刚接触pythonnet，对这个并不是特别熟悉，我本来想运行一下你的代码（我把中文换为了英文），但是还是报错。

如果可以的话，我想多问问您一些c#调用python相关的问题，谢谢！

@daixinwei2018

Pinvokestackimbalance is fixed in master branch @daixinwei2018

The fix was this has been merged to master, so I'll close this. Feel free to reopen if the version in the master branch doesn't solve the issue.

Navigation Menu

Repository navigation

When I try to use the Chinese namespace in C# #662

Environment

Details

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

When I try to use the Chinese namespace in C# #662

Description

Environment

Details

Activity

den-run-ai commented on Apr 18, 2018

charSLee013 commented on Apr 18, 2018

den-run-ai commented on Apr 18, 2018

charSLee013 commented on Apr 19, 2018

yagweb commented on Apr 19, 2018

charSLee013 commented on Apr 19, 2018

daixinwei2018 commented on May 1, 2018

den-run-ai commented on May 1, 2018

filmor commented on Aug 29, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions