test_lock_two_threads flaky in test_capi

Bug report

The lock may have the _Py_HAS_PARKED bit set so that m->_bits == 3 instead of 1. This can happen due to collisions in the parking lot hash table -- that's rare so we don't see it frequently. I think the possible collision here is if the PyEvent that the main thread is waiting on uses the same bucket as the PyMutex.

We should weaken the assertion in the test case.

See in https://buildbot.python.org/all/#/builders/1609/builds/2878/steps/6/logs/stdio:

test_lock_two_threads (test.test_capi.test_misc.Test_PyLock.test_lock_two_threads) ... python: ./Modules/_testinternalcapi/test_lock.c:60: lock_thread: Assertion `m->_bits == 1' failed.
Fatal Python error: Aborted


Current thread's C stack trace (most recent call first):
  Binary file "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/python", at _Py_DumpStack+0x31 [0x75b0f6]
  Binary file "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/python" [0x770ae8]
  Binary file "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/python" [0x770bd5]
  Binary file "/lib64/libc.so.6", at +0x3ebf0 [0x7f3a97a3ebf0]
  Binary file "/lib64/libc.so.6", at +0x8c1ec [0x7f3a97a8c1ec]
  Binary file "/lib64/libc.so.6", at raise+0x16 [0x7f3a97a3eb46]
  Binary file "/lib64/libc.so.6", at abort+0xd3 [0x7f3a97a28833]
  Binary file "/lib64/libc.so.6", at +0x2875b [0x7f3a97a2875b]
  Binary file "/lib64/libc.so.6", at +0x37886 [0x7f3a97a37886]
  Binary file "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/build/lib.linux-x86_64-3.15/_testinternalcapi.cpython-315td-x86_64-linux-gnu.so", at +0xe2cf [0x7f3a970612cf]
  Binary file "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/python" [0x756d51]
  Binary file "/lib64/libc.so.6", at +0x8a4aa [0x7f3a97a8a4aa]
  Binary file "/lib64/libc.so.6", at +0x10f510 [0x7f3a97b0f510]

cpython/Modules/_testinternalcapi/test_lock.c

Line 60 in cb39410

assert(m->_bits == 1);

cpython/Modules/_testinternalcapi/test_lock.c

Lines 52 to 98 in cb39410

    
           static void 
        
           lock_thread(void *arg) 
        
           { 
        
               struct test_lock2_data *test_data = arg; 
        
               PyMutex *m = &test_data->m; 
        
               _Py_atomic_store_int(&test_data->started, 1); 
        
               PyMutex_Lock(m); 
        
               assert(m->_bits == 1); 
        
               PyMutex_Unlock(m); 
        
               assert(m->_bits == 0); 
        
               _PyEvent_Notify(&test_data->done); 
        
           } 
        
           static PyObject * 
        
           test_lock_two_threads(PyObject *self, PyObject *obj) 
        
           { 
        
               // lock attempt by two threads 
        
               struct test_lock2_data test_data; 
        
               memset(&test_data, 0, sizeof(test_data)); 
        
               PyMutex_Lock(&test_data.m); 
        
               assert(test_data.m._bits == 1); 
        
               PyThread_start_new_thread(lock_thread, &test_data); 
        
               // wait up to two seconds for the lock_thread to attempt to lock "m" 
        
               int iters = 0; 
        
               uint8_t v; 
        
               do { 
        
                   pysleep(10);  // allow some time for the other thread to try to lock 
        
                   v = _Py_atomic_load_uint8_relaxed(&test_data.m._bits); 
        
                   assert(v == 1 || v == 3); 
        
                   iters++; 
        
               } while (v != 3 && iters < 200); 
        
               // both the "locked" and the "has parked" bits should be set 
        
               assert(test_data.m._bits == 3); 
        
               PyMutex_Unlock(&test_data.m); 
        
               PyEvent_Wait(&test_data.done); 
        
               assert(test_data.m._bits == 0); 
        
               Py_RETURN_NONE; 
        
           }

Linked PRs

pythongh-135641: Fix flaky test_lock_two_threads test case

gh-135641: Fix flaky `test_capi.test_lock_two_threads` test case (gh-…

pythongh-135641: Fix flaky `test_capi.test_lock_two_threads` test case (

[3.13] gh-135641: Fix flaky `test_capi.test_lock_two_threads` test ca…

[3.14] gh-135641: Fix flaky `test_capi.test_lock_two_threads` test ca…

pythongh-135641: Fix flaky `test_capi.test_lock_two_threads` test case (

Navigation Menu

Repository navigation

`test_lock_two_threads` flaky in `test_capi` #135641

Bug report

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	static void
	lock_thread(void *arg)
	{
	struct test_lock2_data *test_data = arg;
	PyMutex *m = &test_data->m;
	_Py_atomic_store_int(&test_data->started, 1);

	PyMutex_Lock(m);
	assert(m->_bits == 1);

	PyMutex_Unlock(m);
	assert(m->_bits == 0);

	_PyEvent_Notify(&test_data->done);
	}

	static PyObject *
	test_lock_two_threads(PyObject self, PyObject obj)
	{
	// lock attempt by two threads
	struct test_lock2_data test_data;
	memset(&test_data, 0, sizeof(test_data));

	PyMutex_Lock(&test_data.m);
	assert(test_data.m._bits == 1);

	PyThread_start_new_thread(lock_thread, &test_data);

	// wait up to two seconds for the lock_thread to attempt to lock "m"
	int iters = 0;
	uint8_t v;
	do {
	pysleep(10); // allow some time for the other thread to try to lock
	v = _Py_atomic_load_uint8_relaxed(&test_data.m._bits);
	assert(v == 1 \|\| v == 3);
	iters++;
	} while (v != 3 && iters < 200);

	// both the "locked" and the "has parked" bits should be set
	assert(test_data.m._bits == 3);

	PyMutex_Unlock(&test_data.m);
	PyEvent_Wait(&test_data.done);
	assert(test_data.m._bits == 0);

	Py_RETURN_NONE;
	}

test_lock_two_threads flaky in test_capi #135641

Description

Bug report

Linked PRs

Activity

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`test_lock_two_threads` flaky in `test_capi` #135641