Skip to content

Commit ca682bb

Browse files
vstinnervstinner
committed
WIP: [C API] Add PyTupleWriter API
1 parent e3262d9 commit ca682bb

11 files changed

Lines changed: 671 additions & 128 deletions

File tree

‎Doc/c-api/tuple.rst‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,9 @@ Tuple Objects
164164
.. versionchanged:: next
165165
The function now fails if the tuple hash value was already computed.
166166
167+
.. soft-deprecated:: next
168+
Use the :ref:`PyTupleWriter API <pytuplewriter>` instead.
169+
167170
168171
.. _struct-sequence-objects:
169172
@@ -293,3 +296,101 @@ type.
293296
294297
.. versionchanged:: 3.13
295298
Now implemented as an alias to :c:func:`PyStructSequence_SetItem`.
299+
300+
301+
.. _pytuplewriter:
302+
303+
PyTupleWriter
304+
-------------
305+
306+
The ``PyTupleWriter`` API can be used to create a Python :class:`tuple` object
307+
when the final size is not known in advance.
308+
309+
.. versionadded:: next
310+
311+
.. c:type:: PyTupleWriter
312+
313+
A tuple writer instance.
314+
315+
The instance must be destroyed by :c:func:`PyTupleWriter_Finish` on
316+
success, or :c:func:`PyTupleWriter_Discard` on error.
317+
318+
The API is **not thread safe**. To share a writer with multiple threads, a
319+
critical section or a lock is needed.
320+
321+
.. c:function:: PyTupleWriter* PyTupleWriter_Create(Py_ssize_t size)
322+
323+
Create a tuple writer instance.
324+
325+
*size* must be greater than or equal to ``0``.
326+
327+
If *size* is greater than ``0``, allocate an internal buffer of *size* items
328+
and all items must be set with :c:func:`PyTupleWriter_SetItem` or
329+
:c:func:`PyTupleWriter_SetItems`.
330+
331+
Set an exception and return ``NULL`` on error.
332+
333+
.. c:function:: PyObject* PyTupleWriter_Finish(PyTupleWriter *writer)
334+
335+
Return the final Python :class:`str` object and destroy the writer instance.
336+
337+
Set an exception and return ``NULL`` on error.
338+
339+
The writer instance is invalid after this call.
340+
341+
.. c:function:: PyObject* PyTupleWriter_FinishWithSize(PyTupleWriter *writer, Py_ssize_t final_size)
342+
343+
Similar to :c:func:`PyTupleWriter_FinishWithSize`, but first truncates
344+
the tuple to *final_size* items.
345+
346+
On success, return a new tuple object.
347+
On error, set an exception and return ``NULL``.
348+
349+
.. c:function:: void PyTupleWriter_Discard(PyTupleWriter *writer)
350+
351+
Discard the internal Tuple buffer and destroy the writer instance.
352+
353+
If *writer* is ``NULL``, no operation is performed.
354+
355+
The writer instance is invalid after this call.
356+
357+
.. c:function:: void PyTupleWriter_Discard(PyTupleWriter *writer)
358+
359+
Discard the internal tuple buffer and destroy the writer instance.
360+
361+
If *writer* is ``NULL``, no operation is performed.
362+
363+
The writer instance is invalid after this call.
364+
365+
.. c:function:: Py_ssize_t PyTupleWriter_GetSize(PyTupleWriter *writer)
366+
367+
Get the writer size.
368+
369+
The function cannot fail.
370+
371+
.. c:function:: int PyTupleWriter_SetItem(PyTupleWriter *writer, Py_ssize_t index, PyObject *item)
372+
373+
Set an item of a tuple writer start at position *index*.
374+
Copy *item* reference.
375+
376+
An item must be set exactly once.
377+
378+
On success, return ``0``.
379+
On error, set an exception and return ``-1``.
380+
381+
.. c:function:: int PyTupleWriter_SetItems(PyTupleWriter *writer, Py_ssize_t index, PyObject *const *items, Py_ssize_t nitem)
382+
383+
Set *nitem* items of a tuple writer start at position *index*.
384+
Copy *items* references.
385+
386+
Items must be set exactly once.
387+
388+
On success, return ``0``.
389+
On error, set an exception and return ``-1``.
390+
391+
.. c:function:: int PyTupleWriter_Resize(PyTupleWriter **writer, Py_ssize_t new_size)
392+
393+
Resize the tuple writer to *new_size* items.
394+
395+
On success, return ``0``.
396+
On error, set an exception and return ``-1``.

‎Doc/whatsnew/3.16.rst‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1398,6 +1398,20 @@ New features
13981398
API.
13991399
(Contributed by Nathan Goldbaum and Victor Stinner in :gh:`158842`.)
14001400

1401+
* Add a new :ref:`PyTupleWriter API <pytuplewriter>` to create a :class:`tuple`
1402+
when the final size is not known in advance. Add functions:
1403+
1404+
* :c:func:`PyTupleWriter_Create`
1405+
* :c:func:`PyTupleWriter_Finish`
1406+
* :c:func:`PyTupleWriter_FinishWithSize`
1407+
* :c:func:`PyTupleWriter_Discard`
1408+
* :c:func:`PyTupleWriter_GetSize`
1409+
* :c:func:`PyTupleWriter_SetItem`
1410+
* :c:func:`PyTupleWriter_SetItems`
1411+
* :c:func:`PyTupleWriter_Resize`
1412+
1413+
(Contributed by Victor Stinner in :gh:`161694`.)
1414+
14011415
* Add the :ref:`PyUnicodeWriter API <pyunicodewriter>` to the limited C API.
14021416
(Contributed by Victor Stinner in :gh:`161172`.)
14031417

@@ -1450,6 +1464,10 @@ Deprecated C APIs
14501464
:c:func:`PyModule_GetFilenameObject` instead is still recommended.
14511465
(Contributed by Victor Stinner in :gh:`154757`.)
14521466

1467+
* Soft deprecate the :c:func:`_PyTuple_Resize` function: use the new
1468+
:ref:`PyTupleWriter API <pytuplewriter>` instead.
1469+
(Contributed by Victor Stinner in :gh:`161694`.)
1470+
14531471
* Soft deprecate functions modifying Unicode strings:
14541472
:c:func:`PyUnicode_New`, :c:func:`PyUnicode_CopyCharacters`,
14551473
:c:func:`PyUnicode_Fill`, :c:func:`PyUnicode_Resize`,

‎Include/cpython/tupleobject.h‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,12 @@ static inline Py_ssize_t PyTuple_GET_SIZE(PyObject *op) {
3030

3131
/* Function *only* to be used to fill in brand new tuples */
3232
static inline void
33-
PyTuple_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) {
33+
PyTuple_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value)
34+
{
35+
// For backward compatibility, accept mutating a tuple even if
36+
// _PyObject_IsUniquelyReferenced(op) is false and/or tuple->ob_hash != -1.
37+
// See for example _PyTuple_Recycle(). Accept also setting an item which
38+
// was already set: the caller must Py_DECREF() the old value in this case.
3439
PyTupleObject *tuple = _PyTuple_CAST(op);
3540
assert(0 <= index);
3641
assert(index < Py_SIZE(tuple));
@@ -42,3 +47,29 @@ PyTuple_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) {
4247
PyAPI_FUNC(PyObject*) PyTuple_FromArray(
4348
PyObject *const *array,
4449
Py_ssize_t size);
50+
51+
52+
/* --- PyTupleWriter API -------------------------------------------------- */
53+
54+
struct PyTupleWriter;
55+
typedef struct PyTupleWriter PyTupleWriter;
56+
57+
PyAPI_FUNC(PyTupleWriter*) PyTupleWriter_Create(Py_ssize_t size);
58+
PyAPI_FUNC(PyObject*) PyTupleWriter_Finish(PyTupleWriter *writer);
59+
PyAPI_FUNC(PyObject*) PyTupleWriter_FinishWithSize(
60+
PyTupleWriter *writer,
61+
Py_ssize_t final_size);
62+
PyAPI_FUNC(void) PyTupleWriter_Discard(PyTupleWriter *writer);
63+
PyAPI_FUNC(Py_ssize_t) PyTupleWriter_GetSize(PyTupleWriter *writer);
64+
PyAPI_FUNC(int) PyTupleWriter_SetItem(
65+
PyTupleWriter *writer,
66+
Py_ssize_t index,
67+
PyObject *item);
68+
PyAPI_FUNC(int) PyTupleWriter_SetItems(
69+
PyTupleWriter *writer,
70+
Py_ssize_t index,
71+
PyObject *const *items,
72+
Py_ssize_t nitem);
73+
PyAPI_FUNC(int) PyTupleWriter_Resize(
74+
PyTupleWriter **writer,
75+
Py_ssize_t new_size);

‎Include/internal/pycore_tuple.h‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,84 @@ _PyTuple_Recycle(PyObject *op)
7676
#endif
7777
#define _PyTuple_HASH_EMPTY (_PyTuple_HASH_XXPRIME_5 + (_PyTuple_HASH_XXPRIME_5 ^ 3527539UL))
7878

79+
80+
#ifndef NDEBUG
81+
// Check that a tuple can be mutated in-place.
82+
// Usage: assert(_PyTuple_CanMutate(v));
83+
static int
84+
_PyTuple_CanMutate(PyTupleObject *tuple)
85+
{
86+
assert(PyTuple_Check(tuple));
87+
88+
// Do not check _PyObject_IsUniquelyReferenced(): the function is called
89+
// by _PyTupleWriter_CheckConsistency() and a tuple writer can be used by
90+
// multiple threads using external locking.
91+
assert(Py_REFCNT(tuple) == 1);
92+
93+
// The empty tuple must not be modified
94+
assert(1 <= Py_SIZE(tuple));
95+
assert(tuple != &_Py_SINGLETON(tuple_empty));
96+
97+
// The hash value must not be cached yet. Otherwise, it means that the
98+
// tuple object was already used in Python somehow (ex: as a dictionary
99+
// key).
100+
assert(tuple->ob_hash == -1);
101+
102+
return 1;
103+
}
104+
#endif
105+
106+
107+
/* --- PyTupleWriter API -------------------------------------------------- */
108+
109+
#ifndef NDEBUG
110+
static inline int
111+
_PyTupleWriter_CheckConsistency(PyTupleWriter *writer)
112+
{
113+
assert(writer != NULL);
114+
assert(PyTuple_CheckExact(_PyObject_CAST(writer)));
115+
PyTupleObject *tuple = (PyTupleObject*)writer;
116+
assert(0 <= Py_SIZE(tuple));
117+
if (Py_SIZE(tuple) != 0) {
118+
// Do not use _PyObject_IsUniquelyReferenced(): the caller can have
119+
// its own lock to prevent a writer from being used by two threads
120+
// at the same time.
121+
assert(Py_REFCNT(tuple) == 1);
122+
123+
assert(!_PyObject_GC_IS_TRACKED(tuple));
124+
}
125+
else {
126+
assert(tuple == &_Py_SINGLETON(tuple_empty));
127+
}
128+
return 1;
129+
}
130+
#endif
131+
132+
133+
static inline void
134+
_PyTupleWriter_SET_ITEM_STEAL(PyTupleWriter *writer,
135+
Py_ssize_t index, PyObject *item)
136+
{
137+
assert(_PyTupleWriter_CheckConsistency(writer));
138+
PyTupleObject *tuple = _PyTuple_CAST(writer);
139+
140+
assert(_PyTuple_CanMutate(tuple));
141+
assert(0 <= index && index < Py_SIZE(tuple));
142+
assert(tuple->ob_item[index] == NULL);
143+
assert(item != NULL);
144+
145+
tuple->ob_item[index] = item;
146+
}
147+
148+
149+
static inline void
150+
_PyTupleWriter_SET_ITEM(PyTupleWriter *writer,
151+
Py_ssize_t index, PyObject *item)
152+
{
153+
assert(item != NULL);
154+
return _PyTupleWriter_SET_ITEM_STEAL(writer, index, Py_NewRef(item));
155+
}
156+
79157
#ifdef __cplusplus
80158
}
81159
#endif

‎Misc/NEWS.d/next/C_API/2026-11-09-18-47-29.gh-issue-161694.D2OxVy.rst‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Add a new :ref:`PyTupleWriter API <pytuplewriter>` to create a
2+
:class:`tuple` when the final size is not known in advance. Add functions:
3+
4+
* :c:func:`PyTupleWriter_Create`
5+
* :c:func:`PyTupleWriter_Finish`
6+
* :c:func:`PyTupleWriter_FinishWithSize`
7+
* :c:func:`PyTupleWriter_Discard`
8+
* :c:func:`PyTupleWriter_GetSize`
9+
* :c:func:`PyTupleWriter_SetItem`
10+
* :c:func:`PyTupleWriter_SetItems`
11+
* :c:func:`PyTupleWriter_Resize`
12+
13+
Patch by Victor Stinner.

‎Misc/NEWS.d/next/C_API/2026-11-09-18-49-27.gh-issue-161694.jB9pTm.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Soft deprecate the :c:func:`_PyTuple_Resize` function: use the new
2+
:ref:`PyTupleWriter API <pytuplewriter>` instead. Patch by Victor Stinner.

‎Modules/_tkinter.c‎

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ Copyright (C) 1994 Steen Lumholt.
3131
#endif
3232

3333
#include "pycore_long.h" // _PyLong_IsNegative()
34+
#include "pycore_tuple.h" // _PyTupleWriter_SET_ITEM()
3435
#include "pycore_unicodeobject.h" // _PyUnicode_AsUTF8String
3536

3637
#include <signal.h> // SIGINT
@@ -3401,7 +3402,7 @@ Tkapp_Dealloc(PyObject *op)
34013402
/**** Tkinter Module ****/
34023403

34033404
typedef struct {
3404-
PyObject* tuple;
3405+
PyTupleWriter *writer;
34053406
Py_ssize_t size; /* current size */
34063407
Py_ssize_t maxsize; /* allocated size */
34073408
} FlattenContext;
@@ -3419,7 +3420,7 @@ _bump(FlattenContext* context, Py_ssize_t size)
34193420

34203421
context->maxsize = maxsize;
34213422

3422-
return _PyTuple_Resize(&context->tuple, maxsize) >= 0;
3423+
return PyTupleWriter_Resize(&context->writer, maxsize) >= 0;
34233424
}
34243425

34253426
static int
@@ -3449,8 +3450,7 @@ _flatten1(FlattenContext* context, PyObject* item, int depth)
34493450
if (context->size + 1 > context->maxsize &&
34503451
!_bump(context, 1))
34513452
return 0;
3452-
PyTuple_SET_ITEM(context->tuple,
3453-
context->size++, Py_NewRef(o));
3453+
_PyTupleWriter_SET_ITEM(context->writer, context->size++, o);
34543454
}
34553455
}
34563456
} else {
@@ -3480,21 +3480,18 @@ _tkinter__flatten(PyObject *module, PyObject *item)
34803480
if (context.maxsize == 0)
34813481
return PyTuple_New(0);
34823482

3483-
context.tuple = PyTuple_New(context.maxsize);
3484-
if (!context.tuple)
3483+
context.writer = PyTupleWriter_Create(context.maxsize);
3484+
if (!context.writer) {
34853485
return NULL;
3486-
3486+
}
34873487
context.size = 0;
34883488

34893489
if (!_flatten1(&context, item, 0)) {
3490-
Py_XDECREF(context.tuple);
3490+
PyTupleWriter_Discard(context.writer);
34913491
return NULL;
34923492
}
34933493

3494-
if (_PyTuple_Resize(&context.tuple, context.size))
3495-
return NULL;
3496-
3497-
return context.tuple;
3494+
return PyTupleWriter_FinishWithSize(context.writer, context.size);
34983495
}
34993496

35003497
/*[clinic input]

0 commit comments

Comments
 (0)