Repository navigation
Regression in 3.14: setting __dict__ on custom type broken (mypyc, pybind11) #133912
Description
Bug report
Bug description:
The work in #115776 by @markshannon broke mypyc and pybind11's tests related to pickling and setting __dict__ on custom types. Specially, #117750 introduced the regression for pybind11, I tested all the released for 3.14 then bisected between the commit after the fork point and 3.14.0a1 to find the commit in the PR above introduced the issue. More types are now inlined, but this breaks dict setting.
- [mypyc] Pickle test failure with Python 3.14.0a1 mypy#17973
- Pickle test failure with Python 3.14.0a5+ mypyc/mypyc#1091
- fix: support Python 3.14 pybind/pybind11#5646
I've prepared a MWE without pybind11 or mypyc:
src/main.c
#define PY_SSIZE_T_CLEAN
#include <Python.h>
typedef struct {
PyObject_VAR_HEAD
} ManagedDictObject;
int ManagedDict_traverse(PyObject *self, visitproc visit, void *arg) {
PyObject_VisitManagedDict(self, visit, arg);
Py_VISIT(Py_TYPE(self));
return 0;
}
int ManagedDict_clear(PyObject *self) {
PyObject_ClearManagedDict(self);
return 0;
}
static PyGetSetDef ManagedDict_getset[] = {
{"__dict__", PyObject_GenericGetDict, PyObject_GenericSetDict, NULL, NULL},
{NULL, NULL, NULL, NULL, NULL},
};
static PyType_Slot ManagedDict_slots[] = {
{Py_tp_new, (void *)PyType_GenericNew},
{Py_tp_getset, (void *)ManagedDict_getset},
{Py_tp_traverse, (void *)ManagedDict_traverse},
{Py_tp_clear, (void *)ManagedDict_clear},
{0}
};
static PyType_Spec ManagedDict_spec = {
"manageddictbug.ManagedDict",
sizeof(ManagedDictObject),
0, // itemsize
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_HEAPTYPE | Py_TPFLAGS_HAVE_GC,
ManagedDict_slots
};
static PyModuleDef manageddictbugmodule = {
PyModuleDef_HEAD_INIT,
"manageddictbug",
NULL,
-1,
NULL,
};
PyMODINIT_FUNC
PyInit_manageddictbug(void) {
PyObject *m = PyModule_Create(&manageddictbugmodule);
if (m == NULL)
return NULL;
PyObject *ManagedDictType = PyType_FromSpec(&ManagedDict_spec);
if (ManagedDictType == NULL) {
Py_DECREF(m);
return NULL;
}
if (PyModule_AddObject(m, "ManagedDict", ManagedDictType) < 0) {
Py_DECREF(ManagedDictType);
Py_DECREF(m);
return NULL;
}
return m;
}pyproject.toml
[build-system]
requires = ["scikit-build-core"]
build-backend = "scikit_build_core.build"
[project]
name = "example-broken"
version = "0.1.0"CMakeLists.txt
cmake_minimum_required(VERSION 3.15...4.0)
project(${SKBUILD_PROJECT_NAME} LANGUAGES C)
find_package(Python REQUIRED COMPONENTS Development.Module)
python_add_library(manageddictbug MODULE WITH_SOABI src/main.c)
install(TARGETS manageddictbug DESTINATION .)example.py
import manageddictbug
obj = manageddictbug.ManagedDict()
obj.foo = 42
print(obj.foo)
print(obj.__dict__)
obj.__dict__ = {"bar": 3}
print(obj.__dict__)
print(obj.bar)$ uv venv -p 3.13 -q && uv pip install . -q && .venv/bin/python example.py
42
{'foo': 42}
{'bar': 3}
3
$ uv venv -p ~/git/software/pybind11/.venv/bin/python3.14 -q && uv pip install . -q && .venv/bin/python example.py
42
{'foo': 42}
{'bar': 3}
Traceback (most recent call last):
File "/Users/henryschreiner/git/scikit-build-proj/example_broken/example.py", line 13, in <module>
print(obj.bar)
^^^^^^^
AttributeError: 'manageddictbug.ManagedDict' object has no attribute 'bar'There is custom code in PyObject_GenericSetDict that is supposed to be handling the inline case, but maybe it wasn't hit before and is faulty?
CPython versions tested on:
3.14.0b1 (and back to 3.14.0a1)
Operating systems tested on:
macOS (and Linux in CI too; Windows CI broken due to https://gitlab.kitware.com/cmake/cmake/-/issues/26926, which I haven't opened a CPython issue for yet).
Activity
encukou commentedon May 12, 2025
Marking as a potential release blocker.
encukou commentedon May 19, 2025
@markshannon, do you want to investigate, or should I?
hugovk commentedon May 26, 2025
I'm not planning on blocking beta 2 for this, but let's try and resolve it for beta 3 -- 2025-06-17, three weeks from now.
@markshannon Please can you look into it?
markshannon commentedon May 26, 2025
Yes. Thanks for the reminder
GH-133912: Fix `PyObject_GenericSetDict` to handle inline values (GH-…
pythonGH-133912: Fix `PyObject_GenericSetDict` to handle inline values (
cdce8p commentedon May 28, 2025
Can confirm #134725 fixes the mypyc test case as well. Thanks @henryiii for providing the independent reproducer and @markshannon for the fix! For mypy / mypyc this was the last test failure for 3.14. Looking forward to
3.14.0b3🚀[3.14] GH-133912: Fix `PyObject_GenericSetDict` to handle inline valu…
itamaro commentedon May 31, 2025
Looks like this is done, thanks everyone!
Closing, but please reopen if there's anything left to do here.
pythonGH-133912: Fix `PyObject_GenericSetDict` to handle inline values (
pythonGH-133912: Fix `PyObject_GenericSetDict` to handle inline values (
pythonGH-133912: Fix `PyObject_GenericSetDict` to handle inline values (