Repository navigation
PyConfig.stdlib_dir does not affect sys._stdlib_dir #106718
Description
In Python 3.11, PyConfig gained a stdlib_dir attribute which is, as far as I can tell, supposed to override sys._stdlib_dir (which is used by importlib). No matter what I set it to or what else I set, however, it seems to be ignored. For example:
#include <Python.h>
#include <stdio.h>
extern wchar_t *_Py_GetStdlibDir(void);
int main(int argc, char **argv)
{
PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
PyConfig_SetString(&config, &config.stdlib_dir, L"/my/stdlib/dir");
Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
PyConfig_Clear(&config);
fprintf(stderr, "stdlib_dir = '%ls'\n", _Py_GetStdlibDir());
Py_Finalize();
}% gcc -Wall $(/python/3.12-opt/bin/python3-config --cflags) -o testprogram testprogram.c $(/python/3.12-opt/bin/python3-config --ldflags) -lpython3.12
# Look ma, no warnings.
% ./testprogram
stdlib_dir = '/python/3.12-opt/lib/python3.12'It doesn't seem to matter whether PyConfig.stdlib_dir is set to an existing directory either (although for my use-case, it must be a non-existant one; we're getting the stdlib from embedded data, and the incorrect setting of sys._stdlib_dir means importlib's FrozenImporter sets the wrong __file__ attribute on frozen/deepfrozen modules, like os.)
Activity
vstinner commentedon Jul 13, 2023
I wasn't involved in these changes. I only designed the original PyConfig structure.
Maybe @zooba or @ncoghlan know about this?
zooba commentedon Jul 13, 2023
Looking at
getpath.py, it doesn't try to readstdlib_dirout of the config when starting, and I assume never has (there are no checks later on to see whether it's already been set).It looks like the
stdlib_dirattribute was added around the same time I was working on portinggetpath.cto Python, so it may just have been a successful merge and nobody noticed that it wasn't being read? That PR took a long time to get in.I won't be able to get to this in time for 3.12.0rc1, but the change ought to be entirely in Python starting from
getpath.pywhere I linked above, and it's relatively easy to test (the test suiteexec's the source file, rather than needing a rebuild)yilei commentedon Aug 31, 2023
I'll try to send a PR for this. I'm not familiar with the code so I may need additional pointers later.
gh-106718: Treat PyConfig.stdlib_dir as highest-priority setting for âĶ
pythongh-106718: Treat PyConfig.stdlib_dir as highest-priority settinâĶ
pythongh-106718: Treat PyConfig.stdlib_dir as highest-priority settinâĶ
pythongh-106718: Treat PyConfig.stdlib_dir as highest-priority settinâĶ