Skip to content

SystemError from calling __iter__ on a released memoryview #126341

Closed
Closed
SystemError from calling __iter__ on a released memoryview#126341
Assignees
ritvikpasham
Labels
3.12only security fixes3.13only security fixes3.14bugs and security fixeseasyinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

Description

@devdanzin
devdanzin
opened on Nov 3, 2024

Crash report

What happened?

The code below causes a SystemError on all tested Python versions in Windows and Linux. In a no-gil debug build of main, besides the expected abort, sometimes a segfault occurs.

from threading import Thread

av = memoryview(b"something")
av.release()

alive = [
    Thread(target=av.release, args=('/bin/sh')),
    Thread(target=av.__iter__, args=())
]

for obj in alive:
    try:
        print('START', obj)
        obj.start()
    except Exception:
        pass

Common abort:

Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
Python runtime state: initialized
ValueError: operation forbidden on released memoryview object

The above exception was the direct cause of the following exception:

SystemError: <method-wrapper '__iter__' of memoryview object at 0x200006a6050> returned a result with an exception set
Aborted

Rare segfault:

Exception in thread Thread-1 (release):
Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
Python runtime state: initialized
ValueError: operation forbidden on released memoryview object

The above exception was the direct cause of the following exception:

SystemError: <method-wrapper '__iter__' of memoryview object at 0x200006a6050> returned a result with an exception set

Current thread 0x00007f750b5d0640 (most recent call first):
  File "/home/danzin/projects/mycpython/Lib/threading.py", line 992 in run
  File "/home/danzin/projects/mycpython/Lib/threading.py", line 1041 in _bootstrap_inner
  File "/home/danzin/projects/mycpython/Lib/threading.py", line 1012 in _bootstrap

Thread 0x00007f750bdd5640 (most recent call first):
  File "<frozen importlib._bootstrap_external>", line 142 in _path_split
Python/traceback.c:979:18: runtime error: member access within misaligned address 0x000000000001 for type 'struct _PyInterpreterFrame', which requires 8 byte alignment
0x000000000001: note: pointer points here
<memory cannot be printed>
Segmentation fault

The backtrace for the segfault is:

0x00005555564d6e71 in _PyFrame_GetCode (f=0x7ffff6bbcca8)
    at ./Include/internal/pycore_frame.h:83
#1  PyUnstable_InterpreterFrame_GetLine (
    frame=frame@entry=0x7ffff6bbcca8) at Python/frame.c:146
#2  0x00005555565ffb1e in dump_frame (fd=fd@entry=2,
    frame=frame@entry=0x7ffff6bbcca8) at Python/traceback.c:905
#3  0x00005555565fffcc in dump_traceback (fd=fd@entry=2,
    tstate=tstate@entry=0x555557130d20,
    write_header=write_header@entry=0)
    at Python/traceback.c:974
#4  0x00005555566001e0 in _Py_DumpTracebackThreads (
    fd=fd@entry=2, interp=<optimized out>,
    interp@entry=0x555557052280 <_PyRuntime+129152>,
    current_tstate=current_tstate@entry=0x5555571351d0)
    at Python/traceback.c:1090
#5  0x00005555565888f1 in _Py_FatalError_DumpTracebacks (
    fd=fd@entry=2,
    interp=interp@entry=0x555557052280 <_PyRuntime+129152>,
    tstate=tstate@entry=0x5555571351d0)
    at Python/pylifecycle.c:2937
#6  0x000055555659b12b in fatal_error (fd=2,
    header=header@entry=1,
    prefix=prefix@entry=0x555556811220 <__func__.23> "_Py_CheckFunctionResult",
    msg=msg@entry=0x555556810c80 "a function returned a result with an exception set", status=status@entry=-1)
    at Python/pylifecycle.c:3245
#7  0x000055555659b2ca in _Py_FatalErrorFunc (
    func=func@entry=0x555556811220 <__func__.23> "_Py_CheckFunctionResult",
    msg=msg@entry=0x555556810c80 "a function returned a result with an exception set") at Python/pylifecycle.c:3286
#8  0x0000555555ee822c in _Py_CheckFunctionResult (
    tstate=tstate@entry=0x5555571351d0,result=result@entry=0x20004040130, where=where@entry=0x0)
    at Objects/call.c:65

Found using fusil by @vstinner.

CPython versions tested on:

3.11, 3.12, 3.13, CPython main branch

Operating systems tested on:

Linux, Windows

Output from running 'python -VV' on the command line:

No response

Linked PRs

Activity

  1. devdanzin
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Nov 3, 2024
  2. ZeroIntensity
    added
    3.13only security fixes
    topic-free-threading
    3.14bugs and security fixes
    on Nov 3, 2024

  3. ZeroIntensity commented on Nov 3, 2024

    @ZeroIntensity
    ZeroIntensity
    on Nov 3, 2024
    Member

    IIUC, it's undecided right now about what to do with method thread safety--most methods on objects aren't thread-safe, so you should be using a lock.

  4. devdanzin
    changed the title [-]`SystemError` from accessing a `memoryview` from two threads[/-] [+]`SystemError` from calling `__iter__` on a released `memoryview`[/+] on Nov 3, 2024

  5. devdanzin commented on Nov 3, 2024

    @devdanzin
    devdanzin
    on Nov 3, 2024
    MemberAuthor

    It seems to be independent of threads, my bad for not reducing further. The code below reliably causes the SystemError for me, but I couldn't get a segfault with it so far.

    av = memoryview(b"something")
av.release()
av.__iter__()

  6. ZeroIntensity commented on Nov 3, 2024

    @ZeroIntensity
    ZeroIntensity
    on Nov 3, 2024
    Member

    I'm assuming the segfault is a result of missing thread safety on memoryview objects, but the SystemError is definitely a bug. Thank you for the report!

  7. ZeroIntensity commented on Nov 3, 2024

    @ZeroIntensity
    ZeroIntensity
    on Nov 3, 2024
    Member

    @picnixz Considering your recent ventures with memoryview objects, would you like to take this one? If not, I can submit a PR.

  8. sobolevn commented on Nov 3, 2024

    @sobolevn
    sobolevn
    on Nov 3, 2024
    Member

    I would expect to see something like:

        CHECK_RELEASED(obj);

    as the fix.

  9. picnixz commented on Nov 3, 2024

    @picnixz
    picnixz
    on Nov 3, 2024
    Member

    @picnixz Considering your recent ventures with memoryview objects, would you like to take this one? If not, I can submit a PR.

    Please do so! I'm working on HACL* HMAC now and won't be available for the next few days. (I'll also fix a simple curses issue in a few hours (DONE)).

  10. picnixz
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Nov 3, 2024

  11. ZeroIntensity commented on Nov 3, 2024

    @ZeroIntensity
    ZeroIntensity
    on Nov 3, 2024
    Member

    Yeah, @sobolevn is right. I'll let someone newer have a chance to contribute since it's a super easy fix :)

    index d4672e8198..25634f997a 100644
--- a/Objects/memoryobject.c
+++ b/Objects/memoryobject.c
@@ -3356,6 +3356,7 @@ memory_iter(PyObject *seq)
         PyErr_BadInternalCall();
         return NULL;
     }
+    CHECK_RELEASED(seq);
     PyMemoryViewObject *obj = (PyMemoryViewObject *)seq;
     int ndims = obj->view.ndim;
     if (ndims == 0) {
  12. ZeroIntensity
    added
    easy
    and removed
    topic-free-threading
    on Nov 3, 2024

  13. 6 remaining items

  14. picnixz
    added
    3.12only security fixes
    on Nov 9, 2024

  15. ritvikpasham commented on Nov 9, 2024

    @ritvikpasham
    ritvikpasham
    on Nov 9, 2024
    Contributor

    Yeah I am!

  16. vstinner commented on Nov 12, 2024

    @vstinner
    vstinner
    on Nov 12, 2024
    Member

    @ritvikpasham:

    Yeah I am!

    Do you need help to write the fix? The fix was provided as a patch in #126341 (comment).

  17. ritvikpasham commented on Nov 12, 2024

    @ritvikpasham
    ritvikpasham
    on Nov 12, 2024
    Contributor

    Sorry for the delay. I'll finish it in the next hour.

  18. sobolevn
    added a commit that references this issue on Nov 13, 2024

    gh-126341: add release check to `__iter__` method of `memoryview` (#1…

    a12690e
  19. miss-islington
    added 2 commits that reference this issue on Nov 13, 2024

    pythongh-126341: add release check to `__iter__` method of `memoryvie…

    a741431

    pythongh-126341: add release check to `__iter__` method of `memoryvie…

    1af10de
  20. Eclips4
    added 2 commits that reference this issue on Nov 13, 2024

    [3.12] gh-126341: add release check to `__iter__` method of `memoryvi…

    0bdcc84

    [3.13] gh-126341: add release check to `__iter__` method of `memoryvi…

    3fe062c
  21. Eclips4
    closed this as completedon Nov 13, 2024
  22. picnixz
    added a commit that references this issue on Dec 8, 2024

    pythongh-126341: add release check to `__iter__` method of `memoryvie…

    0c6ec4a
  23. ebonnal
    added a commit that references this issue on Jan 12, 2025

    pythongh-126341: add release check to `__iter__` method of `memoryvie…

    6adefa1
  24. serhiy-storchaka
    added this to Struct, memoryview and array issues 🏗️on Jul 13, 2026
  25. github-project-automation
    moved this to Todo in Struct, memoryview and array issues 🏗️on Jul 13, 2026
  26. github-project-automation
    moved this from Todo to Done in Struct, memoryview and array issues 🏗️on Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixes3.13only security fixes3.14bugs and security fixeseasyinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dump

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions