[typing: PEP 646]: *tuple[int, int] is improperly evaluated by get_type_hints

Bug report

Unpack information of typing.GenericAlias is not transferred from string annotations to interpreted annotations. typing.Unpacked works as expected.

A fix could be to add if getattr(t, "__unpacked__", False): return next(iter(GenericAlias(t.__origin__, ev_args))) clause here:

cpython/Lib/typing.py

Lines 373 to 374 in 6492492

    
           if isinstance(t, GenericAlias): 
        
               return GenericAlias(t.__origin__, ev_args)

NOTE: This bug is in typing's internal API and I don't think there's issues in the public API (but haven't looked hard). However, this issue pops up quickly if you try to do anything wrt PEP 646 typing at runtime.

Minimal example

from typing import ForwardRef
from typing import TypeVarTuple

Ts = TypeVarTuple("Ts")
typ = ForwardRef("*Ts")._evaluate({}, {"Ts": Ts}, frozenset())
assert typ == next(iter(Ts))  # <-- PASSES AS EXPECTED

typ = ForwardRef("*tuple[int]")._evaluate({}, {}, frozenset())
assert typ == tuple[int]  # <-- PASSES BUT SHOULDN'T
assert typ == next(iter(tuple[int]))  # <-- SHOULD PASS BUT DOESN'T

Your environment

Python 3.11

Linked PRs

pythongh-101015: Fix `typing.ForwardRef` with `*tuple` unpack

It happens because of this line:

cpython/Lib/typing.py

Lines 827 to 831 in ef633e5

    
           # If we do `def f(*args: *Ts)`, then we'll have `arg = '*Ts'`. 
        
           # Unfortunately, this isn't a valid expression on its own, so we 
        
           # do the unpacking manually. 
        
           if arg[0] == '*': 
        
               arg_to_compile = f'({arg},)[0]'  # E.g. (*Ts,)[0]

Moreover, it is not ever tested 😒
If I delete this line all tests pass.

Moreover, it has different semantic for *tuple[int, str] and *Ts, as you have shown.
This happens because TypeVarTuple automatically adds Unpack here:

cpython/Lib/typing.py

Lines 1056 to 1057 in ef633e5

    
           def __iter__(self): 
        
               yield Unpack[self]

I've sent a proposal with the fix, but I am not 100% happy with it. Please, share your feedback!

Can somebody provide an example of the bug that doesn't use private APIs such as typing._eval_type or ForwardRef._evaluate?

See my PR:

def test_get_type_hints_on_unpack_args(self):
        Ts = TypeVarTuple('Ts')

        def func2(*args: '*tuple[int, str]'): pass
        self.assertEqual(gth(func2), {'args': Unpack[tuple[int, str]]})

It was {'args': tuple[int, str]} before, example from the current main:

(.venv) ~/Desktop/cpython  main ✔                                                    
» ./python.exe                                                             
Python 3.12.0a4+ (heads/issue-100942-dirty:a2babb4f58, Jan 12 2023, 01:08:31) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import *
>>> Ts = TypeVarTuple('Ts')
>>> def func2(*args: '*tuple[int, str]'): pass
... 
>>> get_type_hints(func2)
{'args': tuple[int, str]}

@sobolevn

Thanks @sobolevn, that's very helpful! Cc. @mrahtz for interest.

The bug is not unique to forward references. Here, obtaining the annotations directly via .__annotations__ gives the correct result, but going via get_type_hints gives the incorrect result. No forward references here:

>>> import typing
>>> def foo(*args: *tuple[int, int]): ...
>>> foo.__annotations__
{'args': *tuple[int, int]}
>>> typing.get_type_hints(foo)
{'args': tuple[int, int]}

Yes, that's a good catch. My PR does not solve the second problem. Fixing it!

Now it does. With even more hacks 😒

I don't have a setup to test changes so going by intuition to simplify the change: If you try only the fix I proposed in _eval_type using next(iter(...)), are there still errors? ForwardRef.__forward_value__ comes from _eval_type and if the latter returns a correctly unpacked type, it shouldn't be necessary to re-apply it or introducing is_unpack.

@hjalmarlucius

@hjalmarlucius I've managed to remove new ForwardRef flag. However, next(iter(...)) seems like a very unclear hack :)

I prefer to keep my code with __unpacked__.

gh-101015: Fix `typing.get_type_hints` with unpacked `*tuple` (PEP 64…

pythongh-101015: Fix `typing.get_type_hints` with unpacked `*tuple` (…

gh-101015: Fix `typing.get_type_hints` with unpacked `*tuple` (PEP 64…

@hjalmarlucius

Thanks @hjalmarlucius for the report, and @sobolevn for the fix!

Navigation Menu

Repository navigation

[typing: PEP 646]: `*tuple[int, int]` is improperly evaluated by `get_type_hints` #101015

Bug report

Minimal example

Your environment

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	if isinstance(t, GenericAlias):
	return GenericAlias(t.__origin__, ev_args)

	# If we do `def f(args: Ts)`, then we'll have `arg = '*Ts'`.
	# Unfortunately, this isn't a valid expression on its own, so we
	# do the unpacking manually.
	if arg[0] == '*':
	arg_to_compile = f'({arg},)[0]' # E.g. (*Ts,)[0]

[typing: PEP 646]: *tuple[int, int] is improperly evaluated by get_type_hints #101015

Description

Bug report

Minimal example

Your environment

Linked PRs

Activity

sobolevn commented on Jan 14, 2023

AlexWaygood commented on Jan 14, 2023

sobolevn commented on Jan 14, 2023

AlexWaygood commented on Jan 14, 2023

AlexWaygood commented on Jan 14, 2023

sobolevn commented on Jan 14, 2023

sobolevn commented on Jan 14, 2023

hjalmarlucius commented on Jan 14, 2023

sobolevn commented on Jan 14, 2023

AlexWaygood commented on Jan 23, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

[typing: PEP 646]: `*tuple[int, int]` is improperly evaluated by `get_type_hints` #101015