ExceptionGroup ergonomics in pytest.param(..., marks=xfail(raises=...))

Suppose you have a parameterized test, some params of which are expected to raise (e.g.) IndexError:

return pytest.param(
    ...,
    marks=pytest.mark.xfail(raises=IndexError, strict=True),
)

If your test is async though, you'll actually raise an ExceptionGroup(..., [IndexError]) (maybe with even more nesting), so the tests will still fail. Can we make this more ergonomic? What would a parametrize-aware RaisesGroup-like thing look like?

See also #11538 and #11671.

I might be seeing nails everywhere, but I would do

return pytest.param(
    ...,
    marks=pytest.mark.xfail(raises=RaisesGroup(IndexError), strict=True),
)

and afaict the only code doing the matching is

pytest/src/_pytest/skipping.py

Lines 280 to 281 in d9a2fd0

    
           if raises is not None and not isinstance(call.excinfo.value, raises): 
        
               rep.outcome = "failed"

where it would be straightforward to add

if isinstance(raises, RaisesGroup) and not raises.matches(call.excinfo.value):
  rep.outcome = "failed"
elif raises is not None [...]
  ...

This of course depends on #11671 (or a version that fully implements trio.RaisesGroup functionality).

Actually, I don't see why this is about pytest.param at all. If pytest.mark.xfail(raises=...) got support for RaisesGroup (or equivalent) it should Just Work ™️

@pytest.mark.xfail(raises=RaisesGroup(ValueError))
def test_foo():
    raise ExceptionGroup("foo", [ValueError])

#11671 has fallen behind the trio implementation, and is stalled by people wanting to have opinions on the interface design, otherwise I'd resolve this issue in it right away.

Oh this also got fixed in #13192

chore(deps): update dependency pytest to v9 (#305)

Navigation Menu

Repository navigation

`ExceptionGroup` ergonomics in `pytest.param(..., marks=xfail(raises=...))` #12504

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if raises is not None and not isinstance(call.excinfo.value, raises):
	rep.outcome = "failed"

ExceptionGroup ergonomics in pytest.param(..., marks=xfail(raises=...)) #12504

Description

Activity

jakkdl commented on Jul 16, 2024

jakkdl commented on Sep 17, 2024

jakkdl commented on Mar 13, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`ExceptionGroup` ergonomics in `pytest.param(..., marks=xfail(raises=...))` #12504