Skip to content

[M.E.AI.Evaluation.Reporting] Evaluation report re-design - #7609

Merged
peterwald merged 24 commits into
dotnet:mainfrom
grafanaKibana:ai-eval-report-redesign
Jul 29, 2026
Merged

peterwald merged 24 commits into
dotnet:mainfrom
grafanaKibana:ai-eval-report-redesign

Conversation

@grafanaKibana

@grafanaKibana grafanaKibana commented Jul 4, 2026 •

Copy link
Copy Markdown
Contributor

Closes #7593.

Summary

Redesigns the generated AI Evaluation HTML report as a shared Fluent UI app used by both the standalone report and the Azure DevOps extension. The serialized Dataset, ScenarioRunResult, and EvaluationMetric contracts are unchanged.

The previous single-page report is replaced by four focused views:

  • Overview — pass/fail KPIs, largest raw deltas, needs-attention metrics, and pass rate by scenario group.
  • Cases — searchable and filterable case list with inline transcripts, metrics, diagnostics, pagination, and new-case indicators.
  • History — per-metric run trends with mean, median, spread, and chronological deltas.
  • Comparison — side-by-side execution comparison with explicit added, removed, shared, and unavailable metric states.

What changed

  • Reorganized report components by feature and removed superseded Fluent UI v8 code.
  • Added responsive light/dark styling, deterministic development data, and Markdown/GFM rendering.
  • Corrected execution ordering, scenario aggregation, missing-metric handling, and delta presentation without fabricating values.
  • Improved keyboard navigation, ARIA relationships, announcements, focus behavior, and Azure DevOps resize/root lifecycle handling.
  • Added a Vitest and Testing Library suite covering model invariants, views, accessibility, routing, and host behavior.

Verification

  • npm test -- --no-file-parallelism — 216/216 tests across 31 committed test files.
  • Forced TypeScript build — passed.
  • ESLint — 0 errors; one Fast Refresh warning in the Azure DevOps entry point.
  • Standalone report build — passed.
  • Azure DevOps typecheck and build — passed.
  • Light/dark, responsive, keyboard, and shared-host behavior were exercised.

Risk

The primary risk is visual regression because both hosts share the same components. No public API or serialized data-shape changes are introduced.

Microsoft Reviewers: Open in CodeFlow

grafanaKibana and others added 13 commits July 1, 2026 14:31
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķified comparisons

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķed unused styles, refactored context and metric formatting

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
@grafanaKibana

Copy link
Copy Markdown
Contributor Author

@dotnet-policy-service agree

â€Ķpendency

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķrelated logic

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķlity logic. Added more tests to test accessibility.

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķlity fixes, refactoring

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
â€Ķon abstractions consistency

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
- Fixed Summary.ts — collision & duplicate handling
- Removed orphaned @fluentui/react v8
- Fixed yellow shade in light theme
- Deleted more dead code

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
@grafanaKibana
grafanaKibana marked this pull request as draft July 22, 2026 11:55
grafanaKibana and others added 3 commits July 22, 2026 14:56
â€Ķting ones

Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
Signed-off-by: nikitareshetnik <reshetnik.nikita@gmail.com>
@grafanaKibana
grafanaKibana marked this pull request as ready for review July 26, 2026 12:46

@peterwald peterwald left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the contribution.

@peterwald
peterwald merged commit 5207770 into dotnet:main Jul 29, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 29, 2026
@grafanaKibana grafanaKibana changed the title [Microsoft.Extensions.AI.Evaluation.Reporting] Evaluation report redesign [M.E.AI.Evaluation.Reporting] Evaluation report re-design Sep 4, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[AI Evaluation] Redesign the generated HTML report

2 participants