Integrate media features and experiments with Gutenberg's experimental Media Editor

What problem does this address?

As we continue to add media-related AI features/experiments, we're currently targeting the media experience within the post editor and within the Media Library. We may want to consider also integrating with the experimental Gutenberg media editor as a forward-leaning media editing experience within WordPress.

What is your proposed solution?

If Gutenberg is enabled and the Media Editor experiment is activated, then perhaps we consider interjecting our media-related AI features/experiments into that experience?

@jeffpaul

@jeffpaul I've been playing around with this and the new Media Editor and have something working for the Alt Text field, however there are currently no filters or Slots/Fills in the new Media Editor, which means we have to go through DataViews with registerEntityField() to get there. This also means we can't hook into the existing field output.

I played around with a version of this where we just added a new "Generate Alt Text" button to the end of the field list, but it felt disconnected not being inline with or right next to the Alt Text input field.

So then I looked into replacing the core alt_text input with our own, wrapped in the AI Experiment. This works (see screenshot below) but I couldn't just replace the alt text field because because registerEntityField runs asynchronously from usePostFields() (which is what the Media Editor uses), and whatever field is registered last wins. Because they run async from each other we don't know which field (core or ours) would be registered first.

To get around this I'm setting a timeout of 1ms on the registerEntityField() method. This seems to be working to override the core field, but I'm not sure how stable it is given the unknown time it might take for usePostFields() to complete. In the event that it takes longer than 1ms to complete, the core field might override our field instead. The other downside is we can't retain the current field ordering, because they are registered independently from each other. Our alt_text field is being added to the end of the current Media fields because we can't hook into the current field rendering.

Here is a screenshot of what I have implemented:

All this said, I think we have a couple paths forward:

I can open the PR for what I have for testing and we can proceed with that for now.
We can open an issue (or comment on an existing one, open a PR, etc..) in the Gutenberg repository to add some filters to the Media Editor field output. Either allowing us to extend each individual field or replace them.

Option 2 is probably the ideal long-term solution, but I'm happy to push up what I have now for testing or an interim solution if we want. Just let me know how you want to proceed here. (cc @dkotter )

@andrewserong

@andrewserong any advice on forthcoming filters/slotfills for the AI plugin to be able to extend the experimental media editor?

Here's the MVP/PoC PR for the Alt Text Generation: #446 -- altered the custom Field registration a little bit to subscribe to the postType state change to attachment. This ensures the re-register of the Field only happens once.

@andrewserong

@andrewserong any advice on forthcoming filters/slotfills for the AI plugin to be able to extend the experimental media editor?

From my side of things it's a little too early at this stage to start to build in filters and slots as I'm not 100% sure of the current media editor experiment's direction. I'm working with @ramonjd to explore cropping as the next step, where we'll get a feel for how the whole thing is hooked up, and it'd be good to get that in place before adding extension points.

One thing we're going to be looking into is the idea of putting media editing into a modal instead of a separate route/state, as I think that's going to make it easier to work with updating ids for blocks, etc. But I'm planning to hack on this over the next couple of weeks and see how it all might look and feel.

Here's the MVP/PoC PR for the Alt Text Generation: #446 -- altered the custom Field registration a little bit to subscribe to the postType state change to attachment. This ensures the re-register of the Field only happens once.

For now re-registering fields is probably the pragmatic way to go about it 👍

One thing I would mention when interacting with the experiments in Gutenberg is that it's highly likely that we'll be changing things a fair bit and/or removing or updating experiments with little notice, so please don't treat anything behind the experiment as a stable API. What I meant to say is, I wouldn't want to accidentally undo any work y'all are doing with our explorations at this stage.

For the next bit of work on the experimental media editor stuff, I'm planning to use these issues to keep track:

One thing I would mention when interacting with the experiments in Gutenberg is that it's highly likely that we'll be changing things a fair bit and/or removing or updating experiments with little notice, so please don't treat anything behind the experiment as a stable API. What I meant to say is, I wouldn't want to accidentally undo any work y'all are doing with our explorations at this stage.

Totally, we're keeping any integration we build into the AI plugin as an Experiment and not a Feature for that very reason. Hopefully we can at least help drive some testing to the media experiment in Gutenberg and help with real world user feedback. 🤞🏼

Totally, we're keeping any integration we build into the AI plugin as an Experiment and not a Feature for that very reason. Hopefully we can at least help drive some testing to the media experiment in Gutenberg and help with real world user feedback. 🤞🏼

Excellent, sounds good! 🙂

@ramonjd

Just a quick update for everyone, @ramonjd and I have been making good progress on the media editor modal experiment (tasks tracked in WordPress/gutenberg#73771). That's now the main entry point to the new media editor in Gutenberg (in terms of the experiments) and it has an image cropper wired up alongside folks being able to edit metadata. You can activate it in Gutenberg trunk under the Media Editor Modal experiment. Then, it'll open up when you click the Crop icon on an image block.

For the existing Media Editor experiment, I'm going to repurpose it slightly to mean the media editor as accessed via a dedicated route. I have a WIP PR up (WordPress/gutenberg#77994) that's looking to better clarify the two media experiments.

For your purposes in the AI plugin, I believe what this means is that if you want to attempt to extend or customise things, there are now two experiments to check for:

window.__experimentalMediaEditor (this will be for the media editor accessed via a dedicated route)
window.__experimentalMediaEditorModal (this will be for the media editor in a modal, accessed via the crop button in the image block)

With cropping now wired up, the media editor(s) are much more fleshed out now. It'll be interesting to see what the right ways will be to make all this extensible, not just in terms of fields and slots, but also in how we hook in save behaviour and how plugins will expect to interact with this.

As always, happy for feedback and ideas! (Either here, or on any of the relevant Gutenberg issues, but #73771 is a good place as it's what we're working from currently).

@andrewserong

Thanks for that update, @andrewserong ! I'll make the necessary updates in the associated PR for the new window global.

Re:

With cropping now wired up, the media editor(s) are much more fleshed out now. It'll be interesting to see what the right ways will be to make all this extensible, not just in terms of fields and slots, but also in how we hook in save behaviour and how plugins will expect to interact with this.

I've actually got a PR prepped in the AI plugin to integrate with that cropping solution: #494 -- if that's merged in now I can flesh out that PR more.

I started looking at slots and interaction, and have a bunch of PRs to test out things. I've listed them under To-do > Make image editor extensible on WordPress/gutenberg#73771

A bit of a way to go yet, but the general idea is to narrow the scope for extends to interact with the image. So:

Extensions target the image surface/source state, not the crop UI itself.
Crop remains a native/core tool with first-class UI and server /edit modifier support.
Extensions can still drive cropper state through the API when it is part of their workflow.

On the last point, we have a model to programmatically apply crop operations: https://github.com/WordPress/gutenberg/blob/trunk/packages/media-editor/src/image-editor/docs/recipes.md#2-transform-pipeline-ai-agent-integration

In a previous commit there was a JSON schema in the repo specifically for agents to use as a reference, but it was removed as part of the MVP. If there's anything that you think would help let us know!

Navigation Menu

Repository navigation

Integrate media features and experiments with Gutenberg's experimental Media Editor #325

What problem does this address?

What is your proposed solution?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Integrate media features and experiments with Gutenberg's experimental Media Editor #325

Description

What problem does this address?

What is your proposed solution?

Activity

TylerB24890 commented on Apr 17, 2026

jeffpaul commented on Apr 17, 2026

TylerB24890 commented on Apr 17, 2026

andrewserong commented on Apr 19, 2026

jeffpaul commented on Apr 21, 2026

andrewserong commented on Apr 22, 2026

andrewserong commented on May 7, 2026

TylerB24890 commented on May 7, 2026

ramonjd commented on May 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions