Repository navigation
media-http, gateway, vendors, ui: add Gemini image clients - #43
Merged
Merged
Conversation
A client could ask for images only in OpenAI's Images API: a Gemini client calling `generateContent` with `gpt-image-2` was taken for a chat and failed, and an OpenAI Images client could not reach Gemini's image models, as Gemini's OpenAI-compatible images endpoint serves only Imagen, which its API no longer lists. Thus, add the `gemini-images` format, Gemini's image models through `generateContent`, to clients and providers, and declare it on the `gemini` vendor in place of `openai-images`. A call is served at `gemini-generate`'s paths: it is an image's if routing sends its model, of the image kind, to a provider that cannot take it as `generateContent`, else a chat's.
github-actions
Bot
added
A-plugins
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A client could ask for images only in OpenAI's Images API: a Gemini client calling
generateContentwithgpt-image-2was taken for a chat and failed, and an OpenAI Images client could not reach Gemini's image models, as Gemini's OpenAI-compatible images endpoint serves only Imagen, which its API no longer lists. Thus, add thegemini-imagesformat: Gemini's image models throughgenerateContent, of the image kind, to clients and providers, and declare it on thegeminivendor in place ofopenai-images.To Gemini, a call is one user turn of the images to edit and the prompt; a size is the nearest aspect ratio Gemini draws, with
2Kor4Kabove 1K; a mask is refused, as Gemini edits by prompt alone; an answer with no image is refused, saying why. A client of it is one user turn of text and images, itsimageConfigread as a size, answered with one candidate of the images and the usage, whole or as a stream. A call is served atgemini-generate's paths: it is an image's if routing sends its model, of the image kind, to a provider that cannot take it asgenerateContent, else a chat's, so Gemini's own image models are asked as they were, keeping a conversation and its text. A format that claims some of its endpoints' requests is registered before the rest, once the routes are known. Gemini's image models (-image,nano-banana) are listed as image models, and the playground's image tab speaks Gemini Images too.Checked live through the gateway: an OpenAI Images client generates and edits with
gemini-3.1-flash-image; Google'sgoogle-genaiSDK generates withgpt-image-2, whole and streamed, edits withgpt-image-1-mini, and still asksgemini-3.1-flash-imageas a chat; a history togpt-image-2is refused, 400.docs/, or the vendor tables)