Support to access model gardens using endpoints: rawPredict or streamRawPredict

[cmoulliard]

Description

The Vertex AI documentation describes that we can select one of the different models: open, partner, gemini-family and access them using curl, python and probably this Java SDK (as documented here) but when we use the following code

        Client client = null;
        try {
            GoogleCredentials creds = GoogleCredentials.getApplicationDefault();
            creds.refreshIfExpired();

            client = Client.builder()
                    .credentials(creds)
                    .build();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

        GenerateContentResponse response =
                client.models.generateContent("anthropic/claude-opus-4-6", "What is your name?", null);

then we got this error

com.google.genai.errors.ClientException: 400 . claude-opus-4-6 is not supported in the generateContent API.
    at com.google.genai.errors.ApiException.throwFromResponse (ApiException.java:94)
    at com.google.genai.HttpApiResponse.getBody (HttpApiResponse.java:37)
    at com.google.genai.Models.processResponseForPrivateGenerateContent (Models.java:5480)
    at com.google.genai.Models.privateGenerateContent (Models.java:5546)
    at com.google.genai.Models.generateContent (Models.java:7076)
    at com.google.genai.Models.generateContent (Models.java:7150)
    at com.google.genai.examples.GenerateContent.main (GenerateContent.java:89)

Suggestion

This Java SDK project is a great library as it allows third-party projects to access the Google Cloud platform and Vertex AI using HTTP(s) with JSON body messages instead of using gRPC.

As the access to one of the model partners like Claude, Mistral, etc relies on the following endpoints

https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:rawPredict
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict

I suggest that this Java SDK also support them. Here is the API documentation: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/projects.locations.endpoints/rawPredict

FYI: The projects Langchain4j and Quarkus Langchains4j currently use the gRPC code of the following GAV and will be more than happy to use now this SDK with HTTP(S).

<dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>google-cloud-aiplatform</artifactId>
            <version>3.69.0</version>
        </dependency>

[cmoulliard]

Technically speaking, we need a new method able to generate the URL for rawPredict or streamRawPredict similar to

java-genai/src/main/java/com/google/genai/Models.java

Lines 5584 to 5613 in 959c01b

	BuiltRequest buildRequestForPrivateGenerateContent(
	String model, List<Content> contents, GenerateContentConfig config) {
	GenerateContentParameters.Builder parameterBuilder = GenerateContentParameters.builder();
	if (!Common.isZero(model)) {
	parameterBuilder.model(model);
	}
	if (!Common.isZero(contents)) {
	parameterBuilder.contents(contents);
	}
	if (!Common.isZero(config)) {
	parameterBuilder.config(config);
	}
	JsonNode parameterNode = JsonSerializable.toJsonNode(parameterBuilder.build());
	ObjectNode body;
	String path;
	if (this.apiClient.vertexAI()) {
	body = generateContentParametersToVertex(this.apiClient, parameterNode, null, parameterNode);
	path = Common.formatMap("{model}:generateContent", body.get("_url"));
	} else {
	body = generateContentParametersToMldev(this.apiClient, parameterNode, null, parameterNode);
	if (body.get("_url") != null) {
	path = Common.formatMap("{model}:generateContent", body.get("_url"));
	} else {
	path = "{model}:generateContent";
	}
	}
	body.remove("_url");

but where we could rely on a new model Class and not the Content [class](-

java-genai/src/main/java/com/google/genai/types/Content.java

Line 44 in 850d527

public abstract Optional<List<Part>> parts();

) to define the content of the message

❯ set -x PROJECT_ID itpc-gcp-cp-pe-eng-claude
  set -x LOCATION europe-west1
  set -x MODEL_ID claude-opus-4-6

  curl -X POST \
           -H "Authorization: Bearer $(gcloud auth print-access-token)" \
           -H "Content-Type: application/json; charset=utf-8" \
           -d '{
         "anthropic_version": "vertex-2023-10-16",
         "messages": [
           {
             "role": "user",
             "content": "Hi Claude. Can you tell me what is Quarkus"
           }
         ],
         "max_tokens": 1024
       }' \
           "https://$LOCATION-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/anthropic/models/$MODEL_ID:rawPredict" | jq

{
  "model": "claude-opus-4-6",
  "id": "msg_vrtx_013uuCjoqtBKBG5U6mf9jGY5",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "# Quarkus\n\n**Quarkus** is an open-source, Kubernetes-native Java framework tailored for **GraalVM** and **OpenJDK HotSpot**. It is developed by **Red Hat** and is designed to optimize Java specifically for cloud-native, containerized, and serverless environments.\n\n## Key Features\n\n### ⚡ Performance\n- **Fast startup time** – Especially when compiled to native execut

[hemasekhar-p]

Hi @cmoulliard, Thank you for raising feature request and we appreciate you taking the time to share your thoughts with us. The SDK currently hardcodes the :generateContent endpoint and Gemini schemas, which prevents direct passthrough to Model Garden partner models via :rawPredict and :streamRawPredict. we will be filing a feature request internally to track it.

Thank you.

[cmoulliard]

Thank you.

Many thanks for your reply.

I still have some questions:

• Why do we have to use as endpoint rawPredict or streamRawPredict to access Anthropic, Mistral, etc models (part of the models garden) as they are mainly used as Gemini models to generate content, text and are not used at all as predictive AI ?
• Is there a way to figure which URL/endpoint is created when we use: client.models, client.chats, etc ?
• Is there a table listing for each model (gemini, partners or open) which endpoints we could use and which publisher support them: google, anthropic, mistral, etc ?