Skip to content

[JS] regression - schema accepted under v1.22 fail from v1.23 onwards #4110

@christophe-g

Description

@christophe-g

Describe the bug

Relatively simple output schema break under v1.23 (up to v1.27) - but works with v1.22:

GenkitError: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent: [400 Bad Request] Invalid JSON payload received. Unknown name "$ref" at 'generation_config.response_schema.properties[1].value.items.properties[1].value.items.properties[1].value.items.any_of[0][1].properties[0].value': Cannot find field.
Invalid JSON payload received. Unknown name "$ref" at 'generation_config.response_schema.properties[1].value.items.properties[1].value.items.properties[1].value.items.any_of[0][1].properties[1].value': Cannot find field.

To Reproduce

import { googleAI } from '@genkit-ai/google-genai';
import { getDownloadURL, getStorage } from 'firebase-admin/storage';
import { genkit, z } from 'genkit';
import { beforeEach, describe, expect, it } from 'vitest';

describe('importFormFlow Regression Test', () => {

  // 1. Simplified Schema
  const questionTypeEnums = z.enum([
    'textfield',
    'radio',
  ])

  const baseQuestionSchema = z.object({
    code: z.string().describe('The code of the question, camelCase'),
    label: z.string().describe('The question text'),
  });

  const textfieldSchema = baseQuestionSchema.extend({
    subType: questionTypeEnums.extract(['textfield']),
  }).describe('Text field question, allowing users to enter a single line of text');

  const optionSchema = z.object({
    code: z.string().describe('The code of the question, camelCase'),
    label: z.string().describe('The question text'),
  });

  const radioSchema = baseQuestionSchema.extend({
    subType: questionTypeEnums.extract(['radio']),
    items: z.array(optionSchema).describe('The options for the radio question'),
  }).describe('Radio question, allowing users to select one option from a list');

  const questionSchema = z.discriminatedUnion('subType', [
    textfieldSchema,
    radioSchema,
  ]);

  const textSchema = z.object({
    code: z.string().describe('The code of the question, camelCase'),
    label: z.string().describe('The question text'),
    // items: z.array(optionSchema).describe('List of option for the question'),
  });

  const sectionSchema = z.object({
    title: z.string().optional().describe('Section title'),
    items: z.array(z.union([questionSchema, textSchema])).describe('List of questions in the section'),
  });

  const pageSchema = z.object({
    heading: z.string().describe('Page heading'),
    sections: z.array(sectionSchema).describe('Sections in the page'),
  });

  const formSchema = z.object({
    title: z.string().describe('Form title to display'),
    pages: z.array(pageSchema).describe('Pages in the form'),
  });

  const ai = genkit({
    plugins: [
      googleAI({
        apiKey: geminiApiKey,
        // legacyResponseSchema: true // this is necessary from v1.23, otherwise another error is triggered
      })
    ]
  });

  const model = googleAI.model('gemini-3-flash-preview')

  const importFlow = ai.defineFlow(
    {
      name: 'importFlow',
      inputSchema: z.string(), // Input is just a prompt string for this test
      outputSchema: formSchema,
    },
    // @ts-expect-error - temporary workaround for genkit typing issue
    async (prompt) => {
      const { output } = await ai.generate({
        model,
        prompt: prompt,
        output: { schema: formSchema },
      });
      return output;
    }
  );

  it('should successfully parse generated form data', async () => {
    const result = await importFlow('Generate a form');
    console.info('Generated Result:', JSON.stringify(result, null, 2));
    expect(result).not.toBeNull();
    expect(result?.pages).toBeInstanceOf(Array);
    expect(result?.pages[0].sections[0].items).toBeInstanceOf(Array);
  }, 10000);
});

Expected behavior

The schema is accepted as an output schema.

Runtime (please complete the following information):

  • OS: linux
  • Version: 1.23

** Node version

  • 23.6.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingjs

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions