Skip to content

Allows Mistral byok to be used for Codestral #4000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chrarnoldus wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Allows Mistral byok to be used for Codestral #4000

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions apps/web/src/app/api/fim/completions/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import { readDb } from '@/lib/drizzle';
import { debugSaveProxyRequest } from '@/lib/debugUtils';
import { sentryLogger } from '@/lib/utils.server';
import { getBYOKforOrganization, getBYOKforUser } from '@/lib/ai-gateway/byok';
import type { UserByokProviderId } from '@/lib/ai-gateway/providers/openrouter/inference-provider-id';

// Mistral exposes FIM on two separate, key-incompatible endpoints:
// - https://api.mistral.ai (La Plateforme, paid tier keys)
Expand Down Expand Up @@ -153,11 +154,12 @@ export async function POST(request: NextRequest) {
// Extract properties for usage context
const promptInfo = extractFimPromptInfo(requestBody);

const byokProviderKey = fimProvider === 'mistral' ? 'codestral' : 'inception';
const byokProviderKeys: UserByokProviderId[] =
fimProvider === 'mistral' ? ['codestral', 'mistral'] : ['inception'];

@kilo-code-bot kilo-code-bot Bot Jun 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: BYOK key priority not honoured — DB orderBy(created_at) overrides the ['codestral', 'mistral'] intent

getBYOKforUser/getBYOKforOrganization query with inArray(byok_api_keys.provider_id, providerIds) and orderBy(byok_api_keys.created_at). SQL's IN (…) clause does not preserve the input array order, so when a user has both a codestral key and a mistral key the DB returns them oldest-first.

userByok?.at(0) (line 237) then picks the oldest key regardless of which provider it is. If the mistral key was created before the codestral key, the codestral key is silently ignored and the user's Mistral La Plateforme key is used with api.mistral.ai — the correct endpoint for that key type, but the codestral key preference is lost.

To honour the priority ordering, consider re-sorting the returned entries client-side by their position in byokProviderKeys before calling .at(0):

const sorted = userByok?.slice().sort(
  (a, b) => byokProviderKeys.indexOf(a.providerId) - byokProviderKeys.indexOf(b.providerId)
);
const userByokEntry = sorted?.at(0);

Reply with @kilocode-bot fix it to have Kilo Code address this issue.


const userByok = organizationId
? await getBYOKforOrganization(readDb, organizationId, [byokProviderKey])
: await getBYOKforUser(readDb, user.id, [byokProviderKey]);
? await getBYOKforOrganization(readDb, organizationId, byokProviderKeys)
: await getBYOKforUser(readDb, user.id, byokProviderKeys);

const usageContext: MicrodollarUsageContext = {
api_kind: 'fim_completions',
Expand Down
4 changes: 2 additions & 2 deletions apps/web/src/components/organizations/byok/BYOKKeysManager.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ const VERCEL_BYOK_PROVIDER_NAMES = {
fireworks: 'Fireworks',
google: 'Google AI Studio',
minimax: 'MiniMax',
mistral: 'Mistral AI (other models)',
mistral: 'Mistral AI',
moonshotai: 'Moonshot AI',
novita: 'Novita',
perplexity: 'Perplexity',
Expand All @@ -81,7 +81,7 @@ const VERCEL_BYOK_PROVIDER_NAMES = {

const VERCEL_BYOK_PROVIDERS = [
...Object.entries(VERCEL_BYOK_PROVIDER_NAMES).map(([id, name]) => ({ id, name })),
{ id: DirectUserByokInferenceProviderIdSchema.enum.codestral, name: 'Mistral AI (Codestral)' },
{ id: DirectUserByokInferenceProviderIdSchema.enum.codestral, name: 'Legacy Codestral-only key' },
];

const DIRECT_BYOK_PROVIDERS_LIST = Object.entries(DIRECT_BYOK_PROVIDERS_META).map(([id, name]) => ({
Expand Down
8 changes: 4 additions & 4 deletions apps/web/src/lib/ai-gateway/byok/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,22 @@ import type { BYOKResult } from '@/lib/ai-gateway/providers/types';
import { getVercelModelsMetadata } from '@/lib/ai-gateway/providers/gateway-models-cache';

export async function getModelUserByokProviders(modelId: string): Promise<UserByokProviderId[]> {
if (isCodestralModel(modelId)) {
return ['codestral'];
}
const vercelModelMetadata = await getVercelModelsMetadata();
if (Object.keys(vercelModelMetadata).length === 0) {
console.error('[getModelUserByokProviders] no Vercel model metadata in the database');
return [];
}
const providers =
const providers: UserByokProviderId[] =
vercelModelMetadata[mapModelIdToVercel(modelId, false)]?.endpoints
.map(ep => VercelUserByokInferenceProviderIdSchema.safeParse(ep.provider_name ?? ep.tag).data)
.filter(providerId => providerId !== undefined) ?? [];
if (providers.length === 0) {
console.debug(`[getModelUserByokProviders] no user byok providers for ${modelId}`);
return [];
}
if (isCodestralModel(modelId)) {
providers.splice(0, 0, 'codestral');

@kilo-code-bot kilo-code-bot Bot Jun 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Array.prototype.splice is mutating — prefer unshift or a spread

providers.splice(0, 0, 'codestral') works here because the array is local, but providers.unshift('codestral') is the conventional way to prepend in-place, and ['codestral', ...providers] (reassigning the const) makes the intent most readable. Neither change is required for correctness, but the current idiom is surprising to readers who expect splice for removal, not insertion at index 0.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

}
console.debug('[getModelUserByokProviders] found user byok providers for %s', modelId, providers);
return providers;
}
Expand Down
2 changes: 0 additions & 2 deletions apps/web/src/routers/byok-router.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ import { getVercelInferenceProviderConfigForUserByok } from '@/lib/ai-gateway/pr
import { decryptByokRow } from '@/lib/ai-gateway/byok';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
import { mapModelIdToVercel } from '@/lib/ai-gateway/providers/vercel/mapModelIdToVercel';
import { isCodestralModel } from '@/lib/ai-gateway/providers/mistral';
import { isKiloExclusiveModel } from '@/lib/ai-gateway/models';
import DIRECT_BYOK_PROVIDERS from '@/lib/ai-gateway/providers/direct-byok/direct-byok-definitions';
import {
Expand Down Expand Up @@ -114,7 +113,6 @@ async function fetchSupportedModels(): Promise<Record<string, string[]>> {
if (isKiloExclusiveModel(openRouterModel.id)) continue;
const vercelModel = vercelModelMetadata[mapModelIdToVercel(openRouterModel.id, false)];
if (!vercelModel) continue;
if (isCodestralModel(vercelModel.id)) continue;
if (vercelModel.type !== 'language') continue;
for (const endpoint of vercelModel.endpoints) {
const providerParsed = VercelUserByokInferenceProviderIdSchema.safeParse(
Expand Down