Skip to content

feat(kb-open): P0-A 认证体系 + 数据模型 + 管理端 + 限流 + 集中授权#444

Merged
mateaix merged 2 commits into
mateaix:devfrom
ncw1992120:feat/kb-open-api-p0a
Jun 28, 2026
Merged

feat(kb-open): P0-A 认证体系 + 数据模型 + 管理端 + 限流 + 集中授权#444
mateaix merged 2 commits into
mateaix:devfrom
ncw1992120:feat/kb-open-api-p0a

Conversation

@ncw1992120

Copy link
Copy Markdown
Contributor

Closes #441 · Part of #440

改动

知识库开放 API 的认证骨架。这是 P0-B(9 个端点,#442)的前置依赖。

认证与鉴权

  • TokenHashUtil(共享 hash 内核,A4):SHA-256 生成/校验,KB-key 与 PAT 共享(先 hash 层,CRUD/UI 后续)
  • KbApiKeyService:签发(mck_ 前缀)/ 校验(hash 索引 O(1))/ 撤销 / 多 KB 绑定管理
    • R3:空绑定拒绝签发(对外 key 空绑定 = 零访问)
  • KbOpenApiAuthFilter:permitAll 路径的唯一守门人
    • R1:缺失/无效 key 当场返回 401,绝不 pass-through
    • R2:每 key 滑动窗口限流,超限返回 429
  • @RequireKbScope + KbScopeInterceptor(A1):集中授权(scope 检查 + KB 归属校验),仿 @RequireWorkspaceRole,不逐端点手写
  • KbApiKeyRateLimiter:滑动窗口限流器(复用 TriggerRateLimiter 范式)

管理端

  • KbApiKeyAdminController:JWT 认证 CRUD(list/create/detail/update/revoke),workspace 隔离,@RequireWorkspaceRole("admin")

数据模型

  • V162 迁移(h2/mysql/kingbase):mate_kb_api_key + mate_kb_api_key_binding
    • token_hash SHA-256(UNIQUE 自带索引)、prefix(4 位)、rate_limit_per_min(P0 强制)
    • binding 表 kb_id 索引(Wiki 面板反查)

SecurityConfig

  • /api/v1/open/kb/** permitAll + KbOpenApiAuthFilter 注册
  • CORS 已由现有 WebMvcConfig.addCorsMappings("/api/**") 覆盖(R6)

测试

Tests run: 17, Failures: 0, Errors: 0, Skipped: 0
  • KbApiKeyServiceTest(13):R3 空绑定拒绝、create+authenticate 往返、过期/禁用/错误前缀拒绝、kb:* 通配、revoke 跨工作区 404
  • KbApiKeyRateLimiterTest(4):滑动窗口、per-key 隔离、过期恢复、limit≤0 禁用

回归验证:WebChatApprovalInteractionTest(9)全绿。

安全设计要点

要求 实现
R1 不 pass-through filter 缺失/无效 key 当场 401
R2 限流 每 key 滑动窗口,search 消耗 embedding token 必须限流
R3 空绑定 对外 key 空绑定 = 零访问,拒绝签发
A1 集中授权 @RequireKbScope 注解 + interceptor,避免重蹈 #438/#439 IDOR
A4 共享内核 TokenHashUtil,P1 可与 PAT 共享

…authz

Implements the authentication backbone for the KB Open API (mateaix#441):
API key lifecycle, a permitAll-path filter that rejects (never
pass-through), per-key sliding-window rate limiting, and a centralized
@RequireKbScope interceptor for scope + KB-ownership checks.

Components:
- TokenHashUtil: shared SHA-256 hash kernel (A4), reusable by PAT later
- KbApiKeyService: mint/authenticate/revoke/update + multi-KB binding
  (R3: empty binding = zero access, not "all KBs")
- KbOpenApiAuthFilter: sole gatekeeper for /api/v1/open/kb/** (R1: must
  return 401, no pass-through); R2: per-key rate limit (429)
- KbApiKeyRateLimiter: sliding-window limiter (TriggerRateLimiter pattern)
- @RequireKbScope + KbScopeInterceptor: centralized authorization (A1),
  scope check + kbId ownership from path variable
- KbApiKeyAdminController: JWT-authenticated CRUD (list/create/detail/
  update/revoke), workspace-scoped
- V162 migration (h2/mysql/kingbase): mate_kb_api_key + _binding tables

Security:
- mck_ prefix (distinct from PAT mc_ and JWT eyJ)
- SHA-256 hash storage, plaintext shown once at creation
- prefix column (4 chars) for UI display only

Tests (17 new, all green):
- KbApiKeyServiceTest: R3 empty-binding rejection, auth round-trip,
  expired/disabled/wrong-prefix rejection, kb:* wildcard, revoke
- KbApiKeyRateLimiterTest: sliding window, per-key isolation, recovery

Closes mateaix#441
@mateaix

mateaix commented Jun 28, 2026

Copy link
Copy Markdown
Owner

感谢这份 P0-A 开放 API 基座 🙏 安全设计整体很扎实:token 只存 SHA-256(@JsonIgnore + 唯一索引)、明文仅创建时返回一次、不打印明文;KbOpenApiAuthFilterpermitAll 路径上 fail-closed,KbScopeInterceptor 再做 scope + KB 归属校验;空绑定=零访问也在 mint/update/check 三处都强制了。没有发现后门或硬编码凭据。

不过有两个阻塞项需要先修,合并前请处理:

1. prefix 列宽不够,会在 MySQL 上直接打断建 key 主流程。
KbApiKeyService.create()displayPrefix 取的是 min(PREFIX_DISPLAY_LEN + KEY_PREFIX.length(), len) = min(4+4,…) = 8 字符mck_ + 4),但三套迁移里列定义都是 prefix VARCHAR(6)。MySQL 严格模式下会因数据截断抛错(核心建 key 路径直接挂),H2 上则静默截断。请把列放宽到 VARCHAR(12),或把 substring 改成只取 PREFIX_DISPLAY_LEN。实体 Javadoc 写「前 4 位」也与代码(mck_+4)不一致,顺手统一。

2. 迁移版本号 V162 与已合并的 #437 冲突。
#435(V161)、#437(V162/V163)刚刚合并到 dev,磁盘上 V161/V162/V163 已被占用。本 PR 的 V162__kb_open_api_key.sql 会和 #437 撞号,Flyway 在后落地的一方会报 checksum/already-applied。请把它顺延到 V164 之后(并相应更新文件里那段写成 -- V161: 的过期注释 + 「see mysql/V161」的失效引用)。

非阻塞建议:

  • SecurityConfig.java / WebMvcConfig.javaprivate final vip.mate.kbopen.auth.KbOpenApiAuthFilter … 用了内联全限定名,按仓库规范改成顶部 import + 简单名。
  • parseScopesSet.of(scopes.split(",")) 没 trim,存成 "kb:search, kb:read"" kb:read" 永远匹配不上,建议 .map(String::trim)
  • extractBearerToken 接受 ?token= 作为 SSE 兜底,但 P0 没有 SSE 端点,API key 走 query 会落进访问日志/代理日志,建议在 SSE 真正落地前先去掉这个兜底。
  • 限流是单节点内存级(Javadoc 已诚实标注),多节点部署会是 N×,后续可跟一个 issue。
  • ⚠️ kb-open-api-design.md 放在仓库根目录:按 CLAUDE.md 的开源同步规则,根目录下未被 PRIVATE_ITEMS 收录的文件会被 sync-opensource.sh 推到公开镜像,而这份设计文档含 RFC-090 等内部引用。请移到 rfcs/(或加进 PRIVATE_ITEMS),别留在根目录。

P0-A 是整条 P0-B/Deep Research 栈的地基,建议先在这里把上面两个阻塞项(列宽 + 版本号)和设计文档位置改掉,#445/#446 再相应 rebase。改好后我们就合并 🙏

BLOCKERS:
- prefix column VARCHAR(6) → VARCHAR(12) across all 3 migration dialects;
  KbApiKeyService.create() produces 8 chars (mck_ + 4 random), VARCHAR(6)
  would silently truncate on H2 and throw on MySQL strict mode
- Rename migration V162 → V164 to avoid collision with merged mateaix#437
  (V162=wiki_raw_material_error_code, V163=wiki_raw_material_warning)
  and fix stale V161 references in h2/kingbase comments

NITS:
- SecurityConfig/WebMvcConfig: replace inline FQN with import + simple name
- parseScopes: add .map(String::trim) so ' kb:read' matches correctly
- Remove ?token= SSE query fallback in KbOpenApiAuthFilter (P0-A has no
  SSE endpoint; key would leak into access/proxy logs — R5)
- Move kb-open-api-design.md from repo root to rfcs/ (contains RFC-090
  internal reference that would be exposed by sync-opensource)
- KbApiKeyEntity Javadoc: 'first 4 chars' → 'first 8 chars (mck_ + 4)'
  to match actual behavior
@ncw1992120

Copy link
Copy Markdown
Contributor Author

Fixed. Commit 6fd6244 addresses both blockers and the four nits:

Blockers:

  1. prefix VARCHAR(6)VARCHAR(12) across all 3 dialects (mysql/h2/kingbase). KbApiKeyService.create() produces 8 chars (mck_ + 4 random) — VARCHAR(6) would silently truncate on H2 and throw on MySQL strict mode. Entity Javadoc updated from "first 4 chars" → "first 8 chars".
  2. Migration renamed V162V164 (V162/V163 now owned by merged feat(wiki): 知识库处理失败的可见性改造(错误码链路 + 静默子步骤告警 + 跨KB失败中心) #437). Stale -- V161: comment + See mysql/V161 references in h2/kingbase files updated to V164.

Nits:
3. SecurityConfig / WebMvcConfig — inline FQN replaced with import + simple name.
4. parseScopes — added .map(String::trim) so " kb:read" matches correctly.
5. ?token= SSE fallback removed from KbOpenApiAuthFilter (P0-A has no SSE endpoint; key would leak into access/proxy logs. TODO comment left for when Deep Research SSE lands).
6. kb-open-api-design.md moved from repo root to rfcs/ (contains RFC-090 internal reference).

Compilation and all 17 KB Open API tests pass. Ready for re-review.

@mateaix

mateaix commented Jun 28, 2026

Copy link
Copy Markdown
Owner

感谢按 review 全部改好了 🙏 两个阻塞项都已解决:

  • prefix 列宽:三套方言迁移都改成 VARCHAR(12),实体 Javadoc 也同步成「前 8 位(mck_ + 4 随机)」,和代码一致。
  • 迁移版本号:从 V162 顺延到 V164(基线已到 V163,无冲突),三个方言目录齐全,过期的 -- V161 头注释也都修正了。

其余建议也都采纳了:SecurityConfig/WebMvcConfig 改 import + 简单名、parseScopes 加了 .map(String::trim)、去掉 ?token= query 兜底(留了带安全说明的 TODO)、设计文档移到 rfcs/(属 PRIVATE_ITEMS,不会同步到开源镜像)。安全设计复核通过(token 只存 SHA-256、filter fail-closed、空绑定=零访问)。已合并到 dev

一个可选小尾巴(不阻塞):KbApiKeyServiceparseScopes.collect(java.util.stream.Collectors.toUnmodifiableSet()) 用了内联全限定名,而文件顶部其实已经 import Collectors 了,下次顺手改成 Collectors.toUnmodifiableSet() 即可。

@mateaix mateaix merged commit 2d04ce9 into mateaix:dev Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(kb-open): P0-A 认证体系 + 数据模型 + 管理端

2 participants