Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/quick-create-llm-suggest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"server": minor
"dashboard": minor
"sdk": minor
---

Add `risk.customRules.suggest` endpoint that calls OpenRouter to turn a one-line description ("what do you want to detect?") into a prefilled custom detection rule. The dashboard's New Custom Detection Rule sheet now opens on a single textarea, calls the new endpoint, and lands the operator in the editable review form with the suggested rule_id, title, description, regex, and severity.
9 changes: 9 additions & 0 deletions .changeset/rule-playground-and-drop-severity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"server": minor
"dashboard": minor
"sdk": minor
---

Add a rule playground: from the Detection Rules detail sheet, the operator pastes a sample into a textarea and the dashboard calls the new `risk.rules.test` endpoint which dispatches to the same scanner code (gitleaks, Presidio, prompt-injection, regex) the worker uses. The response is a list of `TestDetectionRuleMatch`es mirroring the runtime risk_result shape.

Drop the severity-override UI from the rule detail sheet. The override edit / reset affordances will return in a follow-up PR; default severity continues to render as a row badge for context.
334 changes: 334 additions & 0 deletions .speakeasy/out.openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18463,6 +18463,113 @@ paths:
x-speakeasy-name-override: list
x-speakeasy-react-hook:
name: RiskCategories
/rpc/risk.customRules.suggest:
post:
description: Suggest a custom detection rule (rule_id, title, description, regex, severity) from a natural-language prompt. Calls the configured LLM with a JSON-schema constrained response so the dashboard can prefill the create form.
operationId: suggestCustomDetectionRule
parameters:
- allowEmptyValue: true
description: API Key header
in: header
name: Gram-Key
schema:
description: API Key header
type: string
- allowEmptyValue: true
description: Session header
in: header
name: Gram-Session
schema:
description: Session header
type: string
- allowEmptyValue: true
description: project header
in: header
name: Gram-Project
schema:
description: project header
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/SuggestCustomDetectionRuleRequestBody'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/SuggestCustomDetectionRuleResult'
description: OK response.
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'bad_request: request is invalid'
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unauthorized: unauthorized access'
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'forbidden: permission denied'
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'not_found: resource not found'
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'conflict: resource already exists'
"415":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unsupported_media: unsupported media type'
"422":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'invalid: request contains one or more invalidation fields'
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unexpected: an unexpected error occurred'
"502":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'gateway_error: an unexpected error occurred'
security:
- apikey_header_Gram-Key: []
project_slug_header_Gram-Project: []
- project_slug_header_Gram-Project: []
session_header_Gram-Session: []
- {}
summary: suggestCustomDetectionRule risk
tags:
- risk
x-speakeasy-group: risk.customRules
x-speakeasy-name-override: suggest
x-speakeasy-react-hook:
name: RiskSuggestCustomRule
type: mutation
/rpc/risk.overview.get:
get:
description: Get risk overview metrics and trend data for the current project.
Expand Down Expand Up @@ -19842,6 +19949,113 @@ paths:
x-speakeasy-name-override: list
x-speakeasy-react-hook:
name: RiskListResults
/rpc/risk.rules.test:
post:
description: Run a single detection rule against pasted sample text and return any matches. Reuses the same scanner code (gitleaks, Presidio, prompt-injection, custom regex) that the analyzer runs in production so the playground match shape mirrors the chat-message path.
operationId: testDetectionRule
parameters:
- allowEmptyValue: true
description: API Key header
in: header
name: Gram-Key
schema:
description: API Key header
type: string
- allowEmptyValue: true
description: Session header
in: header
name: Gram-Session
schema:
description: Session header
type: string
- allowEmptyValue: true
description: project header
in: header
name: Gram-Project
schema:
description: project header
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/TestDetectionRuleRequestBody'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/TestDetectionRuleResult'
description: OK response.
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'bad_request: request is invalid'
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unauthorized: unauthorized access'
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'forbidden: permission denied'
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'not_found: resource not found'
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'conflict: resource already exists'
"415":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unsupported_media: unsupported media type'
"422":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'invalid: request contains one or more invalidation fields'
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'unexpected: an unexpected error occurred'
"502":
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
description: 'gateway_error: an unexpected error occurred'
security:
- apikey_header_Gram-Key: []
project_slug_header_Gram-Project: []
- project_slug_header_Gram-Project: []
session_header_Gram-Session: []
- {}
summary: testDetectionRule risk
tags:
- risk
x-speakeasy-group: risk.rules
x-speakeasy-name-override: test
x-speakeasy-react-hook:
name: RiskTestDetectionRule
type: mutation
/rpc/slack-apps.configure:
post:
description: Store Slack credentials (client ID, client secret, signing secret) for an app.
Expand Down Expand Up @@ -36319,6 +36533,51 @@ components:
required:
- id
- feedback
SuggestCustomDetectionRuleRequestBody:
type: object
properties:
existing_rule_ids:
type: array
items:
type: string
description: Existing built-in and custom rule ids the suggested id must avoid colliding with.
prompt:
type: string
description: Natural-language description of what the rule should detect.
minLength: 3
maxLength: 500
required:
- prompt
SuggestCustomDetectionRuleResult:
type: object
properties:
description:
type: string
description: Description of what the rule detects and why it matters.
regex:
type: string
description: RE2-compatible regex pattern the rule should match against.
rule_id:
type: string
description: Suggested stable identifier, prefixed with `custom.`.
severity:
type: string
description: Suggested severity level.
enum:
- info
- low
- medium
- high
- critical
title:
type: string
description: Short, human-friendly title for the rule.
required:
- rule_id
- title
- description
- regex
- severity
TelemetryFilter:
type: object
properties:
Expand Down Expand Up @@ -36384,6 +36643,81 @@ components:
- attributes
- resource_attributes
- service
TestDetectionRuleMatch:
type: object
properties:
confidence:
type: number
description: Confidence score in the range 0.0 to 1.0.
format: double
description:
type: string
description: Human-readable description of why this match was flagged.
end_pos:
type: integer
description: Exclusive end byte offset of the match in the sample.
format: int64
match:
type: string
description: Matched substring of the sample.
rule_id:
type: string
description: Canonical rule id of the match (may differ from the requested rule id when one input matches multiple rules).
source:
type: string
description: Detection source (e.g. `gitleaks`, `presidio`, `prompt_injection`, `custom`).
start_pos:
type: integer
description: Inclusive start byte offset of the match in the sample.
format: int64
tags:
type: array
items:
type: string
description: Tags from the underlying rule.
required:
- rule_id
- match
- start_pos
- end_pos
- source
- confidence
TestDetectionRuleRequestBody:
type: object
properties:
regex:
type: string
description: Regex pattern. Required for `custom.*` rule ids since the server doesn't persist custom rules yet; ignored for built-in rules.
rule_id:
type: string
description: Rule identifier to evaluate (e.g. `secret.aws_access_token`, `pii.email_address`, `custom.acme_token`).
minLength: 1
maxLength: 200
text:
type: string
description: Sample text to scan.
minLength: 1
maxLength: 50000
required:
- rule_id
- text
TestDetectionRuleResult:
type: object
properties:
matches:
type: array
items:
$ref: '#/components/schemas/TestDetectionRuleMatch'
description: Matches the rule found in the sample.
reason:
type: string
description: Why the rule isn't supported when `supported` is false.
supported:
type: boolean
description: False when the rule has no text-only detector (e.g. `shadow_mcp`, `destructive_tool`).
required:
- matches
- supported
TierLimits:
type: object
properties:
Expand Down
2 changes: 0 additions & 2 deletions .speakeasy/workflow.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading