Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CONTRACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Forge incidents may point to:
- `WorkflowRef`
- `EvidenceRef`
- `WorkflowEvidenceSnapshot`
- `ControlRef`
- `SubjectRef`
- `AssessmentRef`
- `PolicyDecisionRef`
Expand Down Expand Up @@ -98,6 +99,7 @@ Projection fields:
- `workflow_ref`
- `evidence_ref`
- `workflow_evidence_snapshot`
- `control_refs`
- `subject_ref`
- `assessment_ref`
- `policy_decision_ref`
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ Each incident is a YAML file in `incidents/YYYY-MM/` with structured fields cove
- resolution: `root_cause`, `immediate_fix`, `systemic_takeaway`
- metadata: `tags`, `related_incidents`, `playbook_entry`
- optional Proofhouse axes: `capability_area`, `lifecycle_stage`, `issue_class`, `workflow_archetype`, `subject_type`, `blocked_use_class`, `observed_state`
- optional pointer refs: `workflow_ref`, `evidence_ref`, `workflow_evidence_snapshot`, `subject_ref`, `assessment_ref`, `policy_decision_ref`, `use_approval_ref`, `asset_ref`, `derivation_ref`, `transform_ref`
- optional pointer refs: `workflow_ref`, `evidence_ref`, `workflow_evidence_snapshot`, `control_refs`, `subject_ref`, `assessment_ref`, `policy_decision_ref`, `use_approval_ref`, `asset_ref`, `derivation_ref`, `transform_ref`

Existing incident YAML remains compatible: all Proofhouse axes and pointer refs are optional. Older files that only contain the original classification/event/resolution/metadata fields still load, list, analyze, and emit a compatibility `IncidentRef`.

Expand All @@ -185,7 +185,7 @@ Existing incident YAML remains compatible: all Proofhouse axes and pointer refs
When an incident relates to Proofhouse workflow evidence or Operational Learning, keep Forge records pointer-based:

- use structured axes for document-operations and Operational Learning failure classes
- use pointer ref fields for `WorkflowRef`, `EvidenceRef` / `WorkflowEvidenceSnapshot`, `SubjectRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and Operational Learning `AssetRef`, `DerivationRef`, or `TransformRef` placeholders
- use pointer ref fields for `WorkflowRef`, `EvidenceRef` / `WorkflowEvidenceSnapshot`, `ControlRef`, `SubjectRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and Operational Learning `AssetRef`, `DerivationRef`, or `TransformRef` placeholders
- use `context` only for short human-readable summaries
- use `tags` as secondary discovery aids, not as the only structure
- use `related_incidents` only for Forge incident IDs
Expand Down
13 changes: 13 additions & 0 deletions examples/claims/rate-source-ambiguity-incident.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,19 @@ workflow_evidence_snapshot:
digest: "sha256:claims-rate-source-fixture-placeholder"
summary: "Digest and summary posture for synthetic claims evidence classes"

control_refs:
- contract_version: "proofhouse-shared-contracts/v0.1"
contract_name: "ControlRef"
canonical_owner: "workflow_context"
cache_policy: "summary_snapshot"
ref:
ref_id: "control:claims-hybrid-high-dollar-review-v0:rate-source-traceability:g31"
control_assignment_id: "control-assignment:claims-rate-source-traceability:g31"
control_key: "rate_source_traceability"
control_family: "claims_review"
implementation_status: "implemented"
evidence_status: "incomplete"

assessment_ref:
contract_version: "proofhouse-shared-contracts/v0.1"
contract_name: "AssessmentRef"
Expand Down
13 changes: 13 additions & 0 deletions examples/document-operations/redaction-miss-incident.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,19 @@ workflow_evidence_snapshot:
snapshot_id: "workflow-evidence-snapshot:document_ops_regulated_review_v0:g1"
digest: "sha256:document-ops-fixture-placeholder"

control_refs:
- contract_version: "proofhouse-shared-contracts/v0.1"
contract_name: "ControlRef"
canonical_owner: "workflow_context"
cache_policy: "summary_snapshot"
ref:
ref_id: "control:document_ops_regulated_review_v0:redaction_required:g2"
control_assignment_id: "control-assignment:document_ops_redaction_required:g2"
control_key: "redaction_required"
control_family: "document_operations"
implementation_status: "implemented"
evidence_status: "incomplete"

assessment_ref:
contract_version: "proofhouse-shared-contracts/v0.1"
contract_name: "AssessmentRef"
Expand Down
15 changes: 15 additions & 0 deletions forge_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
Incident,
Severity,
parse_observed_state,
parse_pointer_list_value,
parse_pointer_value,
)
from forge_cli.analyzer import (
Expand Down Expand Up @@ -124,6 +125,14 @@ def _parse_optional_ref(field_name: str, value: str | None) -> dict | None:
raise typer.Exit(1)


def _parse_optional_ref_list(field_name: str, values: list[str] | None) -> list[dict]:
try:
return parse_pointer_list_value(values, field_name)
except (TypeError, ValueError) as e:
print_error(str(e))
raise typer.Exit(1)


def _parse_optional_observed_state(value: str | None) -> dict | None:
try:
return parse_observed_state(value)
Expand Down Expand Up @@ -189,6 +198,11 @@ def log(
"--workflow-evidence-snapshot",
help="WorkflowEvidenceSnapshot pointer as JSON object or snapshot id",
),
control_refs: Optional[list[str]] = typer.Option(
None,
"--control-ref",
help="Repeatable ControlRef pointer as a JSON object or ref id",
),
subject_ref: Optional[str] = typer.Option(
None,
"--subject-ref",
Expand Down Expand Up @@ -312,6 +326,7 @@ def log(
}.items()
if field_name in PROOFHOUSE_REF_FIELDS
}
pointer_refs["control_refs"] = _parse_optional_ref_list("control_refs", control_refs)

# Build incident
now = datetime.now(timezone.utc)
Expand Down
8 changes: 6 additions & 2 deletions forge_cli/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
Incident,
Severity,
parse_observed_state,
parse_pointer_list_value,
parse_pointer_value,
)
from forge_cli.schema_metadata import STRUCTURED_AXIS_METADATA
Expand Down Expand Up @@ -111,6 +112,7 @@ def forge_log(
workflow_ref: str = "",
evidence_ref: str = "",
workflow_evidence_snapshot: str = "",
control_refs: str = "",
subject_ref: str = "",
assessment_ref: str = "",
policy_decision_ref: str = "",
Expand Down Expand Up @@ -147,6 +149,7 @@ def forge_log(
workflow_ref: Optional WorkflowRef pointer as JSON object or ref id.
evidence_ref: Optional EvidenceRef pointer as JSON object or ref id.
workflow_evidence_snapshot: Optional WorkflowEvidenceSnapshot pointer as JSON object or id.
control_refs: Optional ControlRef pointers as JSON array, JSON object, or ref id.
subject_ref: Optional SubjectRef pointer as JSON object or ref id.
assessment_ref: Optional AssessmentRef pointer as JSON object or ref id.
policy_decision_ref: Optional PolicyDecisionRef pointer as JSON object or ref id.
Expand Down Expand Up @@ -184,6 +187,7 @@ def forge_log(
"workflow_evidence_snapshot": parse_pointer_value(
workflow_evidence_snapshot, "workflow_evidence_snapshot"
),
"control_refs": parse_pointer_list_value(control_refs, "control_refs"),
"subject_ref": parse_pointer_value(subject_ref, "subject_ref"),
"assessment_ref": parse_pointer_value(assessment_ref, "assessment_ref"),
"policy_decision_ref": parse_pointer_value(policy_decision_ref, "policy_decision_ref"),
Expand Down Expand Up @@ -477,7 +481,7 @@ def forge_schema() -> str:
"capability_area", "lifecycle_stage", "issue_class", "workflow_archetype",
"subject_type", "blocked_use_class", "observed_state",
"workflow_ref", "evidence_ref", "workflow_evidence_snapshot",
"subject_ref", "assessment_ref", "policy_decision_ref", "use_approval_ref",
"control_refs", "subject_ref", "assessment_ref", "policy_decision_ref", "use_approval_ref",
"asset_ref", "derivation_ref", "transform_ref",
],
"structured_axis_fields": PROOFHOUSE_AXIS_FIELDS,
Expand All @@ -499,7 +503,7 @@ def forge_schema() -> str:
"incident_id", "failure_type", "severity", "capability_area",
"lifecycle_stage", "issue_class", "workflow_archetype", "subject_type",
"blocked_use_class", "workflow_ref", "evidence_ref",
"workflow_evidence_snapshot", "subject_ref", "assessment_ref", "policy_decision_ref",
"workflow_evidence_snapshot", "control_refs", "subject_ref", "assessment_ref", "policy_decision_ref",
"use_approval_ref", "asset_ref", "derivation_ref", "transform_ref",
"playbook_entry",
],
Expand Down
36 changes: 36 additions & 0 deletions forge_cli/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"workflow_ref",
"evidence_ref",
"workflow_evidence_snapshot",
"control_refs",
"subject_ref",
"assessment_ref",
"policy_decision_ref",
Expand Down Expand Up @@ -63,6 +64,7 @@
"workflow_ref": "workflow",
"evidence_ref": "evidence",
"workflow_evidence_snapshot": "workflow_evidence_snapshot",
"control_refs": "control",
"subject_ref": "subject",
"assessment_ref": "assessment",
"policy_decision_ref": "policy_decision",
Expand Down Expand Up @@ -403,6 +405,36 @@ def parse_pointer_value(value: Any, field_name: str) -> dict[str, Any] | None:
raise TypeError(f"{field_name} must be a mapping, JSON object string, string ref id, or empty")


def parse_pointer_list_value(value: Any, field_name: str) -> list[dict[str, Any]]:
"""Parse one or more pointer refs from YAML data or CLI/MCP text input."""
if value is None:
return []
if isinstance(value, str):
stripped = value.strip()
if not stripped:
return []
if stripped.startswith("["):
parsed = json.loads(stripped)
if not isinstance(parsed, list):
raise ValueError(f"{field_name} must be a JSON array when JSON array syntax is supplied")
return parse_pointer_list_value(parsed, field_name)
parsed_ref = parse_pointer_value(stripped, field_name)
return [parsed_ref] if parsed_ref else []
if isinstance(value, dict):
parsed_ref = parse_pointer_value(value, field_name)
return [parsed_ref] if parsed_ref else []
if isinstance(value, (list, tuple)):
refs: list[dict[str, Any]] = []
for item in value:
parsed_ref = parse_pointer_value(item, field_name)
if parsed_ref:
refs.append(parsed_ref)
return refs
raise TypeError(
f"{field_name} must be a list, JSON array string, mapping, string ref id, or empty"
)


def parse_observed_state(value: Any) -> dict[str, Any] | None:
"""Parse optional incident-local observed state without assuming canonical truth."""
if value is None:
Expand Down Expand Up @@ -479,6 +511,7 @@ class IncidentRef:
workflow_ref: dict[str, Any] | None = None
evidence_ref: dict[str, Any] | None = None
workflow_evidence_snapshot: dict[str, Any] | None = None
control_refs: list[dict[str, Any]] = field(default_factory=list)
subject_ref: dict[str, Any] | None = None
assessment_ref: dict[str, Any] | None = None
policy_decision_ref: dict[str, Any] | None = None
Expand Down Expand Up @@ -520,6 +553,7 @@ class Incident:
workflow_ref: dict[str, Any] | None = None
evidence_ref: dict[str, Any] | None = None
workflow_evidence_snapshot: dict[str, Any] | None = None
control_refs: list[dict[str, Any]] = field(default_factory=list)
subject_ref: dict[str, Any] | None = None
assessment_ref: dict[str, Any] | None = None
policy_decision_ref: dict[str, Any] | None = None
Expand Down Expand Up @@ -584,6 +618,7 @@ def from_dict(cls, data: dict) -> Incident:
workflow_evidence_snapshot=parse_pointer_value(
data.get("workflow_evidence_snapshot"), "workflow_evidence_snapshot"
),
control_refs=parse_pointer_list_value(data.get("control_refs"), "control_refs"),
subject_ref=parse_pointer_value(data.get("subject_ref"), "subject_ref"),
assessment_ref=parse_pointer_value(data.get("assessment_ref"), "assessment_ref"),
policy_decision_ref=parse_pointer_value(
Expand Down Expand Up @@ -642,6 +677,7 @@ def build_incident_ref(incident: Incident) -> IncidentRef:
workflow_ref=incident.workflow_ref,
evidence_ref=incident.evidence_ref,
workflow_evidence_snapshot=incident.workflow_evidence_snapshot,
control_refs=list(incident.control_refs),
subject_ref=incident.subject_ref,
assessment_ref=incident.assessment_ref,
policy_decision_ref=incident.policy_decision_ref,
Expand Down
2 changes: 1 addition & 1 deletion integrations/codex/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ For document-operations or Operational Learning incidents, prefer structured axe
- `subject_type`
- `blocked_use_class`

Use pointer refs for `WorkflowRef`, `EvidenceRef` / `WorkflowEvidenceSnapshot`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and optional `AssetRef`, `DerivationRef`, or `TransformRef`. Forge should record the incident and recurrence pattern, not source workflow truth, readiness score truth, approval state, export manifests, or source document / asset payloads.
Use pointer refs for `WorkflowRef`, `EvidenceRef` / `WorkflowEvidenceSnapshot`, `ControlRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and optional `AssetRef`, `DerivationRef`, or `TransformRef`. Forge should record the incident and recurrence pattern, not source workflow truth, workflow-control truth, readiness score truth, approval state, export manifests, or source document / asset payloads.

Example:
```
Expand Down
2 changes: 1 addition & 1 deletion templates/analysis-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ For document-operations or Operational Learning incidents, prefer the structured
- `subject_type`
- `blocked_use_class`

Expected document-operations issue classes include `redaction_miss`, `rights_ambiguity`, `promotion_failure`, `export_control_failure`, `transform_failure`, `derivation_quality_failure`, `evidence_gap`, `escalation_miss`, and `reviewer_disagreement`. Treat `workflow_ref`, `evidence_ref`, `workflow_evidence_snapshot`, `subject_ref`, `assessment_ref`, `policy_decision_ref`, `use_approval_ref`, `asset_ref`, `derivation_ref`, and `transform_ref` as pointer refs only.
Expected document-operations issue classes include `redaction_miss`, `rights_ambiguity`, `promotion_failure`, `export_control_failure`, `transform_failure`, `derivation_quality_failure`, `evidence_gap`, `escalation_miss`, and `reviewer_disagreement`. Treat `workflow_ref`, `evidence_ref`, `workflow_evidence_snapshot`, `control_refs`, `subject_ref`, `assessment_ref`, `policy_decision_ref`, `use_approval_ref`, `asset_ref`, `derivation_ref`, and `transform_ref` as pointer refs only.

## Incident Data

Expand Down
1 change: 1 addition & 0 deletions templates/incident.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ observed_state: {} # incident-local state summary only
workflow_ref: null
evidence_ref: null
workflow_evidence_snapshot: null
control_refs: []
subject_ref: null
assessment_ref: null
policy_decision_ref: null
Expand Down
11 changes: 6 additions & 5 deletions templates/playbooks/claims-rate-source-ambiguity.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ internal eval, downstream handoff, audit export, or savings recognition.
`allowed_amount_conflict`, or `missing_claim_evidence`.
- `observed_state` names the affected evidence class, but source detail remains
a summary or digest.
- `WorkflowRef` / `WorkflowEvidenceSnapshot` exists, while the rate-source
evidence summary, method, source version, or licensed-access posture is
missing or unreviewed.
- `WorkflowRef` / `WorkflowEvidenceSnapshot` and relevant `ControlRef` pointers
exist, while the rate-source evidence summary, method, source version, or
licensed-access posture is missing or unreviewed.
- Governance `PolicyDecisionRef` or `UseApprovalRef` is incomplete, denied, or
review-required for the attempted use.

Expand All @@ -38,8 +38,9 @@ internal eval, downstream handoff, audit export, or savings recognition.
1. Record structured axes: `capability_area`, `lifecycle_stage`, `issue_class`,
`workflow_archetype`, `subject_type`, and `blocked_use_class`.
2. Attach pointer-style refs only: `WorkflowRef`, `WorkflowEvidenceSnapshot` or
`EvidenceRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and
any relevant `AssetRef`, `DerivationRef`, or `TransformRef`.
`EvidenceRef`, `ControlRef`, `AssessmentRef`, `PolicyDecisionRef`,
`UseApprovalRef`, and any relevant `AssetRef`, `DerivationRef`, or
`TransformRef`.
3. Route workflow/source evidence gaps to Workflow Context.
4. Route suitability and trust-gap interpretation to Readiness.
5. Route approvals, redaction, use, export, action, and audit readback to
Expand Down
4 changes: 2 additions & 2 deletions templates/playbooks/document-review-redaction-miss.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A document-operations incident involves a candidate eval, training, policy-learn
- Incident has `workflow_archetype: document_operations`.
- Incident has `issue_class: redaction_miss`.
- `blocked_use_class` is `internal_eval`, `internal_training`, `policy_learning`, or `external_export`.
- The incident has `WorkflowRef` and `WorkflowEvidenceSnapshot` pointers, but the redaction `TransformRef` or Governance `UseApprovalRef` pointer is missing, stale, or non-dereferenceable.
- The incident has `WorkflowRef`, `WorkflowEvidenceSnapshot`, and relevant `ControlRef` pointers, but the redaction `TransformRef` or Governance `UseApprovalRef` pointer is missing, stale, or non-dereferenceable.

## Prevention

Expand All @@ -21,7 +21,7 @@ A document-operations incident involves a candidate eval, training, policy-learn
## Response Protocol

1. Record the incident with structured axes: `capability_area`, `lifecycle_stage`, `issue_class`, `workflow_archetype`, `subject_type`, and `blocked_use_class`.
2. Attach pointer-style refs only: `WorkflowRef`, `WorkflowEvidenceSnapshot` or `EvidenceRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and any relevant `AssetRef`, `DerivationRef`, or `TransformRef`.
2. Attach pointer-style refs only: `WorkflowRef`, `WorkflowEvidenceSnapshot` or `EvidenceRef`, `ControlRef`, `AssessmentRef`, `PolicyDecisionRef`, `UseApprovalRef`, and any relevant `AssetRef`, `DerivationRef`, or `TransformRef`.
3. Confirm the attempted use class is blocked until the owning Governance and Operational Learning systems provide valid refs.
4. Add the pattern to analysis if recurrence appears across workflows, agents, or document families.

Expand Down
8 changes: 8 additions & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ def test_log_command_accepts_structured_axes_and_pointer_refs(tmp_path):
"internal_eval",
"--workflow-ref",
"workflow:document_ops_regulated_review_v0",
"--control-ref",
"control:document_ops_redaction_required:g2",
"--control-ref",
"control:document_ops_use_gate:g2",
"--assessment-ref",
"assessment:document_ops_regulated_review_v0:g2",
"--use-approval-ref",
Expand All @@ -86,6 +90,10 @@ def test_log_command_accepts_structured_axes_and_pointer_refs(tmp_path):
assert incident.capability_area == "governance"
assert incident.issue_class == "redaction_miss"
assert incident.workflow_ref["ref_id"] == "workflow:document_ops_regulated_review_v0"
assert [ref["ref_id"] for ref in incident.control_refs] == [
"control:document_ops_redaction_required:g2",
"control:document_ops_use_gate:g2",
]
assert incident.use_approval_ref["ref_id"] == "use-approval:document_ops_internal_eval:g2"


Expand Down
Loading
Loading