Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,16 +77,39 @@ Results: 0 errors, 0 warnings

## Validation Levels

The validator implements 4 of the 6 TDD cascade validation levels:
The validator implements all 6 TDD cascade validation levels:

| Level | Name | What It Checks |
|:------|:-----|:---------------|
| 1 | Schema Validation | YAML files conform to JSON Schema (structure, types, required fields) |
| 2 | Cross-Reference Integrity | `satisfies_signal` and `satisfies_experience` annotations reference valid targets |
| 3 | Contract Satisfaction | Every declared L0 `constraint_contracts` / `commitment_contracts` is satisfied: its `validated_by` provider exists, every `satisfies_constraint` / `satisfies_commitment` reference resolves to a declared contract of the matching kind, and no declared contract is left uncovered |
| 4 | Signal Coverage | Every signal requirement in `perception/signal_requirements.yaml` has at least one satisfying specification |
| 5 | Experience Traceability | Every product and process traces upward to an L0 customer experience goal |
| 6 | Waste Detection | Declared units that are never consumed: orphaned `organization.yaml` roles, unconsumed products (no `implemented_by`, no references), empty contracts (no `requires`), and duplicate `implemented_by` allocations |

Levels 3 (contract satisfaction) and 6 (waste detection) are planned for future releases.
Levels 1–3 emit **errors** (non-zero exit code); Levels 4–6 emit advisory **warnings**.

## Deterministic Query CLI

`orgschema-query` is a plain deterministic query interface over the Level 3 and Level 6 checks — there is **no natural-language / LLM layer**. OrgSchema's constraint space is boolean (a contract is satisfied or it is not; a unit is consumed or it is not), so the appropriate interface is a deterministic query rather than a grounded NL→DSL pipeline. Results are exactly the validator's, so the two tools never disagree.

```bash
# Run all deterministic checks (contracts + waste) over a schema
orgschema-query --schema ./orgschema-demo

# Run a single check
orgschema-query --schema ./orgschema-demo --check contracts
orgschema-query --schema ./orgschema-demo --check waste

# Select by level number (3 = contracts, 6 = waste)
orgschema-query --schema ./orgschema-demo --level 6

# Machine-readable JSON output (for CI / tooling)
orgschema-query --schema ./orgschema-demo --format json
```

Exit code is `1` when an error-severity check (contracts) reports a violation, else `0` — usable in CI alongside `orgschema-validate`.

## Schemas

Expand Down Expand Up @@ -117,7 +140,8 @@ All schemas use JSON Schema Draft 2020-12 and allow additional properties for ex
orgschema-framework/
├── orgschema_framework/
│ ├── __init__.py
│ ├── validate.py # CLI validator (4 validation levels)
│ ├── validate.py # CLI validator (6 validation levels)
│ ├── query.py # Deterministic query CLI (orgschema-query)
│ └── schemas/
│ ├── compliance.json
│ ├── organization.json
Expand Down Expand Up @@ -296,6 +320,7 @@ Install with `uv sync` (runtime) or `uv sync --extra dev` (with dev tools).
| Block | Entry point | Inputs | Outputs |
|-------|-------------|--------|---------|
| Validation CLI | `orgschema-validate <path>` | YAML specs at `<path>` | stdout PASS/FAIL summary; exit code 0/1 |
| Query CLI | `orgschema-query --schema <path>` | YAML specs at `<path>` | structural text/JSON of L3/L6 violations; exit code 0/1 |
| Schema files | `orgschema_framework/schemas/*.json` | — | Reused by `validate.py` |
| Test suite | `pytest` | `tests/` (if present) | `output/logs/master_run.log` |
| Full reproduction | `./reproduce.sh` | repo root | `output/logs/master_run.log` |
Expand Down
177 changes: 177 additions & 0 deletions orgschema_framework/query.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
"""Deterministic query interface over an Organizational Schema directory.

This is a plain, deterministic query CLI — there is NO natural-language or
LLM layer. OrgSchema's constraint space is boolean (a contract is satisfied
or it is not; a unit is consumed or it is not), so the appropriate interface
is a deterministic query over the validator's existing level functions rather
than a grounded NL->DSL pipeline.

The CLI re-uses the Level 3 (contract satisfaction) and Level 6 (waste
detection) functions from ``orgschema_framework.validate`` so the query
results are exactly the validator's results — single source of truth.

Usage:
orgschema-query --schema ./orgschema-demo
orgschema-query --schema ./orgschema-demo --check contracts
orgschema-query --schema ./orgschema-demo --check waste --format json
orgschema-query --schema ./orgschema-demo --level 3

Exit code is 1 if any selected error-severity check (contracts) reports a
violation, else 0 — making it usable in CI alongside ``orgschema-validate``.
"""

import argparse
import json
import sys
from pathlib import Path

from orgschema_framework.validate import (
validate_contract_satisfaction,
validate_waste,
)

# Each query maps to (level, severity, function).
# "error" severity affects the exit code; "warning" is advisory.
CHECKS = {
"contracts": (3, "error", validate_contract_satisfaction),
"waste": (6, "warning", validate_waste),
}

# Map --level N to the corresponding check name (deterministic queries only
# expose the levels that this module owns: 3 and 6).
LEVEL_TO_CHECK = {3: "contracts", 6: "waste"}


def run_query(root: Path, checks: list[str]) -> dict:
"""Run the requested deterministic checks and return a structured result.

The returned dict is stable and JSON-serializable:

{
"schema": "<abs path>",
"checks": [
{"check": "contracts", "level": 3, "severity": "error",
"violation_count": N, "violations": [..]},
...
],
"error_count": <sum of error-severity violations>,
"warning_count": <sum of warning-severity violations>,
"ok": <bool: no error-severity violations>
}
"""
results = []
error_count = 0
warning_count = 0

for name in checks:
level, severity, func = CHECKS[name]
violations = func(root)
results.append(
{
"check": name,
"level": level,
"severity": severity,
"violation_count": len(violations),
"violations": violations,
}
)
if severity == "error":
error_count += len(violations)
else:
warning_count += len(violations)

return {
"schema": str(root),
"checks": results,
"error_count": error_count,
"warning_count": warning_count,
"ok": error_count == 0,
}


def format_text(result: dict) -> str:
"""Render a query result as a human-readable structural report."""
lines = [f"=== OrgSchema deterministic query: {result['schema']} ==="]
for check in result["checks"]:
sev = check["severity"].upper()
header = (
f"\nLevel {check['level']} — {check['check']} "
f"({check['violation_count']} {sev} violations)"
)
lines.append(header)
if not check["violations"]:
lines.append(" PASSED")
else:
for v in check["violations"]:
lines.append(f" [{sev}] {v}")
lines.append(
f"\nSummary: {result['error_count']} errors, "
f"{result['warning_count']} warnings"
)
return "\n".join(lines)


def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="orgschema-query",
description=(
"Deterministic query interface over an OrgSchema directory "
"(no natural-language / LLM layer)."
),
)
parser.add_argument(
"--schema",
required=True,
type=Path,
help="Path to the orgschema specifications directory.",
)
parser.add_argument(
"--check",
choices=sorted(CHECKS) + ["all"],
default="all",
help="Which deterministic check(s) to run (default: all).",
)
parser.add_argument(
"--level",
type=int,
choices=sorted(LEVEL_TO_CHECK),
help="Run a single validation level by number (3=contracts, 6=waste). "
"Overrides --check when given.",
)
parser.add_argument(
"--format",
choices=["text", "json"],
default="text",
help="Output format (default: text).",
)
return parser


def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)

root = args.schema.resolve()
if not root.is_dir():
print(f"Error: {root} is not a directory", file=sys.stderr)
return 2

if args.level is not None:
checks = [LEVEL_TO_CHECK[args.level]]
elif args.check == "all":
checks = sorted(CHECKS)
else:
checks = [args.check]

result = run_query(root, checks)

if args.format == "json":
print(json.dumps(result, indent=2))
else:
print(format_text(result))

return 0 if result["ok"] else 1


if __name__ == "__main__":
sys.exit(main())
Loading
Loading