docs: refresh core kit docs

CONTRIBUTING.md

@@ -0,0 +1,37 @@
+# Contributing to OGN Core Kit
+
+Thanks for helping improve the OGN Core Kit adoption surface (runner, contracts, SDKs, workflow glue).
+
+## Quick dev setup (Python)
+
+```bash
+python -m pip install -e sdk/python[dev]
+pytest -q
+```
+
+## Contract / conformance changes
+
+Job Spec v1 is **additive-only**. Do not make optional fields required or change meanings in-place.
+
+When editing anything under `spec/v1/`:
+
+```bash
+python tools/validate_conformance.py
+```
+
+If you add or modify fixtures, keep them minimal and deterministic.
+
+## Public repo hygiene
+
+Before opening a PR:
+
+```bash
+./public_audit.sh
+git ls-files -s | awk '$1==120000{print}'
+```
+
+## Pull requests
+
+- Keep PRs small and contract-focused.
+- If you change runner behavior, add or update a focused test in `tests/`.
+- Avoid adding proprietary engine references, hard-coded machine paths, or secrets.

README.md

@@ -1,40 +1,125 @@
 # OGN Core Kit
 
-OGN Core Kit is the open adoption surface for OGN:
+OGN Core Kit is the open adoption surface for Omnis Genome Nexus (OGN).
 
-- Job Spec JSON v1 contract (docs + fixtures)
-- `ogn-runner`: a stable worker entrypoint for executing an OGN engine and uploading artifacts
-- SDKs and workflow adapters (Nextflow, etc.)
+It contains:
 
-This repository does **not** include the proprietary OGN engine implementation or GPU kernels.
-The engine is expected to be provided separately as a binary (e.g. `ogn_run`) or container image.
+- Job Spec JSON v1 contract (`spec/v1/jobspec.md` + `spec/v1/jobspec.schema.json` + fixtures)
+- `ogn-runner`: stable worker entrypoint that executes an OGN engine and uploads artifacts
+- Python package `ogn-sdk` (CLI `ogn`, SDK `ogn_sdk`, runner `ogn_runner`)
+- Workflow adapters (e.g. `pipelines/ogn_variant.nf`)
 
-## Quickstart (local runner)
+It does **not** include the proprietary OGN engine implementation, GPU kernels, or engine container images. The engine
+must be provided separately as a binary (e.g. `ogn_run`) or container image.
 
-1) Install the Python package:
+## Install
+
+### Python (CLI + runner + SDK)
+
+Requires Python 3.9+.
+
+From PyPI:
 
 ```bash
-pip install ./sdk/python
+python -m pip install ogn-sdk
 ```
 
-2) Point the runner at your engine binary (optional; defaults to `ogn_run` on PATH):
+From source (dev / editable):
+
+```bash
+python -m pip install -e sdk/python[dev]
+```
+
+This installs:
+
+- `ogn` (CLI wrapper; requires access to an engine binary such as `ogn_variant_runner`)
+- `ogn-runner` (worker entrypoint for Job Spec v1)
+- `ogn_sdk` / `ogn_runner` Python modules
+
+### Rust (SDK)
+
+The Rust crate lives at `sdk/rust/ogn-sdk`.
+
+- If/when published, install via crates.io.
+- Until then, consume it via a path dependency in a clone:
+
+```toml
+ogn-sdk = { path = "sdk/rust/ogn-sdk" }
+```
+
+## Quickstart: `ogn-runner` (Job Spec v1)
+
+`ogn-runner` accepts a jobspec path or `-` for stdin (the gateway worker contract is `ogn-runner -`).
+
+Engine selection:
 
 ```bash
 export OGN_ENGINE_BIN=/path/to/ogn_run
+# or: ogn-runner -E /path/to/ogn_run job_spec.json
 ```
 
-3) Run a Job Spec v1:
+Run from a file:
 
 ```bash
 ogn-runner job_spec.json
 ```
 
+Run from stdin:
+
+```bash
+cat job_spec.json | ogn-runner -
+```
+
+### Upload semantics
+
+For each `outputs.*` artifact:
+
+- If `put_url` is present, `ogn-runner` uploads via HTTP PUT to the presigned URL.
+- Otherwise it uploads to `uri` directly when supported:
+  - `s3://...` via `aws s3 cp` (AWS CLI required)
+  - `file://...` or a plain path via a local copy
+
+If an output is marked `optional: true`, upload failures are non-fatal.
+
 ## Contracts
 
-The canonical pilot contract lives in:
+Canonical artifacts:
 
 - `spec/v1/jobspec.md`
+- `spec/v1/jobspec.schema.json`
 - `spec/v1/conformance/`
 
+Schema-level validation:
+
+```bash
+python tools/validate_conformance.py
+```
+
 Unknown fields are ignored (additive-only). Breaking changes require a new schema version (v2+).
 
+## Development
+
+Run tests:
+
+```bash
+python -m pip install -e sdk/python[dev]
+pytest -q
+```
+
+Public repo hygiene:
+
+```bash
+./public_audit.sh
+```
+
+## License
+
+Apache 2.0; see `LICENSE` and `NOTICE`.
+
+## Security
+
+See `SECURITY.md`.
+
+## Contributing
+
+See `CONTRIBUTING.md`.

sdk/python/README.md

@@ -1,6 +1,14 @@
 # ogn-sdk (Python)
 
-`ogn-sdk` is a tiny convenience layer around the `ogn_run` binary. Install it with:
+`ogn-sdk` is a small set of Python wrappers and CLIs for integrating with the OGN engine.
+
+It includes:
+
+- `ogn_sdk`: a thin convenience layer around the `ogn_run` binary
+- `ogn` CLI: local orchestration helpers (doctor/setup/run/results)
+- `ogn-runner`: the stable Job Spec v1 worker entrypoint (`ogn-runner -`)
+
+Install it with:
 
 ```bash
 python3 -m pip install ogn-sdk

spec/v1/conformance/README.md

@@ -1,6 +1,6 @@
 # Job Spec v1 conformance fixtures
 
-These fixtures exist to keep `schema_version: "v1"` stable and enforceable.
+These fixtures exist to keep `schema_version: "v1"` stable and enforceable across the OGN ecosystem.
 
 Each `*.json` file is a single conformance case with this shape:
 
@@ -16,5 +16,5 @@ Each `*.json` file is a single conformance case with this shape:
 }
 ```
 
-The runner lives in `internal/jobspec/conformance_test.go`.
-
+In this repo, `tools/validate_conformance.py` performs schema-level validation for `kind: "jobspec"` cases.
+Timeline/state-machine semantics (`kind: "timeline"`) are enforced by the control plane implementation.

spec/v1/jobspec.md

@@ -1,6 +1,7 @@
 # Job Spec JSON v1 (frozen pilot contract)
 
-Job Spec v1 is the payload published on `ogn.run.submit.v1` (gateway → worker). It is validated by `internal/jobspec/v1.go` and used end-to-end in the pilot.
+Job Spec v1 is the payload produced by the OGN control plane (gateway) and consumed by workers (typically via
+`ogn-runner -`). It is intentionally strict and boring: this contract is the integration moat.
 
 ## Versioning policy
 
@@ -14,7 +15,10 @@ Job Spec v1 is the payload published on `ogn.run.submit.v1` (gateway → worker)
   - You MUST NOT remove fields.
 - Any breaking change requires a new `schema_version` (v2+) and continued support for v1 during the pilot.
 
-Conformance fixtures live in `spec/v1/conformance/` and are enforced by `internal/jobspec/conformance_test.go`.
+Conformance fixtures live in `spec/v1/conformance/`.
+
+- This repo includes schema-level checks via `tools/validate_conformance.py`.
+- The control plane (gateway) is responsible for enforcing additional semantic rules (e.g. timeline/state machines).
 
 ## Schema (high level)
 
@@ -44,4 +48,3 @@ Optional fields:
 - `outputs.*.uri` is the **stable artifact identity** (e.g. `s3://bucket/key`).
 - `outputs.*.put_url` is a short-lived presigned URL used only for upload.
 - Finalization events MUST reference stable URIs (never presigned URLs).
-