Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ result.xml
test/vendor/
tmp/

osapi
/osapi
36 changes: 16 additions & 20 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ go test -run TestName -v ./internal/job/... # Run a single test
- **`internal/agent/`** - Node agent: consumer/handler/processor pipeline for job execution
- **`internal/provider/`** - Operation implementations: `node/{host,disk,mem,load}`, `network/{dns,ping}`
- **`internal/config/`** - Viper-based config from `osapi.yaml`
- **`osapi-sdk`** - External SDK for programmatic REST API access (sibling repo, linked via `replace` in `go.mod`)
- **`pkg/sdk/`** - Go SDK for programmatic REST API access (`osapi/` client library, `orchestrator/` DAG runner)
- Shared `nats-client` and `nats-server` are sibling repos linked via `replace` in `go.mod`
- **`github/`** - Temporary GitHub org config tooling (`repos.json` for declarative repo settings, `sync.sh` for drift detection via `gh` CLI). Untracked and intended to move to its own repo.

Expand Down Expand Up @@ -171,29 +171,25 @@ Create `internal/api/{domain}/`:

### Step 5: Update SDK

The `osapi-sdk` (sibling repo) provides the generated HTTP client used by
the CLI. The SDK syncs its `api.yaml` files from this repo via `gilt`
overlay (configured in `osapi-sdk/.gilt.yml`). When `just generate` runs
in the SDK, gilt pulls the latest specs from osapi's `main` branch and
regenerates the client code.
The SDK client library lives in `pkg/sdk/osapi/`. Its generated HTTP client
uses the same combined OpenAPI spec as the server
(`internal/api/gen/api.yaml`).

**When adding a new API domain:**
**When modifying existing API specs:**

1. Add the domain's `api.yaml` to `osapi-sdk/pkg/osapi/gen/{domain}/`
2. Run `just generate` in the SDK repo to regenerate the merged spec and
client code
3. Add a service wrapper in `osapi-sdk/pkg/osapi/{domain}.go`
1. Make changes to `internal/api/{domain}/gen/api.yaml` in this repo
2. Run `just generate` to regenerate server code (this also regenerates the
combined spec via `redocly join`)
3. Run `go generate ./pkg/sdk/osapi/gen/...` to regenerate the SDK client
4. Update the SDK service wrappers in `pkg/sdk/osapi/{domain}.go` if new
response codes were added
5. Update CLI switch blocks in `cmd/` if new response codes were added

**When modifying existing API specs** (adding responses, parameters, or
schemas to existing endpoints):
**When adding a new API domain:**

1. Make changes to `internal/api/{domain}/gen/api.yaml` in this repo
2. Run `just generate` here to regenerate server code
3. After merging to `main`, run `just generate` in `osapi-sdk` — gilt
will pull the updated specs and regenerate the client
4. Update the SDK service wrappers and CLI switch blocks if new response
codes were added (e.g., adding a 404 response requires a
`case http.StatusNotFound:` in the CLI)
1. Add a service wrapper in `pkg/sdk/osapi/{domain}.go`
2. Run `go generate ./pkg/sdk/osapi/gen/...` to pick up the new domain's
spec from the combined `api.yaml`

### Step 6: CLI Commands

Expand Down
39 changes: 2 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,56 +22,23 @@ them to be used as appliances.

<img src="asset/demo.gif" alt="OSAPI demo" />

## ✨ Features

| | |
|---|---|
| 🖥️ **[Node Management][]** | Hostname, uptime, OS info, disk, memory, load |
| 🌐 **[Network Management][]** | DNS read/update, ping |
| ⚙️ **[Command Execution][]** | Remote exec and shell across managed hosts |
| 📁 **[File Management][]** | Upload, deploy, and template files with SHA-based idempotency |
| 📊 **[System Facts][]** | Agent-collected system facts — architecture, kernel, FQDN, CPUs, network interfaces, service/package manager |
| 🔄 **[Agent Lifecycle][]** | Node conditions (memory, disk, load pressure), graceful drain/cordon for maintenance |
| ⚡ **[Async Job System][]** | NATS JetStream with KV-first architecture — broadcast, load-balanced, and label-based routing across hosts |
| 💚 **[Health][] & [Metrics][]** | Liveness, readiness, system status endpoints, Prometheus `/metrics` |
| 📋 **[Audit Logging][]** | Structured API audit trail in NATS KV with 30-day retention and admin-only read access |
| 🔐 **[Auth & RBAC][]** | JWT with fine-grained `resource:verb` permissions, built-in and custom roles, direct permission grants |
| 🔍 **[Distributed Tracing][]** | OpenTelemetry with trace context propagation across HTTP and NATS |
| 🖥️ **CLI Parity** | Every API operation has a CLI equivalent with `--json` for scripting |
| 🏢 **Multi-Tenant** | Namespace isolation lets multiple deployments share a single NATS cluster |

[Node Management]: https://osapi-io.github.io/osapi/sidebar/features/node-management
[Network Management]: https://osapi-io.github.io/osapi/sidebar/features/network-management
[Command Execution]: https://osapi-io.github.io/osapi/sidebar/features/command-execution
[File Management]: https://osapi-io.github.io/osapi/sidebar/features/file-management
[Async Job System]: https://osapi-io.github.io/osapi/sidebar/features/job-system
[Health]: https://osapi-io.github.io/osapi/sidebar/features/health-checks
[Metrics]: https://osapi-io.github.io/osapi/sidebar/features/metrics
[Audit Logging]: https://osapi-io.github.io/osapi/sidebar/features/audit-logging
[Auth & RBAC]: https://osapi-io.github.io/osapi/sidebar/features/authentication
[Distributed Tracing]: https://osapi-io.github.io/osapi/sidebar/features/distributed-tracing

## 📖 Documentation

[Features][] | [Architecture][] | [Getting Started][] | [API][] | [Usage][] | [Roadmap][]
[Getting Started][] | [API][] | [Usage][] | [SDK][]

[Features]: https://osapi-io.github.io/osapi/category/features
[Architecture]: https://osapi-io.github.io/osapi/sidebar/architecture
[Getting Started]: https://osapi-io.github.io/osapi/
[API]: https://osapi-io.github.io/osapi/category/api
[Usage]: https://osapi-io.github.io/osapi/sidebar/usage
[Roadmap]: https://osapi-io.github.io/osapi/sidebar/development/roadmap
[SDK]: https://osapi-io.github.io/osapi/sidebar/sdk/sdk

## 🔗 Sister Projects

| Project | Description |
| --- | --- |
| [osapi-sdk][] | Go SDK for OSAPI — client library and orchestration primitives |
| [osapi-orchestrator][] | A Go package for orchestrating operations across OSAPI-managed hosts — typed operations, chaining, conditions, and result decoding built on top of the osapi-sdk engine |
| [nats-client][] | A Go package for connecting to and interacting with a NATS server |
| [nats-server][] | A Go package for running an embedded NATS server |

[osapi-sdk]: https://github.com/osapi-io/osapi-sdk
[osapi-orchestrator]: https://github.com/osapi-io/osapi-orchestrator
[nats-client]: https://github.com/osapi-io/nats-client
[nats-server]: https://github.com/osapi-io/nats-server
Expand All @@ -80,6 +47,4 @@ them to be used as appliances.

The [MIT][] License.

[Agent Lifecycle]: https://osapi-io.github.io/osapi/sidebar/features/agent-lifecycle
[System Facts]: https://osapi-io.github.io/osapi/sidebar/features/node-management
[MIT]: LICENSE
2 changes: 1 addition & 1 deletion cmd/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import (
"context"
"log/slog"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"
"github.com/spf13/viper"

Expand Down
2 changes: 1 addition & 1 deletion cmd/client_agent_get.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"strings"
"time"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_audit_export.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"fmt"
"strconv"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/audit/export"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_file_upload.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ import (

"github.com/spf13/cobra"

osapi "github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/internal/cli"
osapi "github.com/retr0h/osapi/pkg/sdk/osapi"
)

// clientFileUploadCmd represents the clientFileUpload command.
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_health_status.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ package cmd
import (
"fmt"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_job_list.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
"strings"
"time"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_job_run.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import (
"os"
"time"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_node_command_exec.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"os"
"strconv"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_node_command_shell.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"os"
"strconv"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_node_file_deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import (
"fmt"
"strings"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
2 changes: 1 addition & 1 deletion cmd/client_node_status_get.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ package cmd
import (
"fmt"

"github.com/osapi-io/osapi-sdk/pkg/osapi"
"github.com/retr0h/osapi/pkg/sdk/osapi"
"github.com/spf13/cobra"

"github.com/retr0h/osapi/internal/cli"
Expand Down
7 changes: 0 additions & 7 deletions docs/docs/gen/api/agent-management-api.info.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@ import Export from "@theme/ApiExplorer/Export";
>
</span>

<Export
url={"https://github.com/osapi-io/osapi-sdk/blob/main/pkg/osapi/gen/api.yaml"}
proxy={undefined}
>

</Export>

<Heading
as={"h1"}
className={"openapi__heading"}
Expand Down
95 changes: 93 additions & 2 deletions docs/docs/gen/api/drain-agent.api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "Stop the agent from accepting new jobs. In-flight jobs continue to
sidebar_label: "Drain an agent"
hide_title: true
hide_table_of_contents: true
api: eJztVsFu20YQ/ZXFnFqAlpQ0PZQ3FakBBQhixAp6cAVjxB2Ja5O7zOzQjkrw34NZyjJt2UEOCdACPmmX3Jl5895w9ToIDTGKC35hIQfL6Px8S14gA0uxYNfoS8jhXEJjpCSD+tpsONQGi4IacX5rPN2aq7COE7PwJ5vKbUtJe1MEL863ZCSYItRNRZpv8o+HDAS3EfILSAUv36PHLdW6nJ8tLlOZywO8CKsMIhUtO9lBftHBn4RMPG+l1BzpeH7LTghW/SqDBhlrEuKYTnusCXIoQ5S0zMBpVw1KCRkwfW4dk4VcuKUMYlFSjZB3ILtG46Kw81voH7OyLMnc5TRhMyJIgklsTkDRMMUm+EhRc76ezfTnYaZEwhBinHfiUMhOIANlUAXJO8CmqVyR+JheRQ3rjqGG9RUVql/Dyp64oWhNMeKWnuipH/d/cTi46nt99Wb26hjsJ4+tlIHdv2TNiZmfLcw17cwhzQ9DTcyBjzE/lmFuRvs7HVKskRLFhKJomQc+6QvqGEIOp+gqsqoUk7CjGzJRUNo4GYQWdFX8juLWOl1iZfYxBtehlXsQT5a1w0fhSW4DXxtxNYVWUuki2LFQzgttiY8KLw9NasCDIr/PZiqeOEkl/9JTH/czCHfC/nYs7GngtbOWvDkxCx/bzcYVTgezIa5djOlLfFH3/6Dum+fuGB/EbELrf+Tt8qLkT1Tyj+eUxIoJ7c44n65eikI2UUMv0v73pe0zqEnKoMarCTExr3Ykh2myENPuzln00+QLQB0Q3wyWZmSHzlXJQayxKTqAL0Ua2Jsa3a/TIcj2i9PANQrk8O7vZXIDzm9CCt9DH4bt3p/p/z1koEAGFl5NZpOZsqZt1JjGa2+43iZDg35wRY/56+6H9KcYzKF/oS8ybSplsM+g5UoLD1TvjSNkkI+s4cD2Kkt2UQ913RojfeKq7/Xx55Z4N2hwg+xwrTRddGBd1LWFfINVpG80+8vHvVX61XyPhXymlf1D9DuVA6tWd5DBNe3GXrdf9RmUhJY4wRxezxO3o8CjS0KN62FCzz6cLyEDfDhaj0YppX8SVdcNJ5bhmnzfH0CK7hVh338FZL5baA==
api: eJztV01v4zYQ/SvEnFqA/ki6PlTAHly0AVy0aLDxoofUCMbSWGIikVpylNgV9N+LoWTHiZNiD7uHADmZFGc4M++NyccWXE0e2Ti7yCCBzKOx85wsg4aMQupNLYuQwBW7WnFBCmVZbbyrFKYp1Wxsriw9qFu3DmO1sKNNafKC41ylzrKxDSl2KnVVXZLsN/7HggbGPEByDTHgzZ9oMadKhvPLxU0Mc3NIL8BKQ6C08YZ3kFy38AuhJz9vuJA9onny4A0TrLqVhho9VsTkQ7S2WBEkULjAcajBSFU1cgEaPH1pjKcMEvYNadiOHNZmlLqMcrIj2rLHUZ9uC/dYmgxZttv76crYj2e6wu3H89kMOg0hLahCMeddLaaBvbE5aKiM/YNsLmmfaahwu5+dz2bdc9SXBal9zsptjghgpyJbY5BqPYXa2UAxwfPpVH6e7hRB7l2UsYYNMmVj0CAMCeFJC1jXpUkj3pPbIG7taSFufUup9EfthR02fdCKQsCcTivuumN8rw+Gq66TpQ8vJbuwEeRD6d8wT/Le+Zd4eQaXOprvkY++igtk5dK08b5HkLYojQ0JXKApKRNuPLE3dE8qMHITxhCpZTRl+IrgWWZkiKUafBSuXcOPSbwYNuv/Zpb4wfk7xaYi13AMLZ18FNdYppz8SeDloUhxeBJkNp0KXWw4hvxNrD4NXQd7Ks9OqfxsseHCefMvZWqk5pcLdUc7deiId2LfArE/nRJ74fzaZBlZNVILG5rNxqRGzpiafGVCiIf2O7tvgd0Pr10X1rHauMZ+y4vincnvyOTPrzGJpSfMdsrYePRSYMoiNO936xugttNQERdONHrtQkRelGsCk6gGJ+1eKXWTKPFAxLK/79XvkXK+EiZ7so718yH5grmGQb3KfB2NQA+DC+crZEjg97+XUdgZu3HRfUi9b7ZHKS/3PWiQRHoUzsbT8VRQkzIqjO01aPNfozZF2wvc5/i1j036Xd4iff1MW57UpSDYaWh8KYF7qIc3BmhIjl4RPdorHV8WYtS2awz02ZddJ5+/NOR3PQf36A2uBabrFjITZJxBssEy0P8U+8OnQSr9qL7mNfBKKcNHtDuhA8tGZqDhjnbHz6Ju1WkoCDPyMc1+eR6xPXI8OSTkDXLo0Mu/rpagAZ+21rNWitu/mFXb9hZLd0e26w5Jsswlw677Dx0fG34=
sidebar_class_name: "post api-method"
info_path: gen/api/agent-management-api
custom_edit_url: null
Expand Down Expand Up @@ -69,7 +69,7 @@ Stop the agent from accepting new jobs. In-flight jobs continue to completion.
<ul>
<ParamsItem
className={"paramsItem"}
param={{"name":"hostname","in":"path","required":true,"schema":{"type":"string"},"description":"The hostname of the agent to drain."}}
param={{"name":"hostname","in":"path","required":true,"x-oapi-codegen-extra-tags":{"validate":"required,min=1,max=255"},"schema":{"type":"string","minLength":1,"maxLength":255},"description":"The hostname of the agent to drain."}}
>

</ParamsItem>
Expand Down Expand Up @@ -155,6 +155,97 @@ Stop the agent from accepting new jobs. In-flight jobs continue to completion.
</MimeTabs>
</div>
</TabItem><TabItem
label={"400"}
value={"400"}
>
<div>


Invalid hostname.


</div><div>
<MimeTabs
className={"openapi-tabs__mime"}
schemaType={"response"}
>
<TabItem
label={"application/json"}
value={"application/json"}
>
<SchemaTabs
className={"openapi-tabs__schema"}
>
<TabItem
label={"Schema"}
value={"Schema"}
>
<details
style={{}}
className={"openapi-markdown__details response"}
data-collapsed={false}
open={true}
>
<summary
style={{}}
className={"openapi-markdown__details-summary-response"}
>
<strong>
Schema
</strong>
</summary><div
style={{"textAlign":"left","marginLeft":"1rem"}}
>

</div><ul
style={{"marginLeft":"1rem"}}
>
<SchemaItem
collapsible={false}
name={"error"}
required={false}
schemaName={"string"}
qualifierMessage={undefined}
schema={{"type":"string","description":"A description of the error that occurred.","example":"Failed to retrieve status."}}
>

</SchemaItem><SchemaItem
collapsible={false}
name={"details"}
required={false}
schemaName={"string"}
qualifierMessage={undefined}
schema={{"type":"string","description":"Additional details about the error.","example":"Failed due to network timeout."}}
>

</SchemaItem><SchemaItem
collapsible={false}
name={"code"}
required={false}
schemaName={"integer"}
qualifierMessage={undefined}
schema={{"type":"integer","description":"The error code.","example":500}}
>

</SchemaItem>
</ul>
</details>
</TabItem><TabItem
label={"Example (from schema)"}
value={"Example (from schema)"}
>
<ResponseSamples
responseExample={"{\n \"error\": \"Failed to retrieve status.\",\n \"details\": \"Failed due to network timeout.\",\n \"code\": 500\n}"}
language={"json"}
>

</ResponseSamples>
</TabItem>
</SchemaTabs>
</TabItem>
</MimeTabs>
</div>
</TabItem><TabItem
label={"401"}
value={"401"}
>
Expand Down
Loading
Loading