Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,40 @@ async def file_object_content(storage_id: str) -> str

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### file_object_content_by_id

```python
async def file_object_content_by_id(node_id: str) -> str
```

| Input | Output | Error |
| ----- | ------ | ----- |
| Valid node UUID (text file) | Raw file content (text) | — |
| Valid node UUID (binary file) | — | `JinjaFilterError("file_object_content_by_id", "binary content not supported...")` |
| `None` | — | `JinjaFilterError("file_object_content_by_id", "node_id is null", hint="...")` |
| `""` (empty) | — | `JinjaFilterError("file_object_content_by_id", "node_id is empty", hint="...")` |
| Permission denied (401/403) | — | `JinjaFilterError("file_object_content_by_id", "permission denied for node_id: {id}")` |
| No client provided | — | `JinjaFilterError("file_object_content_by_id", "requires InfrahubClient", hint="...")` |

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### file_object_content_by_hfid

```python
async def file_object_content_by_hfid(hfid: str | list[str], kind: str = "") -> str
```

| Input | Output | Error |
| ----- | ------ | ----- |
| Valid HFID + kind (text file) | Raw file content (text) | — |
| Valid HFID + kind (binary file) | — | `JinjaFilterError("file_object_content_by_hfid", "binary content not supported...")` |
| `None` | — | `JinjaFilterError("file_object_content_by_hfid", "hfid is null", hint="...")` |
| Missing `kind` argument | — | `JinjaFilterError("file_object_content_by_hfid", "'kind' argument is required", hint="...")` |
| Permission denied (401/403) | — | `JinjaFilterError("file_object_content_by_hfid", "permission denied for hfid: {hfid}")` |
| No client provided | — | `JinjaFilterError("file_object_content_by_hfid", "requires InfrahubClient", hint="...")` |

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### from_json

```python
Expand Down Expand Up @@ -73,9 +107,15 @@ def from_yaml(value: str) -> dict | list

Used by `artifact_content`. Returns plain text content.

### GET /api/files/by-storage-id/{storage_id} (new)
### File object endpoints

All three endpoints return file content with the node's `file_type` as content-type. The SDK validates that the content-type is text-based.

Used by `file_object_content`. Returns file content with appropriate content-type header.
| Endpoint | Used by | Identifier |
| -------- | ------- | ---------- |
| `GET /api/files/by-storage-id/{storage_id}` | `file_object_content` | storage_id |
| `GET /api/files/{node_id}` | `file_object_content_by_id` | node UUID |
| `GET /api/files/by-hfid/{kind}?hfid=...` | `file_object_content_by_hfid` | kind + HFID components |

**Accepted content-types** (text-based):

Expand Down
33 changes: 24 additions & 9 deletions dev/specs/infp-504-artifact-composition/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,23 +65,32 @@ class JinjaFilterError(JinjaTemplateError):

```python
class InfrahubFilters:
def __init__(self, client: InfrahubClient) -> None:
@classmethod
def get_filter_names(cls) -> tuple[str, ...]:
"""Discover filter names from public methods."""
...

def __init__(self, client: InfrahubClient | None = None) -> None:
self.client = client

async def artifact_content(self, storage_id: str) -> str:
"""Retrieve artifact content by storage_id."""
def _require_client(self, filter_name: str) -> InfrahubClient:
"""Raise JinjaFilterError if no client is available."""
...

async def file_object_content(self, storage_id: str) -> str:
"""Retrieve file object content by storage_id."""
...
async def artifact_content(self, storage_id: str) -> str: ...
async def file_object_content(self, storage_id: str) -> str: ...
async def file_object_content_by_id(self, node_id: str) -> str: ...
async def file_object_content_by_hfid(self, hfid: str | list[str], kind: str = "") -> str: ...
```

**Key design decisions**:

- Client is optional — `InfrahubFilters` is always instantiated, each method checks for a client at call time via `_require_client()`
- `get_filter_names()` discovers client-dependent filter names automatically from all public methods — adding a new filter only requires adding a method
- Methods are `async` — Jinja2's `auto_await` handles them in async rendering mode
- Holds an `InfrahubClient` (async only), not `InfrahubClientSync`
- Each method validates inputs and catches `AuthenticationError` to wrap in `JinjaFilterError`
- File object retrieval is split into 3 filters matching the server's 3 endpoints (`by-storage-id`, `by-id`, `by-hfid`)

## Modified Entities

Expand Down Expand Up @@ -114,9 +123,9 @@ def set_client(self, client: InfrahubClient) -> None:

**Purpose**: Deferred client injection — allows creating a `Jinja2Template` first and adding the client later. Also supports replacing a previously set client.

- Calls `_register_client_filters(client)` to bind real filter methods
- Updates `self._infrahub_filters.client` on the existing `InfrahubFilters` instance (no re-registration needed since the bound methods are already registered)
- If the Jinja2 environment was already created, patches it in place
- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time
- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time via `_require_client()`

### Jinja2Template.validate() (modified signature)

Expand All @@ -143,7 +152,11 @@ async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = No
...
```

**API endpoint**: `GET /api/files/by-storage-id/{storage_id}`
**API endpoints**:

- `GET /api/files/by-storage-id/{storage_id}` — used by `file_object_content`
- `GET /api/files/{node_id}` — used by `file_object_content_by_id`
- `GET /api/files/by-hfid/{kind}?hfid=...` — used by `file_object_content_by_hfid`

**Content-type check**: Allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`. Reject all others.

Expand All @@ -155,6 +168,8 @@ async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = No
# Infrahub client-dependent filters (worker and local contexts)
FilterDefinition("artifact_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content_by_hfid", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content_by_id", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),

# Parsing filters (trusted, all contexts)
FilterDefinition("from_json", allowed_contexts=ExecutionContext.ALL, source="infrahub"),
Expand Down
12 changes: 10 additions & 2 deletions dev/specs/infp-504-artifact-composition/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,17 @@ hostname {{ device.hostname.value }}

### Inline file object content

File objects can be retrieved by storage ID, node UUID, or HFID:

```jinja2
{% set file_content = file_object.storage_id.value | file_object_content %}
{{ file_content }}
{# By storage_id (most common) #}
{{ file_object.storage_id.value | file_object_content }}
{# By node UUID #}
{{ file_object.id | file_object_content_by_id }}
{# By Human-Friendly ID #}
{{ hfid_components | file_object_content_by_hfid(kind="NetworkCircuitContract") }}
```

### Parse structured content
Expand Down
17 changes: 12 additions & 5 deletions dev/specs/infp-504-artifact-composition/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,19 +94,26 @@ The existing single `restricted: bool` parameter on `validate()` is insufficient
### Functional requirements

- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). Additionally, `Jinja2Template` MUST expose a `set_client(client)` method for deferred client injection, allowing the template to be created first and the client added later. `InfrahubClientSync` is not supported.
- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided (via `__init__` or `set_client()`) and registers its filters into the Jinja2 environment.
- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold an optional client reference and expose the Infrahub-specific filter callable methods. `InfrahubFilters` is always instantiated by `Jinja2Template` (even without a client); each filter method checks for a client at call time and raises `JinjaFilterError` if none is available. `set_client()` updates the existing `InfrahubFilters` instance rather than creating a new one. `InfrahubFilters.get_filter_names()` discovers client-dependent filter names automatically from all public methods on the class.
- **FR-003**: The system MUST provide an `artifact_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced artifact, using the artifact-specific API path.
- **FR-004**: The system MUST provide a `file_object_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced file object, using the file-object-specific API path or metadata handling — this implementation is distinct from `artifact_content`.
- **FR-005**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when the input `storage_id` is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, `file_object_content` MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`).
- **FR-006**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when invoked and no `InfrahubClient` was supplied to `Jinja2Template` at construction time. The error message MUST name the filter and explain that an `InfrahubClient` is required.
- **FR-007**: Both `artifact_content` and `file_object_content` MUST be registered with `allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers) and `LOCAL` context (CLI/unrestricted rendering). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
- **FR-004**: The system MUST provide three file object content filters, each retrieving content via a different identifier:
- `file_object_content` — accepts a `storage_id` string, uses `GET /api/files/by-storage-id/{storage_id}`
- `file_object_content_by_id` — accepts a node UUID string, uses `GET /api/files/{node_id}`
- `file_object_content_by_hfid` — accepts an HFID string or list and a required `kind` argument, uses `GET /api/files/by-hfid/{kind}?hfid=...`
All three share the same binary content-type rejection and error handling behavior.
- **FR-005**: All client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) MUST raise `JinjaFilterError` when the input identifier is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, all file object filters MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`). `file_object_content_by_hfid` MUST also raise `JinjaFilterError` when the `kind` argument is missing.
- **FR-006**: All client-dependent filters MUST raise `JinjaFilterError` when invoked and no `InfrahubClient` was supplied to `Jinja2Template` at construction time. The error message MUST name the filter and explain that an `InfrahubClient` is required.
- **FR-007**: All client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) MUST be registered with `allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers) and `LOCAL` context (CLI/unrestricted rendering). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
- **FR-008**: The system MUST provide `from_json` and `from_yaml` Jinja2 filters (adding them only if not already present in the environment) that parse a string into a Python dict/list. Applying them to an empty string MUST return an empty dict without raising. Applying them to malformed content MUST raise `JinjaFilterError`.
- **FR-009**: `from_json` and `from_yaml` MUST be registered as trusted filters (`trusted=True`) since they perform no external I/O.
- **FR-010**: All new filters MUST work correctly with `InfrahubClient` (async). `InfrahubClientSync` is not a supported client type for `Jinja2Template`. Both the sandboxed environment (string-based templates) and the file-based environment MUST have `enable_async=True` to support async filter callables via Jinja2's `auto_await`.
- **FR-011**: All `JinjaFilterError` instances MUST carry an actionable error message that identifies the filter name, the cause of failure, and any remediation hint (for example: "artifact_content requires an InfrahubClient — pass one via Jinja2Template(client=...)").
- **FR-012**: A new `JinjaFilterError` exception class MUST be added to `infrahub_sdk/template/exceptions.py` as a subclass of `JinjaTemplateError`.
- **FR-013**: Documentation MUST include a Python transform example demonstrating artifact content retrieval via `client.object_store.get(identifier=storage_id)`. No new SDK convenience method will be added.
- **FR-014**: If the current user isn't allowed due to a permission denied error to query for the artifact or object file the filter should catch such permission error and raise a Jinja2 error specifically related to the permission issue.
- **FR-015**: The auto-generated templating reference (`sdk_template_reference.j2` and `_generate_infrahub_sdk_template_documentation()`) MUST be updated to include `INFRAHUB_FILTERS` as a third section ("Infrahub filters"). The table MUST show which execution contexts each filter is allowed in (`CORE`, `WORKER`, `LOCAL`) rather than only a binary "Trusted" column.
- **FR-016**: The templating reference documentation MUST explain the `ExecutionContext` model — what `CORE`, `WORKER`, and `LOCAL` contexts mean, how they map to Infrahub's execution environments (computed attributes, Prefect workers, local CLI), and how `validate(context=...)` is used to enforce filter restrictions.
- **FR-017**: Usage examples for the new Jinja2 filters MUST be included in the SDK documentation, covering at minimum: `artifact_content` with a storage_id, `file_object_content` by storage_id, `file_object_content_by_id` by node UUID, `file_object_content_by_hfid` with `kind` argument, and `from_json`/`from_yaml` chaining.

### Key entities

Expand Down
Loading