Skip to content

Per-node KV persistence instead of whole-tree JSON blob #8

@devangb3

Description

@devangb3

Every folder creation or file upload rewrites the entire user tree to the KV store. For a tree with 10,000 nodes this is tens of KB per operation. Per-node storage with parent-keyed reads is closer to how a real filesystem works.

Current state:
set_kv(username + " ROOT", root.to_json()) serializes the whole tree on every write.

Proposed implementation:

  1. New KV layout: "<user>:node:<path_hash>" → node JSON (one node per key), plus "<user>:node:<path>:children" → sorted list of child names. Or: "<user>:tree" still exists as a small index of root-level names only.
  2. /create-folder writes one new key for the folder + updates the parent's children list (two writes instead of one, both small).
  3. /delete removes the subtree (N writes) but is still faster in the common case of small deletes.
  4. Write an offline migration reading the old JSON blob into the new layout.
  5. Trade-off: listing a large folder is now N reads; add a single "listing" key per folder to compensate.

Files likely affected:

  • backend/node.py (add save/load methods)
  • backend/RSDB_kv_service.py (no changes needed — it's still KV)
  • app.py (every mutating route)
  • scripts/migrate_tree_to_per_node.py (new)
  • Tests.

Acceptance criteria:

  • Average bytes written per folder operation drops from ~O(tree) to O(1).
  • Round-trip test: old format migrated, all nodes readable, tree structure identical.
  • New users use the per-node layout from first write.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions