docs: port missing best-practices content from portal

docs/concepts/app-architecture.md

@@ -71,6 +71,19 @@ For maximum throughput, distribute canisters across multiple [subnets](network-o
 
 **Trade-offs:** cross-subnet calls have higher latency and bandwidth limits. You need to design data partitioning carefully.
 
+### Canister-per-user
+
+Each user gets their own canister that they control. The main application canister orchestrates user canisters to implement the application's functionality. Since users control their canisters, they can install their own code, decide how to participate in the application, and determine what data to share with the main canister.
+
+**When to use:** only when user sovereignty over data is a core product requirement and you accept the significant development cost.
+
+**Things to know:**
+- The main canister must treat every user canister as potentially malicious. Any code path that interacts with user canisters must assume adversarial behavior and be hardened against it.
+- Development cost is very high. Handling all possible actions from potentially malicious user canisters requires expert knowledge of the ICP security and messaging model.
+- Spawning a user canister (via an actor class in Motoko or a management canister `create_canister` call in Rust) carries the same cost as a fresh canister install. Do it once per user at account creation, never on a hot call path.
+- **There is no known successful end-to-end implementation of the full canister-per-user vision.** A few projects have explored variations, but the architecture remains experimental.
+- Common misconception: canister-per-user is not the most scalable pattern. Canister-per-subnet is more performant because it can utilize multiple subnets without the overhead of managing a large number of small canisters.
+
 ## Data storage
 
 Canisters store data in heap memory during execution and can persist data across upgrades using [stable memory](orthogonal-persistence.md#stable-memory): there is no external database. Libraries provide familiar data-structure abstractions on top of raw stable memory:
@@ -98,6 +111,7 @@ Start with a [single canister](#single-canister): it is the right choice for mos
 | Does the app have a web UI? | Add an [asset canister](#frontend-options) | Backend-only canister |
 | Do you need separation of concerns or hit platform limits? | [Canister-per-service](#canister-per-service) | Stay with a single canister |
 | Do you need to scale beyond one subnet? | [Canister-per-subnet](#canister-per-subnet) | Stay on one subnet |
+| Is user sovereignty over data a core requirement and are you prepared for high dev cost? | [Canister-per-user](#canister-per-user) (experimental) | None of the above |
 
 Start with the simplest architecture that meets your requirements. You can always split a canister into multiple canisters later: it is much harder to merge canisters that were split prematurely.
 
@@ -108,4 +122,4 @@ Start with the simplest architecture that meets your requirements. You can alway
 - [Asset canister](../guides/frontends/asset-canister.md): frontend deployment
 - [Canisters](canisters.md): canister internals
 
-<!-- Upstream: informed by dfinity/portal docs/building-apps/best-practices/application-architectures.mdx, docs/building-apps/getting-started/app-architecture.mdx -->
+<!-- Upstream: informed by dfinity/portal docs/building-apps/best-practices/application-architectures.mdx, docs/building-apps/getting-started/app-architecture.mdx; canister-per-user section ported from application-architectures.mdx -->

docs/guides/backends/data-persistence.mdx

@@ -571,11 +571,88 @@ icp canister call backend get_user '(0 : nat64)'
 
 If the count drops to 0 after upgrade, the data is not in stable memory. Review your storage declarations.
 
+## Storage recommendations
+
+### Choose the right storage type
+
+| Memory type | Max size | Persists across upgrades | Best for |
+|-------------|----------|--------------------------|----------|
+| Heap | 4 GiB | No (Rust) / Yes (Motoko `persistent actor`) | Frequently accessed data, caches, ephemeral computation |
+| Stable | 500 GiB | Yes | All important data, large datasets, anything that must survive upgrades |
+
+The practical rule: **use stable structures directly for any data that matters**. Avoid relying on `pre_upgrade` / `post_upgrade` hooks to serialize heap data to stable memory. Serializing large heap state during an upgrade can hit the instruction limit and trap, leaving the canister on the old code. Data in stable structures is already in stable memory from the first write — no serialization step required on upgrade.
+
+For Motoko, `persistent actor` makes all `let` and `var` declarations persistent automatically. There is no need to choose manually between heap and stable memory.
+
+### Language-specific recommendations
+
+<Tabs syncKey="lang">
+<TabItem label="Motoko">
+
+**Choose efficient data structures.**
+
+The `mo:core` library provides stable-friendly, performant data structures. Use these in preference to the legacy `mo:base` equivalents:
+
+| Use case | `mo:core` type | Replaces (`mo:base`) |
+|----------|----------------|----------------------|
+| Key-value map | `Map` | `HashMap`, `TrieMap`, `Trie`, `RBTree` |
+| Dynamic sequence | `List` | `Buffer` |
+| Double-ended queue | `Queue` | `Deque` |
+| Ordered map | `pure/Map` | `OrderedMap` |
+| Ordered set | `pure/Set` | `OrderedSet` |
+
+`Map` avoids the automatic resizing overhead that `HashMap` incurs on growth. `List` handles dynamic sequences without the fragile array-copy pattern of `Buffer`.
+
+**Prefer `Blob` over `[Nat8]` for binary data.**
+
+`Blob` is 4× more compact than `[Nat8]` and produces significantly less GC pressure. Use `Blob` for binary assets, cryptographic values, and anywhere you would send or receive `vec nat8` in Candid. Store large `Blob`s in stable memory.
+
+**Use `compacting-gc` for append-only workloads (classical persistence only).**
+
+If your canister grows the heap by appending data without frequent deletions, the `--compacting-gc` flag allows the GC to handle larger heaps and reduces the cost of copying large, stationary objects. Enable it in `icp.yaml` under canister build args. Note: `--compacting-gc` applies only to the legacy classical persistence mode (`--legacy-persistence`); it is not used with the default enhanced orthogonal persistence.
+
+</TabItem>
+<TabItem label="Rust">
+
+**Exercise caution with `Vec<u8>` and `String` in state serialization.**
+
+If you serialize/deserialize state that contains `Vec<u8>` or `String` values, Rust's memory layout requires copying each value during encoding and decoding. For large state, this increases the instruction cost significantly. Prefer `StableBTreeMap<Vec<u8>, ...>` (or a typed key) backed directly by stable memory over serializing heap collections on upgrade.
+
+**Use `ic-stable-structures` for all persistent state.**
+
+Put all important data in `StableBTreeMap`, `StableCell`, or `StableLog` from the start. This avoids the `pre_upgrade` serialization problem entirely. See [Implementing Storable for custom types](#store-data-durably) above for the correct pattern.
+
+For reference on effective Rust canister patterns, see [Effective Rust Canisters](https://mmapped.blog/posts/01-effective-rust-canisters.html) and [How to audit an Internet Computer canister](https://www.joachim-breitner.de/blog/788-How_to_audit_an_Internet_Computer_canister).
+
+</TabItem>
+</Tabs>
+
+### Implement state backup mechanisms
+
+Even with stable memory, consider implementing explicit backup mechanisms for state that would be catastrophic to lose. This protects against:
+
+- Accidental reinstall (which wipes stable memory)
+- Bugs in upgrade hooks that corrupt the stable layout
+- Application-level errors that require rollback
+
+Common approaches include exporting a snapshot of canister state to a Blob that can be stored externally, or using canister [snapshots](../canister-management/snapshots.md) to checkpoint state before an upgrade.
+
+### Transaction history storage
+
+If your application needs to maintain a history of transactions or events, avoid storing unbounded logs in the same canister as your main application state. Options:
+
+- **Dedicated logging canister.** A separate canister that accepts append-only log entries reduces load on the main canister and keeps the history size from affecting upgrade cost.
+- **`StableLog` (Rust).** For canisters that can accommodate history growth, `StableLog` from `ic-stable-structures` provides an append-only log directly in stable memory.
+- **External history services.** Services like [CAP](https://cap.ooo/) maintain transaction provenance records that integrate with explorers and wallets, which is useful for [digital asset standards](../../references/digital-asset-standards.md) compliance.
+
+Be aware that inter-canister calls to a logging service add latency and cycle cost. Size the logging approach to the transaction volume you expect.
+
 ## Related
 
 - [Orthogonal Persistence](../../concepts/orthogonal-persistence.md): conceptual explanation of heap vs. stable memory
 - [Canister Lifecycle](../canister-management/lifecycle.md#what-happens-during-an-upgrade): upgrade hooks and canister lifecycle
 - [Stable Structures (Rust)](../../languages/rust/stable-structures.md): deep dive into `ic-stable-structures`
+- [Canister snapshots](../canister-management/snapshots.md): checkpoint canister state before upgrades
 - [Motoko](../../languages/motoko/index.md): Motoko language overview and persistence model
 
 {/* Upstream: informed by dfinity/portal docs/building-apps/canister-management/storage.mdx, docs/building-apps/best-practices/storage.mdx, docs/building-apps/best-practices/idempotency.mdx */}

docs/guides/canister-calls/idempotency.md

@@ -1,6 +1,8 @@
 ---
 title: "Safe Retries and Idempotency"
 description: "Design idempotent canister APIs to enable safe retries for ingress calls and bounded-wait inter-canister calls, preventing double-spend and other correctness issues."
+sidebar:
+  order: 5
 ---
 
 In the case of network issues or other unexpected behavior, ICP clients (such as agents) that issue ingress update calls may be unable to determine whether their ingress request has been processed. For example, this can happen if the client loses connection until after the request's ingress expiry ends and the request's status is removed from the system state tree.

docs/guides/canister-calls/inter-canister-calls.mdx

@@ -399,6 +399,7 @@ Calls between canisters on the same subnet complete within a single round. Cross
 ## Next steps
 
 - [Parallel inter-canister calls](parallel-inter-canister-calls.md): make multiple calls concurrently and use composite queries for efficient read patterns
+- [Paginating query results](pagination.md): cursor-based pagination for mutable datasets that avoids duplicates and skipped items
 - [Candid](candid.md): define the interface your canister exposes for inter-canister calls
 - [Certified Variables](../backends/certified-variables.md): make query responses verifiable without update call overhead
 - [Inter-Canister Call Security](../security/inter-canister-calls.md): reentrancy guards, async safety patterns, and trust considerations

docs/guides/canister-calls/pagination.md

@@ -0,0 +1,185 @@
+---
+title: "Paginating query results"
+description: "How to implement reliable pagination for canister query methods, including cursor-based patterns for mutable datasets"
+sidebar:
+  order: 6
+---
+
+Many canisters expose query methods that return lists of items: messages, transactions, tokens, users. When the list grows large, returning all items in a single response is impractical. Pagination splits results into pages, but the approach matters: a naive offset-based implementation produces incorrect results as soon as the underlying dataset changes.
+
+## The problem with offset-based pagination
+
+The simplest pagination approach passes an `offset` (number of items to skip) and a `limit` (maximum items to return). This works correctly when the dataset is immutable, but breaks as soon as items are added or removed between pages.
+
+**Example:** a user fetches page 1 (items 0-9). Before they fetch page 2, a new item is inserted at position 0. Page 2 now returns items 10-19, but item 9 has shifted to position 10 and is returned again. Item 0 from the original ordering is never seen.
+
+This is a common source of bugs in applications that allow concurrent writes.
+
+## Cursor-based pagination
+
+Cursor-based pagination identifies the last item the caller received, rather than its position in the list. The caller passes a cursor (typically the ID or key of the last item they received), and the canister returns the next batch of items that come after that cursor.
+
+Because the cursor is tied to an item identity rather than a position, insertions and deletions before the cursor position do not affect the correctness of subsequent pages.
+
+### Motoko example
+
+```motoko
+import Map "mo:core/Map";
+import Nat "mo:core/Nat";
+import Text "mo:core/Text";
+import Array "mo:core/Array";
+
+persistent actor {
+
+  type Item = { id : Nat; name : Text };
+  type Page = { items : [Item]; nextCursor : ?Nat };
+
+  var nextId : Nat = 0;
+  let items = Map.empty<Nat, Item>();
+
+  public func insert(name : Text) : async Nat {
+    let id = nextId;
+    Map.add(items, Nat.compare, id, { id; name });
+    nextId += 1;
+    id
+  };
+
+  // Returns up to `limit` items with IDs strictly greater than `afterId`.
+  // Pass `null` for `afterId` to start from the beginning.
+  // Returns `nextCursor = null` when there are no more items.
+  public query func listItems(afterId : ?Nat, limit : Nat) : async Page {
+    let threshold = switch afterId { case null 0; case (?n) n + 1 };
+    var collected : [Item] = [];
+    var count = 0;
+    label scan for ((id, item) in Map.entries(items)) {
+      if (id < threshold) continue scan;
+      if (count >= limit) break scan;
+      collected := Array.concat(collected, [item]);
+      count += 1;
+    };
+    let nextCursor = if (count < limit) null else ?(collected[count - 1].id);
+    { items = collected; nextCursor }
+  };
+
+}
+```
+
+### Rust example
+
+```rust
+use ic_stable_structures::{StableBTreeMap, memory_manager::{MemoryId, MemoryManager, VirtualMemory}, DefaultMemoryImpl};
+use ic_stable_structures::storable::{Bound, Storable};
+use ic_cdk::{query, update};
+use candid::{CandidType, Deserialize};
+use serde::Serialize;
+use std::borrow::Cow;
+use std::cell::RefCell;
+
+type Memory = VirtualMemory<DefaultMemoryImpl>;
+
+#[derive(CandidType, Serialize, Deserialize, Clone)]
+struct Item {
+    id: u64,
+    name: String,
+}
+
+impl Storable for Item {
+    const BOUND: Bound = Bound::Unbounded;
+
+    fn to_bytes(&self) -> Cow<'_, [u8]> {
+        let mut buf = vec![];
+        ciborium::into_writer(self, &mut buf).expect("failed to encode Item");
+        Cow::Owned(buf)
+    }
+
+    fn from_bytes(bytes: Cow<'_, [u8]>) -> Self {
+        ciborium::from_reader(bytes.as_ref()).expect("failed to decode Item")
+    }
+}
+
+#[derive(CandidType)]
+struct Page {
+    items: Vec<Item>,
+    next_cursor: Option<u64>,
+}
+
+thread_local! {
+    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
+        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));
+
+    static ITEMS: RefCell<StableBTreeMap<u64, Item, Memory>> =
+        RefCell::new(StableBTreeMap::init(
+            MEMORY_MANAGER.with(|m| m.borrow().get(MemoryId::new(0)))
+        ));
+
+    static NEXT_ID: RefCell<u64> = RefCell::new(0);
+}
+
+#[update]
+fn insert(name: String) -> u64 {
+    let id = NEXT_ID.with(|n| {
+        let current = *n.borrow();
+        *n.borrow_mut() = current + 1;
+        current
+    });
+    ITEMS.with(|items| items.borrow_mut().insert(id, Item { id, name }));
+    id
+}
+
+/// Returns up to `limit` items with IDs strictly greater than `after_id`.
+/// Pass `None` for `after_id` to start from the beginning.
+/// Returns `next_cursor = None` when there are no more items.
+#[query]
+fn list_items(after_id: Option<u64>, limit: u64) -> Page {
+    let start = after_id.map(|id| id + 1).unwrap_or(0);
+    let mut result = Vec::new();
+
+    ITEMS.with(|items| {
+        for (_, item) in items.borrow().range(start..) {
+            if result.len() as u64 >= limit {
+                break;
+            }
+            result.push(item.clone());
+        }
+    });
+
+    let next_cursor = if result.len() as u64 < limit {
+        None
+    } else {
+        result.last().map(|item| item.id)
+    };
+
+    Page { items: result, next_cursor }
+}
+
+ic_cdk::export_candid!();
+```
+
+## Handling deleted cursor items
+
+When a caller resumes pagination using a cursor that no longer exists (the item was deleted), the canister should return items that come after where the cursor item would have been, based on sort order. In both examples above, the cursor is a monotonically increasing integer ID. Because deleted IDs are never reused, a query with `after_id = 42` will always return items with IDs greater than 42, even if ID 42 no longer exists.
+
+If your dataset uses non-integer cursors or allows ID reuse, you need to handle this case explicitly in your query method. Return the next batch that would logically follow the cursor position in the current sort order.
+
+## Sort order stability
+
+Cursor-based pagination requires a consistent sort order. If the sort order can change between pages (for example, items are re-ranked by score while the user is paginating), cursor-based pagination can still produce gaps or duplicates.
+
+For datasets with a stable natural sort order (by insertion time, by monotonic ID, or by an immutable attribute), cursor pagination is reliable. For datasets sorted by frequently changing attributes, consider whether pagination is the right interface at all, or whether the caller should re-fetch from the beginning when the sort order changes.
+
+## Cycle cost considerations
+
+Each call to a query method on a canister on the IC mainnet costs cycles. When paginating a large dataset, the total cycle cost is proportional to the number of pages fetched. If callers frequently paginate through the entire dataset, consider:
+
+- Increasing the page size to reduce the number of round trips
+- Caching recent results at the frontend layer
+- Exposing a bulk export method for callers that need the full dataset
+
+## Related
+
+- [Inter-canister calls](./inter-canister-calls.md): calling query methods from other canisters
+- [Calling from clients](./calling-from-clients.md): making query calls from a browser or CLI
+- [Data persistence](../backends/data-persistence.md): storage patterns for canister state
+- [Stable structures (Rust)](../../languages/rust/stable-structures.md): deep dive into `ic-stable-structures` used in the Rust example above
+
+<!-- Upstream: informed by dfinity/portal docs/building-apps/best-practices/general.mdx (pagination item) -->

docs/guides/canister-management/reproducible-builds.md

@@ -264,5 +264,6 @@ Maintaining a reproducible build over years requires more than getting it workin
 - [Canister lifecycle](lifecycle.md): deploy and upgrade workflow
 - [Canister settings](settings.md): configure controllers and make canisters immutable
 - [Cycles management](cycles-management.md): top up canisters before long-term deployment
+- [Trust in canisters](trust-in-canisters.md): how users can use reproducible build verification to assess whether a canister is safe to interact with
 
 <!-- Upstream: informed by dfinity/portal — docs/building-apps/best-practices/reproducible-builds.mdx; dfinity/icp-cli-recipes — recipes/prebuilt/README.md, recipe.hbs; dfinity/icp-cli — docs/guides/using-recipes.md -->