diff --git a/docs/guides/canister-calls/parallel-calls.md b/docs/guides/canister-calls/parallel-calls.md deleted file mode 100644 index 502b614e..00000000 --- a/docs/guides/canister-calls/parallel-calls.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: "Parallel Calls" -description: "Execute multiple inter-canister calls concurrently for better performance" -sidebar: - order: 5 ---- - -TODO: Write content for this page. - - -Execute multiple inter-canister calls concurrently instead of sequentially. Cover futures::join_all in Rust, async/await patterns in Motoko, composite queries for read-only parallel calls. Handle partial failures when some calls succeed and others fail. Explain performance benefits and when parallel calls are appropriate. - - -- Portal: building-apps/integrations/advanced-calls.mdx (composite queries section) -- icskills: multi-canister -- Examples: parallel_calls (both), composite_query (both) - - -- guides/canister-calls/onchain-calls -- basic inter-canister calls -- guides/canister-management/optimization -- performance improvements -- guides/security/inter-canister-calls -- safety of async calls diff --git a/docs/guides/canister-calls/parallel-calls.mdx b/docs/guides/canister-calls/parallel-calls.mdx new file mode 100644 index 00000000..bb556644 --- /dev/null +++ b/docs/guides/canister-calls/parallel-calls.mdx @@ -0,0 +1,362 @@ +--- +title: "Parallel Calls" +description: "Execute multiple inter-canister calls concurrently to reduce latency, especially across subnets." +sidebar: + order: 5 +--- + +import { Tabs, TabItem } from '@astrojs/starlight/components'; + +By default, each inter-canister call is issued and awaited in sequence. When calls are independent of one another, issuing them in parallel reduces total latency: all calls are dispatched before any response is awaited, so the round-trip times overlap instead of stacking. + +Parallel calls are most beneficial when the caller and callee are on **different subnets**. Cross-subnet calls each take 2–3 consensus rounds; running them sequentially multiplies that cost by the number of calls. On the same subnet, calls often complete within a single round, so the gain is smaller. + +## Prerequisites + + + + +- [icp-cli](https://cli.internetcomputer.org/) installed +- `mops` package manager with `core = "2.0.0"` in `mops.toml` + + + + +- [icp-cli](https://cli.internetcomputer.org/) installed +- `ic-cdk = "0.19"` and `futures = "0.3"` in `Cargo.toml` + + + + +## How parallel calls work + +In Motoko, futures are first-class values. You can start a call by evaluating `c.method()` without immediately awaiting it — this sends the request message and returns an `async T` handle. Collecting all handles before awaiting lets all calls run concurrently. + +In Rust, you can collect calls into a `Vec` by calling `.into_future()` on each [`Call::bounded_wait(...)`](https://docs.rs/ic-cdk/latest/ic_cdk/call/struct.Call.html) expression (since `Call` implements `IntoFuture`), then pass them to [`futures::future::join_all`](https://docs.rs/futures/latest/futures/future/fn.join_all.html), which awaits all of them together. + +## Parallel calls example + +The following example shows a `caller` canister that issues `n` calls to a `callee` canister's `ping` method, either sequentially or in parallel. + +**Sequential version** — each call is awaited before the next is sent: + + + + +```motoko +import Nat "mo:core/Nat"; +import Error "mo:core/Error"; +import Principal "mo:core/Principal"; + +persistent actor { + + type CalleeInterface = actor { ping : () -> async () }; + var callee = null : ?CalleeInterface; + + public func setup_callee(c : Principal) { + callee := ?actor (Principal.toText(c) : CalleeInterface); + }; + + public func sequential_calls(n : Nat) : async Nat { + let c = switch callee { + case null { throw Error.reject("callee not set up") }; + case (?c) { c }; + }; + + var successful = 0; + for (_ in Nat.range(0, n)) { + try { + await c.ping(); // await each call before sending the next + successful += 1; + } catch _ {}; + }; + successful + }; +}; +``` + + + + +```rust +use candid::Principal; +use ic_cdk::call::Call; +use std::cell::RefCell; + +thread_local! { + static CALLEE: RefCell> = RefCell::new(None); +} + +#[ic_cdk::update] +pub async fn setup_callee(id: Principal) { + CALLEE.with(|c| *c.borrow_mut() = Some(id)); +} + +#[ic_cdk::update] +pub async fn sequential_calls(n: u64) -> u64 { + let callee = CALLEE.with(|c| c.borrow().unwrap()); + let mut successful = 0u64; + for _ in 0..n { + // await each call before sending the next + let result = Call::bounded_wait(callee, "ping").await; + if result.is_ok() { + successful += 1; + } + } + successful +} +``` + + + + +**Parallel version** — all requests are dispatched before any response is awaited: + + + + +```motoko +import List "mo:core/List"; +import Nat "mo:core/Nat"; +import Error "mo:core/Error"; +import Principal "mo:core/Principal"; + +persistent actor { + + type CalleeInterface = actor { ping : () -> async () }; + var callee = null : ?CalleeInterface; + + public func setup_callee(c : Principal) { + callee := ?actor (Principal.toText(c) : CalleeInterface); + }; + + // Dispatch all calls first, then collect results. + public func parallel_calls(n : Nat) : async Nat { + let c = switch callee { + case null { throw Error.reject("callee not set up") }; + case (?c) { c }; + }; + + // Evaluate c.ping() without awaiting — sends the request and returns a + // future. Collecting futures before any await dispatches all requests + // concurrently. + var futures = List.empty(); + for (_ in Nat.range(0, n)) { + try { + List.add(futures, c.ping()); + } catch _ {}; + }; + + // Await in the same order as dispatch to minimise scheduler overhead. + // The IC delivers responses in request order in practice, so in-order + // await avoids unnecessary task rescheduling. + var successful = 0; + for (f in List.values(futures)) { + try { + await f; + successful += 1; + } catch _ {}; + }; + successful + }; +}; +``` + + + + +```rust +use candid::Principal; +use futures::future::{self, BoxFuture}; +use ic_cdk::call::Call; +use std::future::IntoFuture; +use std::cell::RefCell; + +thread_local! { + static CALLEE: RefCell> = RefCell::new(None); +} + +#[ic_cdk::update] +pub async fn parallel_calls(n: u64) -> u64 { + let callee = CALLEE.with(|c| c.borrow().unwrap()); + + // Build all futures before awaiting any of them. All requests are + // dispatched when join_all polls each future — all fire before any + // response is awaited. + // Box::pin erases the lifetime parameters so futures can be collected + // into a homogeneous Vec. + let calls: Vec> = (0..n) + .map(|_| -> BoxFuture<_> { + Box::pin(Call::bounded_wait(callee, "ping").into_future()) + }) + .collect(); + + // join_all awaits all calls together; results arrive as they complete. + let results = future::join_all(calls).await; + results.iter().filter(|r| r.is_ok()).count() as u64 +} +``` + + + + +The full working example is available in [`dfinity/examples`](https://github.com/dfinity/examples): [Motoko](https://github.com/dfinity/examples/tree/master/motoko/parallel_calls) | [Rust](https://github.com/dfinity/examples/tree/master/rust/parallel_calls). + +## In-flight call limit + +The IC enforces a limit on the number of in-flight calls a canister can have outstanding to any other single canister — approximately 500 per canister pair. Dispatching more calls than this limit causes the excess to be rejected immediately. Sequential calls stay within the limit because only one call is in-flight at a time. Parallel calls can exceed it when `n` is large. + +If calls fail due to the in-flight limit, do not retry immediately — the limit will still be full right after the failure. Instead, retry from a [timer](../backends/timers.md) or heartbeat after a delay. + +## Handling partial failures + +`join_all` and the Motoko loop both collect all outcomes, including errors. The examples above count only successes. In production, log or handle each failure: + + + + +```motoko +for (f in List.values(futures)) { + try { + await f; + // handle success + } catch (e : Error.Error) { + // log or record Error.message(e) + }; +}; +``` + + + + +```rust +for result in future::join_all(calls).await { + match result { + Ok(_response) => { /* handle success */ } + Err(e) => { /* log or handle e */ } + } +} +``` + + + + +Because each inter-canister call is a separate async boundary, a failure in one call does not roll back state changes made before or after other calls. Design for partial success: identify which calls succeeded, which failed, and whether a retry or compensation is needed. + +## Composite queries + +A **composite query** is a query method that can call other query and composite query methods. Unlike update calls, composite queries are read-only, run without consensus, and complete without going through the full consensus pipeline — making them far lower latency than update-based parallel calls. + +Use composite queries when all the data you need can be read from query endpoints and you do not need to modify state. + +**Restrictions compared to update-based parallel calls:** + +- Composite queries can only be invoked directly from an agent (browser, CLI tool). They cannot be called by another canister as an update. +- Composite queries cannot call canisters on a different subnet. +- Composite queries cannot call update methods. + +**Annotations:** + +| Language | Annotation | +|---|---| +| Motoko | `composite query` keyword on the method | +| Rust | `#[query(composite = true)]` | +| Candid | `composite_query` | + + + + +```motoko +import Array "mo:core/Array"; + +// Bucket canister — regular query +persistent actor class Bucket(n : Nat, i : Nat) { + + // ...state omitted... + + public query func get(k : Nat) : async ?Text { + // look up k in local state + null // placeholder + }; +}; + +// Map canister — composite query calling into Bucket +persistent actor Map { + let n = 4; + type Bucket = actor { get : Nat -> async ?Text }; + let buckets : [var ?Bucket] = Array.init(n, null); + + // Composite query: can call other query methods on other canisters + public composite query func get(k : Nat) : async ?Text { + switch (buckets[k % n]) { + case null null; + case (?bucket) await bucket.get(k); // inter-canister query call + }; + }; +}; +``` + + + + +```rust +use ic_cdk::call::Call; +use candid::Principal; +use std::cell::RefCell; + +thread_local! { + static PARTITIONS: RefCell> = RefCell::new(vec![]); +} + +// Composite query: annotated with composite = true +#[ic_cdk::query(composite = true)] +async fn get(key: u128) -> Option { + let partition_id = get_partition_for(key); + match Call::bounded_wait(partition_id, "get") + .with_arg(key) + .await + { + Ok(response) => response + .candid_tuple::<(Option,)>() + .map(|(v,)| v) + .unwrap_or(None), + Err(_) => None, + } +} + +fn get_partition_for(key: u128) -> Principal { + PARTITIONS.with(|p| { + let p = p.borrow(); + p[key as usize % p.len()] + }) +} +``` + + + + +The full composite query example is available in [`dfinity/examples`](https://github.com/dfinity/examples): [Motoko](https://github.com/dfinity/examples/tree/master/motoko/composite_query) | [Rust](https://github.com/dfinity/examples/tree/master/rust/composite_query). + +## When to use each approach + +| Approach | When to use | +|---|---| +| Sequential update calls | Calls have data dependencies; each result feeds the next call | +| Parallel update calls | Independent calls; latency matters; cross-subnet targets | +| Composite queries | Read-only data retrieval; all targets on the same subnet; lowest latency | + +## Security considerations + +Parallel and composite calls carry the same atomicity properties as any inter-canister call: + +- **No atomic rollback across calls.** State changes committed before the first `await` are persisted even if later parallel calls fail. Design state mutations to be idempotent or use a saga/compensation pattern. +- **Reentrancy.** Dispatching many calls in parallel increases the window during which another ingress message can execute and observe partial state. Acquire any locks before dispatching parallel calls and release them after all calls complete. +- **Callee trust.** A malicious or slow callee can delay your callback. For untrusted callees, prefer `bounded_wait` calls so the timeout prevents indefinite blocking. See [inter-canister call security](../security/inter-canister-calls.md) for full guidance. + +## Next steps + +- [Onchain calls](onchain-calls.md) — making basic inter-canister calls +- [Canister optimization](../canister-management/optimization.md) — profiling and improving throughput +- [Inter-canister call security](../security/inter-canister-calls.md) — atomicity, reentrancy, and call safety + +{/* Upstream: informed by dfinity/portal — docs/building-apps/interact-with-canisters/advanced-calls.mdx, docs/building-apps/interact-with-canisters/query-calls.mdx; dfinity/examples — motoko/parallel_calls, rust/parallel_calls, motoko/composite_query, rust/composite_query; dfinity/cdk-rs — ic-cdk/src/call.rs, ic-cdk/src/futures.rs */}