docs: add protocol stack concept pages (Batch 1) by marc0olo · Pull Request #209 · dfinity/developer-docs

marc0olo · 2026-05-06T14:55:15Z

Summary

Creates docs/concepts/protocol/ subdirectory with 7 new concept pages: protocol stack overview, peer-to-peer, consensus, message routing, execution, state synchronization, and performance
Migrates 6 Learn Hub articles from the "Blockchain Protocol" section and 1 from the "What is ICP?" section (performance); all Learn Hub links within these pages replaced with internal paths
Restores 5 portal diagrams across the protocol pages: core_protocol_layers, consensus_orders_messages, block_maker, consensus_notarization, state-sync; adds 2 performance screenshots from the Learn Hub
Performance page: metric values linked to timestamped IC Dashboard API calls, 12-city RTT matrix table, synthetic benchmark latency data
Converts Concepts sidebar from autogenerate to explicit items array to fix lowercase group labels and control ordering; moves Protocol Stack after architecture pages; adds Chain Fusion and Protocol Stack index slugs labeled "Overview" to avoid duplicate group/child labels
Updates docs/concepts/index.md: adds Protocol Stack to Architecture section
Updates docs/concepts/network-overview.md: trims consensus subsection (detail now in consensus.md), links to protocol stack overview

Staging files deleted

.migration/learn-hub/how-does-icp-work/blockchain-protocol/blockchain-protocol.md → docs/concepts/protocol/index.md
.migration/learn-hub/how-does-icp-work/blockchain-protocol/consensus.md → docs/concepts/protocol/consensus.md
.migration/learn-hub/how-does-icp-work/blockchain-protocol/peer-to-peer.md → docs/concepts/protocol/peer-to-peer.md
.migration/learn-hub/how-does-icp-work/blockchain-protocol/message-routing.md → docs/concepts/protocol/message-routing.md
.migration/learn-hub/how-does-icp-work/blockchain-protocol/execution-layer.md → docs/concepts/protocol/execution.md
.migration/learn-hub/how-does-icp-work/blockchain-protocol/state-synchronization.md → docs/concepts/protocol/state-synchronization.md
.migration/learn-hub/out-of-scope/what-is-icp/performance.md → docs/concepts/protocol/performance.md

Sync recommendation

hand-written

marc0olo · 2026-05-06T15:54:14Z

Review notes

The structure, content, and internal links all look good. One issue needs to be fixed before merge:

Em-dashes (banned per CLAUDE.md): All 6 new files use em-dashes (—) as description separators in the "Further reading" bullet lists (~16 total instances). CLAUDE.md explicitly bans em-dashes in bullet descriptions and "Further reading" sections. Replace each with a colon (e.g. - [Link text](url): description).

No other issues: frontmatter complete on all pages, all internal links resolve,  comment present on every file, glossary and network-overview.md cross-links correctly updated.

marc0olo · 2026-05-06T16:12:26Z

Review feedback

1. Em-dashes (16 instances across 6 files)

Em-dashes are banned project-wide per CLAUDE.md. Replace all — with : .

docs/concepts/protocol/index.md — 9 instances:

Lines 14–17 (numbered list items)
Lines 36–40 ("Further reading" bullets)

docs/concepts/protocol/consensus.md — line 48

docs/concepts/protocol/peer-to-peer.md — line 28

docs/concepts/protocol/message-routing.md — lines 67–68

docs/concepts/protocol/execution.md — line 68

docs/concepts/protocol/state-synchronization.md — lines 28–29

2. Terminology mismatch in `state-synchronization.md` line 22

"The blockchain state" should be "The subnet state" to match the terminology used consistently throughout the rest of the page.

marc0olo · 2026-05-06T16:37:57Z

Feedback addressed:

Replaced all 16 em-dashes with colons in Further reading bullets and numbered list items across all 6 files (index.md, consensus.md, peer-to-peer.md, message-routing.md, execution.md, state-synchronization.md)
Fixed terminology: "The blockchain state" → "The subnet state" in state-synchronization.md line 22

…ers pages - Fix AccountIdentifier em-dash in token-ledgers.md - Revert chain-fusion/ subdirectory links to chain-fusion.md (flat page on main) until PR #213 merges; affects token-ledgers.md and concepts/index.md - Remove broken link to protocol/execution.md#deterministic-time-slicing in cycles.md until PR #209 merges

marc0olo · 2026-05-07T10:25:25Z

Brand voice audit applied. The following fixes were committed in eb66cd9:

Fixed automatically:

protocol/index.md: removed "utility token" framing ("from which its utility token derives its name" → "which gives the network its name"); dropped the blockchain analogy sentence ("Messages on ICP are analogous to transactions on other blockchains")
consensus.md: removed Bitcoin comparison ("as used by Bitcoin-like protocols" → dropped); changed "transaction can be trusted" → "message response can be trusted" (ICP has messages, not transactions)
execution.md:
- Removed "reverse gas model" from the page description
- Reframed the cycles accounting paragraph: canister pays for its own storage and computation; users never send cycles with messages — no mention of "reverse gas model"
- Replaced "decentralized autonomous organization" with "community-governed application"
- Dropped "compared to traditional blockchains" from the orthogonal persistence paragraph

Open items for human review (no code change made):

execution.md title "Execution Layer" — description currently says "WebAssembly, deterministic time slicing, and concurrent execution"; if DTS should be explained earlier in the description that is a content decision, not a brand fix
consensus.md / message-routing.md / state-synchronization.md: the term "cross-subnet" is used throughout as a precise ICP technical term (distinct from the generic "cross-chain" rule). Left as-is — "crosssubnet" is not a recognised compound.
execution.md section "Cycles accounting": the phrase "community-governed application" is a placeholder. If there is a preferred ICP-brand term for SNS-governed canisters, that should be substituted here.
No SNS or DAO content in this PR — that is covered in Batch 7.

…ng, execution, state sync) Migrates 6 Learn Hub articles from the "Blockchain Protocol" section into docs/concepts/protocol/: index, consensus, peer-to-peer, message-routing, execution, and state-synchronization. Updates cross-links in network-overview.md and glossary.md to point to the new internal pages instead of learn.internetcomputer.org.

All em-dashes in Further reading bullets and numbered list items replaced with colons per project style rules. "The blockchain state" corrected to "The subnet state" in state-synchronization.md for terminology consistency.

…minology - protocol/index.md: drop "utility token" framing and blockchain analogy sentence - consensus.md: remove Bitcoin-like protocols comparison; reframe probabilistic finality - execution.md: remove "reverse gas model" from description and body; reframe canister-pays model; drop "traditional blockchains" comparison; replace "decentralized autonomous organization" with "community-governed application"

…urrence New glossary entries: block maker, Deterministic Time Slicing (DTS), orthogonal persistence, random beacon, Trusted Execution Environment (TEE), XNet Per-page link fixes: - consensus.md: link block maker and random beacon to new glossary entries - peer-to-peer.md: link NNS to glossary on first use - message-routing.md: link ingress messages, XNet, induction pool, NNS, catch-up package to glossary - execution.md: link cycles and query to glossary; link orthogonal persistence to its concept page - state-synchronization.md: link catch-up package to glossary

…ers pages - Fix AccountIdentifier em-dash in token-ledgers.md - Revert chain-fusion/ subdirectory links to chain-fusion.md (flat page on main) until PR #213 merges; affects token-ledgers.md and concepts/index.md - Remove broken link to protocol/execution.md#deterministic-time-slicing in cycles.md until PR #209 merges

…put, latency)

…ST APIs

…main /docs paths; remove em-dashes from dashboard API reference

… sorts last

- Convert Concepts from autogenerate to explicit items so subdirectory groups get proper labels: "Protocol Stack" and "Chain Fusion" instead of lowercase directory names - Protocol Stack index page now appears first in the group, leaf pages follow in layer order (peer-to-peer → performance) - Fix Further reading order in protocol/index.md to match sidebar order - Trim redundant four-phase consensus detail from network-overview.md; the full breakdown lives in concepts/protocol/consensus.md now - Add cross-reference to protocol/index.md in network-overview.md Nodes section where the four-layer stack is first named - Add Protocol Stack entry to concepts/index.md Architecture section

…nd synthetic latency - Add latency distribution screenshot (HTTP gateway observations, July 2025) and latency-vs-throughput graph (counter canister experiment, June 2025) sourced from the Learn Hub article and stored in public/concepts/protocol/ - Link each mainnet metric value to the exact timestamped API call that produced it, matching the original Learn Hub approach - Split Synthetic benchmarks into Throughput and Latency under load subsections; add the three latency data points (2.27s / 1.08s / 0.52s) and the experiment setup note (same-DC, 30ms simulated RTT) - Restore the full 12-city inter-datacenter RTT matrix (September 2023) replacing the prose summary; keep the three summary sentences below the table for quick orientation

…ex as Overview - Protocol Stack group now sits after App Architecture (matching the Architecture grouping in concepts/index.md) instead of at the end - Index pages within Protocol Stack and Chain Fusion groups now carry label "Overview" to avoid duplicate "Protocol Stack" / "Chain Fusion" entries in the subnav

…nsus, and state sync - core_protocol_layers.webp: four-layer stack diagram, placed after the numbered list in the Protocol Stack overview - consensus_orders_messages.webp: shows consensus collecting ingress, XNet, and protocol messages into an ordered block; placed after the opening paragraph in Consensus - block_maker.webp: rank-based block maker selection diagram; placed at the end of the Block making subsection before Notarization - state-sync.webp: checkpoint download and replay diagram; placed after the opening paragraph in State Synchronization

…m, restore per-layer description Consensus: - Move consensus_orders_messages diagram to ## Consensus rounds (correct position per Learn Hub: after the crypto-finality section, not before it) - Restore the high-level three-phase overview bullets (block making, notarization, finalization) before the detailed subsections, matching the original orientation paragraph - Add missing consensus_notarization diagram after the notarization paragraph about higher-rank fallback - Update all image alt text to match original Learn Hub captions State synchronization: - Move state-sync diagram from after the opening paragraph to the end of ## Recovering nodes, where it was in the Learn Hub ("Nodes that are behind" section shows the diff-based incremental sync, which the image illustrates) Protocol Stack overview: - Add per-layer description paragraph after the diagram (present in the Learn Hub "Blockchain Protocol" article, omitted in our first draft)

…mance staging file migrated Adds "per canister" to the 2B per-round DTS limit in execution.md to distinguish it from the 8B/s subnet-level throughput figure on performance.md. Removes the out-of-scope performance staging file now that its content has been migrated.

…urther reading link Adds node-infrastructure after app-architecture in the explicit Concepts sidebar (now that PR #209 has merged). Restores the Protocol Stack further reading link that was blocked during cross-branch validation.

…ers pages - Fix AccountIdentifier em-dash in token-ledgers.md - Revert chain-fusion/ subdirectory links to chain-fusion.md (flat page on main) until PR #213 merges; affects token-ledgers.md and concepts/index.md - Remove broken link to protocol/execution.md#deterministic-time-slicing in cycles.md until PR #209 merges

## Summary - Creates `docs/concepts/protocol/` subdirectory with 7 new concept pages: protocol stack overview, peer-to-peer, consensus, message routing, execution, state synchronization, and performance - Migrates 6 Learn Hub articles from the "Blockchain Protocol" section and 1 from the "What is ICP?" section (performance); all Learn Hub links within these pages replaced with internal paths - Restores 5 portal diagrams across the protocol pages: `core_protocol_layers`, `consensus_orders_messages`, `block_maker`, `consensus_notarization`, `state-sync`; adds 2 performance screenshots from the Learn Hub - Performance page: metric values linked to timestamped IC Dashboard API calls, 12-city RTT matrix table, synthetic benchmark latency data - Converts Concepts sidebar from `autogenerate` to explicit `items` array to fix lowercase group labels and control ordering; moves Protocol Stack after architecture pages; adds Chain Fusion and Protocol Stack index slugs labeled "Overview" to avoid duplicate group/child labels - Updates `docs/concepts/index.md`: adds Protocol Stack to Architecture section - Updates `docs/concepts/network-overview.md`: trims consensus subsection (detail now in consensus.md), links to protocol stack overview ## Staging files deleted - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/blockchain-protocol.md` → `docs/concepts/protocol/index.md` - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/consensus.md` → `docs/concepts/protocol/consensus.md` - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/peer-to-peer.md` → `docs/concepts/protocol/peer-to-peer.md` - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/message-routing.md` → `docs/concepts/protocol/message-routing.md` - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/execution-layer.md` → `docs/concepts/protocol/execution.md` - `.migration/learn-hub/how-does-icp-work/blockchain-protocol/state-synchronization.md` → `docs/concepts/protocol/state-synchronization.md` - `.migration/learn-hub/out-of-scope/what-is-icp/performance.md` → `docs/concepts/protocol/performance.md` ## Sync recommendation hand-written

…ocs (#208) ## Summary Migrates all 57 developer-relevant articles from the Learn Hub \"How does ICP work?\" section into the developer docs, replacing Learn Hub as the canonical source for ICP protocol concepts before the site is retired. This was executed as 9 content batch PRs plus a post-migration review pass, all merged into this staging branch before landing on `main`: **Batch 1 — Protocol stack** (#209): Consensus, peer-to-peer, message routing, execution layer, state synchronization, and performance concept pages under `docs/concepts/protocol/`. **Batch 2 — Node infrastructure** (#210): `docs/concepts/node-infrastructure.md` — TEE (SEV-SNP) attestation, IC-OS layer model (SetupOS / HostOS / GuestOS), TEE key derivation and disk encryption. **Batch 3 — Edge infrastructure** (#211): `docs/concepts/edge-infrastructure.md` — boundary nodes, HTTP gateway, edge caching. **Batch 4 — Evolution and scaling** (#212): `docs/concepts/evolution-scaling.md` — horizontal scaling via subnet sharding, fault tolerance, governance-driven subnet creation, forkless protocol upgrades. **Chain fusion deep dives** (#213): Expanded `docs/concepts/chain-fusion/` with dedicated pages for Bitcoin integration, ckBTC mechanics, Ethereum integration, EVM RPC, Solana RPC, Dogecoin integration, and Exchange Rate Canister. Architecture and flow diagrams migrated to `public/concepts/chain-fusion/`. **Cryptography deep dives** (#214): Expanded `docs/concepts/chain-key-cryptography.md` and `docs/concepts/certified-data.md` with subnet keys, threshold signing internals, and certified communication theory. **Tokens, ledgers, and cycles** (#215): Expanded `docs/concepts/ledgers.md` and `docs/concepts/cycles.md` with ledger architecture, accounts, sub-accounts, cycles ledger transfer semantics, and NNS tokenomics. **Governance deep dives** (#216): Expanded `docs/concepts/governance.md` and added `docs/concepts/sns-framework.md` — NNS neuron attributes and bonuses, voting rewards formula, Neurons Fund, proposal taxonomy, SNS framework and architecture, DAO settings, launch flow, SNS neurons and rewards, and SNS proposals. **Principals and canister model** (#217): Expanded `docs/concepts/principals.md` and `docs/concepts/canisters.md` — principal model, canister control, Wasm execution model, and `docs/concepts/app-architecture.md`. **Post-migration holistic review** (#218, #226): - Navigation restructured across all concepts, guides, and references — sentence case applied throughout, section indices completed - Cross-reference audit: all Learn Hub outbound links replaced with internal doc links; first-use terms linked to their concept pages - Glossary expanded with TEE, boundary node, and cycle-related entries - System canisters content placement corrected - Exchange Rate Canister restructured across concept, reference, and guide pages (#XRC commits) **Migration infrastructure** (initial prep commits): - `.docs-plan/learn-hub-migration.md` — agent workflow and authoring rules for 9-batch execution - `.docs-plan/learn-hub-navigation.md` — per-article mapping (target file, action type, cross-link table) - `AGENTS.md / CLAUDE.md` — Learn Hub removed from external-docs list; linking rule updated - `sidebar.mjs` — all new concept pages registered ## Closes - Closes #187 — Content gap analysis: Learn Hub vs developer docs - Closes #190 — Gap analysis: articles on learn.internetcomptuter.org vs developer-docs `reference` and `concepts` - Closes #226 — Post-migration holistic review (content placement, nav structure, glossary, first-use terms) ## Sync recommendation hand-written (all pages authored from Learn Hub source articles via Zendesk API; no upstream repo to sync from)

marc0olo requested a review from a team as a code owner May 6, 2026 14:55

This was referenced May 6, 2026

docs: add node infrastructure concept page (Batch 2) #210

Merged

docs: add evolution & scaling concept page (Batch 4) #212

Merged

infra: cross-reference audit for new concept pages (PRs #209-217) #218

Closed

marc0olo mentioned this pull request May 6, 2026

docs: Batch 8 — tokens, ledgers, and cycles concepts #215

Merged

marc0olo force-pushed the infra/learn-hub-migration-prep branch from dabad66 to d191270 Compare May 8, 2026 09:02

marc0olo added 5 commits May 8, 2026 11:18

fix(validate): remove cross-batch link from TEE glossary entry

97a5efd

marc0olo force-pushed the docs/concepts-protocol-stack branch from 568de86 to 97a5efd Compare May 8, 2026 09:18

marc0olo added 2 commits May 8, 2026 11:36

docs: add performance concept page and glossary terms (MIEPS, through…

0bd629e

…put, latency)

docs: add IC Dashboard API reference page covering all five public RE…

50f10a8

…ST APIs

marc0olo mentioned this pull request May 8, 2026

docs: add cross-references to IC Dashboard APIs in relevant sections #222

Closed

4 tasks

marc0olo added 7 commits May 8, 2026 12:01

fix: tighten retired-docs regex to avoid false positives on API subdo…

5112c64

…main /docs paths; remove em-dashes from dashboard API reference

fix(sidebar): add explicit order to all protocol pages so Performance…

ce0c64f

… sorts last

marc0olo mentioned this pull request May 8, 2026

docs(chain-fusion): review Learn Hub images not migrated in PR #213 #223

Closed

4 tasks

marc0olo merged commit b6a6fad into infra/learn-hub-migration-prep May 8, 2026
3 checks passed

marc0olo deleted the docs/concepts-protocol-stack branch May 8, 2026 11:53

marc0olo mentioned this pull request May 11, 2026

docs: migrate Learn Hub 'How does ICP work?' content into developer docs #208

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add protocol stack concept pages (Batch 1)#209

docs: add protocol stack concept pages (Batch 1)#209
marc0olo merged 15 commits into
infra/learn-hub-migration-prepfrom
docs/concepts-protocol-stack

marc0olo commented May 6, 2026 •

edited

Loading

Uh oh!

marc0olo commented May 6, 2026

Uh oh!

marc0olo commented May 6, 2026

Uh oh!

marc0olo commented May 6, 2026

Uh oh!

marc0olo commented May 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

marc0olo commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Staging files deleted

Sync recommendation

Uh oh!

marc0olo commented May 6, 2026

Uh oh!

marc0olo commented May 6, 2026

Review feedback

1. Em-dashes (16 instances across 6 files)

2. Terminology mismatch in state-synchronization.md line 22

Uh oh!

marc0olo commented May 6, 2026

Uh oh!

marc0olo commented May 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

marc0olo commented May 6, 2026 •

edited

Loading

2. Terminology mismatch in `state-synchronization.md` line 22