Skip to content

docs: comprehensive implementation status revamp and extensions framework guide #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shivv23 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

docs: comprehensive implementation status revamp and extensions framework guide #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions pubsub/gossipsub/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,25 @@ If you are new to Gossipsub and/or PubSub in general, we recommend you to first:
- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. This is a revised specification, to use a more normative language. The original v1.0 specification is [here](gossipsub-v1.0-old.md), still a good read.
- [gossipsub-v1.1](gossipsub-v1.1.md): v1.1 of the gossipsub protocol.
- [gossipsub-v1.2](gossipsub-v1.2.md): v1.2 of the gossipsub protocol. This includes the aggregation of the IDONTWANT control messages to the specs.
- [gossipsub-v1.3](gossipsub-v1.3.md): v1.3 of the gossipsub protocol, introducing the Extensions Control Message framework.
- [Extensions Guide](extensions/README.md): comprehensive documentation of the GossipSub v1.3 extensions framework, including the canonical and experimental extension registry.
- [(not in use) episub](episub.md): a research note on a protocol building on top of gossipsub to implement [epidemic broadcast trees](https://www.gsd.inesc-id.pt/~ler/reports/srds07.pdf).

## Implementation status

Legend: ✅ = complete, 🏗 = in progress, ❕ = not started yet
For a detailed per-extension breakdown across all implementations, see
[implementation-status.md](implementation-status.md).

| Name | v1.0 | v1.1 | v1.2 |
|--------------------------------------------------------------------------------------------------|:-----:|:-----:|:----:|
| [go-libp2p-pubsub (Golang)](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) | ✅ | ✅ | ✅ |
| [js-libp2p-gossipsub (JavaScript)](https://github.com/ChainSafe/js-libp2p-gossipsub) | ✅ | ✅ | ✅ |
| [rust-libp2p (Rust)](https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub) | ✅ | ✅ | ❔ |
| [py-libp2p (Python)](https://github.com/libp2p/py-libp2p/tree/master/libp2p/pubsub) | ✅ | 🏗 | ❔ |
| [jvm-libp2p (Java/Kotlin)](https://github.com/libp2p/jvm-libp2p/tree/develop/src/main/kotlin/io/libp2p/pubsub) | ✅ | 🏗 | ✅ |
| [nim-libp2p (Nim)](https://github.com/status-im/nim-libp2p/blob/master/libp2p/protocols/pubsub/gossipsub.nim) | ✅ | 🏗 | ✅ |
Legend: ✅ = complete, 🏗 = in progress, ❔ = not started

| Name | v1.0 | v1.1 | v1.2 | v1.3 |
|------|:----:|:----:|:----:|:----:|
| [go-libp2p (Golang)](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) | ✅ | ✅ | ✅ | 🏗 |
| [js-libp2p (JavaScript)](https://github.com/ChainSafe/js-libp2p-gossipsub) | ✅ | ✅ | ✅ | ❔ |
| [rust-libp2p (Rust)](https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub) | ✅ | ✅ | ✅ | 🏗 |
| [py-libp2p (Python)](https://github.com/libp2p/py-libp2p/tree/master/libp2p/pubsub) | ✅ | ✅ | ✅ | ✅ |
| [jvm-libp2p (Java/Kotlin)](https://github.com/libp2p/jvm-libp2p/tree/develop/src/main/kotlin/io/libp2p/pubsub) | ✅ | 🏗 | ✅ | ❔ |
| [nim-libp2p (Nim)](https://github.com/status-im/nim-libp2p/blob/master/libp2p/protocols/pubsub/gossipsub.nim) | ✅ | 🏗 | ✅ | ❔ |

Additional tooling:

Expand Down
108 changes: 108 additions & 0 deletions pubsub/gossipsub/extensions/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# GossipSub Extensions Framework

GossipSub v1.3 introduced a generic **Extensions Control Message** mechanism
that allows implementations to negotiate and enable optional protocol features
at connection time without breaking compatibility with peers that don't support
them.

This document explains how the extensions framework works and serves as a
registry of both canonical and experimental extensions.

## How Extensions Work

### 1. Capability Advertisement

When two peers establish a GossipSub stream, the **first RPC message** sent in
each direction MUST include a `ControlExtensions` message inside the existing
`ControlMessage` container if the sending peer supports any extensions:

```protobuf
message ControlExtensions {
// Canonical extensions (field numbers < 0x200000)
optional bool topicObservation = 1;

// Experimental extensions (field numbers > 0x200000)
optional bool testExtension = 6492434;
optional bool largeMessageSegmentation = 6492435;
}
```

This handshake follows the GossipSub v1.3 spec rules:

1. Extensions MUST be sent in the **first control message** on a fresh stream.
2. Extensions MUST NOT be sent **more than once** per peer.
3. Duplicate extensions messages are a protocol violation and SHOULD be reported
as misbehaviour.

### 2. Protocol Activation

Once both peers have exchanged extensions, an extension is considered **active**
if and only if **both** peers set its capability flag to `true`. Extension-
specific behaviour (e.g., sending segmented messages, sending observation
requests) MUST NOT begin before mutual activation.

### 3. Field Number Allocation

Field numbers in `ControlExtensions` and on the `RPC` message are split into
two ranges:

| Range | Purpose | Example |
|-------|---------|---------|
| 1 – 2,097,151 | Canonical extensions — small wire overhead (1-2 bytes) | `topicObservation = 1` |
| > 2,097,152 (0x200000) | Experimental extensions — 4+ byte encoding | `testExtension = 6492434` |

Experimental field numbers are allocated on a first-come, first-served basis.
There is no central registry — implementers choose an unused number in the
experimental range. To avoid collisions, check existing PRs and extensions
before picking a number.

## Registered Extensions

### Canonical

| # | Extension | Field | Status | Spec |
|---|-----------|-------|--------|------|
| 10 | Partial Messages | `RPC.partial = 10` | Draft | [PR #685](https://github.com/libp2p/specs/pull/685) |

### Experimental

| # | Field Number | Extension | Field Name | Status | Spec |
|---|--------------|-----------|------------|--------|------|
| 1 | 6492434 | Test Extension | `RPC.testExtension` | Draft | [test-extension.md](experimental/test-extension.md) |
| 2 | 6492435 | Large Message Segmentation | `RPC.largeMessageSegmentation` | Draft | [PR #2](https://github.com/seetadev/specs/pull/2) |

### How to Register a New Experimental Extension

1. Pick an unused field number > `0x200000` (2,097,152).
2. Add a `bool` field to `ControlExtensions` and a message field to `RPC` in
`extensions.proto`.
3. Write an extension specification document in
`extensions/experimental/<name>.md` following the
[test-extension.md](experimental/test-extension.md) template.
4. Open a PR to this repository.

## Extension Lifecycle

Experimental extensions follow the libp2p spec lifecycle:

```
Experimental (1A) → Working Draft (1B) → Candidate Recommendation (2A) → Recommendation (2B)
```

An extension may progress from Experimental to Working Draft once at least two
independent implementations exist and basic interoperability has been
demonstrated.

## Relationship to GossipSub Versions

| Version | Key Feature | Relationship to Extensions |
|---------|-------------|---------------------------|
| v1.0 | Baseline mesh + gossip | No extension mechanism |
| v1.1 | Peer scoring, PX | No extension mechanism |
| v1.2 | IDONTWANT, adaptive gossip | No extension mechanism |
| v1.3 | **Extensions Control Message** | Framework foundation |
| v1.4 | Large message handling (proposed) | Built as an extension on v1.3 |

Extensions are designed to be independent of the wire protocol version. A v1.3
peer can negotiate any extension with another v1.3 peer without requiring a new
protocol ID.
93 changes: 60 additions & 33 deletions pubsub/gossipsub/implementation-status.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,47 +1,74 @@
# Implementation status of Gossipsub versions and Extensions

This doc is meant to provide an overview of the implementation status of
Gossipsub versions and Extensions.
This doc provides an overview of the implementation status of Gossipsub
versions and Extensions across all major libp2p implementations.

Legend: ✅ = complete, 🏗 = in progress, 🔄 = spec draft / PR open, ❔ = not started

## Gossipsub Versions

| | [1.2] | [1.3-alpha] |
| ------------- | ----- | -------------------------------------------------------------- |
| [Go libp2p] | ✅ | [Open PR](https://github.com/libp2p/go-libp2p-pubsub/pull/630) |
| [Rust libp2p] | ✅ | In Progress |
| [JS libp2p] | ✅ | Not started |
| [Nim libp2p] | ✅ | Not started |
| [Java libp2p] | ✅ | Not started |
| | [v1.0] | [v1.1] | [v1.2] | [v1.3-alpha] |
| ------------- | :----: | :----: | :----: | --------------------------------------------------------------- |
| [Go libp2p] | ✅ | ✅ | ✅ | [Open PR](https://github.com/libp2p/go-libp2p-pubsub/pull/630) |
| [Rust libp2p] | ✅ | ✅ | ✅ | [In Progress](https://github.com/libp2p/rust-libp2p/pull/5878) |
| [JS libp2p] | ✅ | ✅ | ✅ | ❔ |
| [Nim libp2p] | ✅ | ✅ | ✅ | ❔ |
| [Java libp2p] | ✅ | ✅ | ✅ | ❔ |
| [py-libp2p] | ✅ | ✅ | ✅ | ✅ |

> **Note on py-libp2p v1.3:** py-libp2p ships with full v1.3 Extensions Control
> Message support, including both the `ControlExtensions` handshake protocol and
> the Topic Observation extension. See
> [PR libp2p/py-libp2p#1323](https://github.com/libp2p/py-libp2p/pull/1323) for
> the Large Message Segmentation reference implementation.

## Gossipsub Extensions

| | [Choke Extensions] | [Partial Messages] |
| ------------- | ------------------ | --------------------------------------------------------- |
| [Go libp2p] | Not Implemented | [PR](https://github.com/libp2p/go-libp2p-pubsub/pull/631) |
| [Rust libp2p] | Not Implemented | Not Implemented |
| [JS libp2p] | Not Implemented | Not Implemented |
| [Nim libp2p] | Not Implemented | Not Implemented |
| [Java libp2p] | Not Implemented | Not Implemented |
| | [Partial Messages] | [Test Extension] | [Large Message Segmentation] |
| ------------- | --------------------------------------------------------- | ---------------- | ------------------------------------------------------ |
| [Go libp2p] | [PR](https://github.com/libp2p/go-libp2p-pubsub/pull/631) | ✅ | ❔ |
| [Rust libp2p] | ❔ | ❔ | ❔ |
| [JS libp2p] | ❔ | ❔ | ❔ |
| [Nim libp2p] | ❔ | ❔ | 🔄 [Spec Draft](https://github.com/seetadev/specs/pull/2) |
| [Java libp2p] | ❔ | ❔ | ❔ |
| [py-libp2p] | ❔ | ❔ | 🔄 [PR libp2p/py-libp2p#1323](https://github.com/libp2p/py-libp2p/pull/1323) |

> Extensions are negotiated at connection time via the GossipSub v1.3
> `ControlExtensions` handshake. See
> [extensions/README.md](extensions/README.md) for a guide to the extensions
> framework.

## Gossipsub Implementation Improvements

| | [Batch Publishing] | [IDONTWANT on First Publish] | [WFR Gossip] |
| ------------- | ------------------------------------------------------------------------ | --------------------------------------------------------- | --------------------------------------------------------- |
| [Go libp2p] | [✅](https://pkg.go.dev/github.com/libp2p/go-libp2p-pubsub#MessageBatch) | [✅](https://github.com/libp2p/go-libp2p-pubsub/pull/612) | [PR](https://github.com/libp2p/go-libp2p-pubsub/pull/632) |
| [Rust libp2p] | Not Implemented | [✅](https://github.com/libp2p/rust-libp2p/pull/5773) | Not Implemented |
| [JS libp2p] | Not Implemented | Not Implemented | Not Implemented |
| [Nim libp2p] | Not Implemented | Not Implemented | Not Implemented |
| [Java libp2p] | Not Implemented | Not Implemented | Not Implemented |

[Go libp2p]: https://github.com/libp2p/go-libp2p-pubsub
[Rust libp2p]: https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub
[JS libp2p]: https://github.com/ChainSafe/js-libp2p-gossipsub
[Nim libp2p]: https://github.com/vacp2p/nim-libp2p/tree/master/libp2p/protocols/pubsub/gossipsub
[Java libp2p]: https://github.com/libp2p/jvm-libp2p/tree/develop/libp2p/src/test/kotlin/io/libp2p/pubsub/gossip
[1.2]: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md
[1.3-alpha]: https://github.com/libp2p/specs/issues/687
[Choke Extensions]: https://github.com/libp2p/specs/pull/681
[Partial Messages]: https://github.com/libp2p/specs/pull/685
[Batch Publishing]: https://ethresear.ch/t/improving-das-performance-with-gossipsub-batch-publishing/21713
[IDONTWANT on first Publish]: https://github.com/libp2p/go-libp2p-pubsub/issues/610
[WFR Gossip]: https://ethresear.ch/t/the-paths-of-least-resistance-introducing-wfr-gossip/22671/3
| [Rust libp2p] | ❔ | [✅](https://github.com/libp2p/rust-libp2p/pull/5773) | ❔ |
| [JS libp2p] | ❔ | ❔ | ❔ |
| [Nim libp2p] | ❔ | ❔ | ❔ |
| [Java libp2p] | ❔ | ❔ | ❔ |
| [py-libp2p] | ❔ | ✅ | ❔ |

## Reference links

| Key | Implementation | Repository |
| --- | -------------- | ---------- |
| [Go libp2p] | Golang | <https://github.com/libp2p/go-libp2p-pubsub> |
| [Rust libp2p] | Rust | <https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub> |
| [JS libp2p] | JavaScript | <https://github.com/ChainSafe/js-libp2p-gossipsub> |
| [Nim libp2p] | Nim | <https://github.com/vacp2p/nim-libp2p/tree/master/libp2p/protocols/pubsub/gossipsub> |
| [Java libp2p] | Java / Kotlin | <https://github.com/libp2p/jvm-libp2p/tree/develop/libp2p/src/test/kotlin/io/libp2p/pubsub/gossip> |
| [py-libp2p] | Python | <https://github.com/libp2p/py-libp2p/tree/master/libp2p/pubsub> |

| Key | Spec / Proposal |
| --- | --------------- |
| [v1.0] | <https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md> |
| [v1.1] | <https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md> |
| [v1.2] | <https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md> |
| [v1.3-alpha] | <https://github.com/libp2p/specs/issues/687> |
| [Partial Messages] | <https://github.com/libp2p/specs/pull/685> |
| [Test Extension] | <https://github.com/seetadev/specs/blob/master/pubsub/gossipsub/extensions/experimental/test-extension.md> |
| [Large Message Segmentation] | <https://github.com/seetadev/specs/pull/2> |
| [Batch Publishing] | <https://ethresear.ch/t/improving-das-performance-with-gossipsub-batch-publishing/21713> |
| [IDONTWANT on first Publish] | <https://github.com/libp2p/go-libp2p-pubsub/issues/610> |
| [WFR Gossip] | <https://ethresear.ch/t/the-paths-of-least-resistance-introducing-wfr-gossip/22671/3> |