enh/1231-add-gossipsub-1.3-support

[Winter-Soren]

py-libp2p currently lacks full implementation of the GossipSub v1.3 specification (Extensions Control Message). The protocol ID /meshsub/1.3.0 is present and some extension plumbing exists, but the wire format, first-message semantics, and “at most once” rules from the spec are not satisfied. This issue tracks implementing v1.3 and the Topic Observation extension (from the ethresear.ch proposal) so that py-libp2p can interoperate with other implementations and support observing nodes that receive notifications without downloading full message copies.

What was wrong?

Issue #1231 — py-libp2p did not fully implement the GossipSub v1.3 specification (Extensions Control Message) or the Topic Observation extension.

Gaps included:

• Protobuf: rpc.proto did not match the spec: no ControlExtensions (single message with optional fields per extension), and no Topic Observation / TestExtension messages.
• First message: The first message on the stream (hello packet) was built without giving the router a chance to add an Extensions control message, so extensions were never sent in the first message.
• At most once: There was no tracking of “already sent Extensions” per peer, so the “MUST NOT be sent more than once” rule was not enforced.
• Duplicate Extensions: A peer sending Extensions more than once was not treated as misbehaviour (no downscore).
• Topic Observation: OBSERVE/UNOBSERVE and immediate IHAVE notifications to observers were not implemented, so observing nodes could not receive notifications without full message payloads.

---

How was it fixed?

• Spec-aligned protobuf: Updated libp2p/pubsub/pb/rpc.proto to add ControlExtensions (field 6 on ControlMessage) with topicObservation and testExtension plus ControlObserve, ControlUnobserve, and TestExtension on RPC. Regenerated rpc_pb2.py and rpc_pb2.pyi.

• Extensions state machine: Added libp2p/pubsub/extensions.py with PeerExtensions, ExtensionsState, and TopicObservationState. ExtensionsState handles:

  • Building our extensions into the first hello via build_hello_extensions(peer_id, hello).
  • Per-peer “sent extensions” tracking (at most once).
  • Parsing peer extensions from the first RPC and treating duplicate Extensions as misbehaviour (callback to scorer).

• First message injection: In pubsub.py’s _handle_new_peer(), after building the hello packet, we call the router’s extensions_state.build_hello_extensions(peer_id, hello) when the router has extensions_state and supports_v13_features (duck-typing + cast for type-checking). Only v1.3-capable routers add Extensions to the first message.

• GossipSub v1.3 wiring in gossipsub.py: Introduced PROTOCOL_ID_V13 (/meshsub/1.3.0), included it in add_peer and in protocol checks (e.g. _get_in_topic_gossipsub_peers_from_minus). Router now has extensions_state and topic_observation (TopicObservationState), initialised with optional my_extensions: PeerExtensions. In handle_rpc, we call extensions_state.handle_rpc(rpc, sender_peer_id) first for v1.3 peers, then dispatch handle_observe / handle_unobserve for Topic Observation. On remove_peer, we call extensions_state.remove_peer() and topic_observation.remove_peer().

• Topic Observation behaviour: Subscribers record observers via topic_observation.add_observer(topic, peer) on OBSERVE and remove them on UNOBSERVE. In publish(), after validating and caching a message, we call _notify_observers(topic_ids, msg_id) to send IHAVE to all observers of those topics immediately (not only at heartbeat). Added emit_observe / emit_unobserve and the corresponding handlers; observers are only accepted when the peer advertised the Topic Observation extension.

• Misbehaviour on duplicate Extensions: When a peer sends Extensions and we already have their extensions, we call _report_extensions_misbehaviour(peer_id), which uses scorer.penalize_behavior(peer_id, 1.0).

Result: v1.3 protocol ID is supported, Extensions are sent only in the first message and at most once per peer, duplicate Extensions are penalised, and Topic Observation (OBSERVE/UNOBSERVE + immediate IHAVE to observers) is implemented, with a small set of well-scoped stub methods left for new contributors.

To-Do

• Clean up commit history
• Add or update documentation related to these changes
• Add entry to the release notes

Cute Animal Picture

[seetadev]

@Winter-Soren , @IronJam11 : Neat progress, friends. Lets arrive at a good conclusion on this PR this week.

Please resolve CI/CD issues too.

This is a substantial and very thoughtful upgrade.

Bringing full GossipSub v1.3 support (including proper Extensions handling, “at most once” semantics, misbehaviour scoring, and Topic Observation with immediate IHAVE notifications) is a big step forward for py-libp2p interoperability. The separation via extensions.py and the clean state-machine approach make the implementation feel structured and spec-aligned rather than bolted on.

I especially appreciate:

• Spec-accurate protobuf updates and regeneration
• Proper first-message injection for extensions
• Enforcement + penalization of duplicate Extensions
• Well-scoped Topic Observation wiring
• Dedicated test suite for v1.3 extended behavior

Once CI issues and merge conflicts are resolved, this will be a major milestone for pubsub maturity in py-libp2p. Strong progress 🚀