the EDM port scanner is DISABLED.
currently does only Step 1 (quick punch to the anchor-observed address) and Step 2 (parallel punch to all known addresses over a 30s window). No port scan.
+
currently does only Step 1 (quick punch to the anchor-observed address) and Step 2 (parallel punch to all known addresses over a 30s window). No port scan.
@@ -535,9 +592,9 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }STUN probing at startup classifies mapping (EIM/EDM). Filtering is determined reliably via the anchor filter probe.
This probe runs once at startup (during anchor register cycle) and the result feeds into all subsequent InitialExchange payloads, so peers know each other’s exact filtering type.
At startup, after first anchor contact, each node with Unknown filtering sends a NatFilterProbe bi-stream request to its anchor. The anchor creates a temporary QUIC endpoint on a random port and attempts to connect to the node’s observed address (2s timeout). If the connection succeeds, the node is Open (address-restricted or better — accepts packets from any port on the anchor’s IP). If it times out, the node is PortRestricted.
The result feeds into all subsequent InitialExchange payloads, so peers know each other’s exact filtering type.
@@ -580,28 +637,27 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }
-
+
- 11. Port Mapping — UPnP-IGD + NAT-PMP + PCP
- Status: Complete (v0.7.2)
+ 10. Port Mapping — UPnP-IGD + NAT-PMP + PCP Implemented
Purpose
Asks the local gateway router to forward an external port to this node's local QUIC port. A successful mapping makes the node directly reachable from the internet without hole punching — any peer with the external address can connect immediately. Three protocols are attempted in parallel; the first router-response wins.
- Protocols (v0.7.2)
+ Protocols
- UPnP-IGD — long-standing consumer-router default. Discovery via SSDP multicast on 239.255.255.250:1900. Behavior varies; many routers ship with UPnP disabled by default.
- NAT-PMP (RFC 6886) — Apple lineage; widespread on routers that ever shipped Bonjour. Unicast to the gateway on UDP/5351.
- PCP (RFC 6887) — modern IETF-track successor to NAT-PMP. Unicast on UDP/5351. Supports both IPv4 NAT mapping and IPv6 firewall pinholes. Works on iOS without the multicast networking entitlement.
- Implementation uses the portmapper crate (also used by iroh internally). Replaces the v0.7.1 hand-rolled igd-next-only path.
+ Implementation uses the portmapper crate (also used by iroh internally), wrapped in the upnp.rs module (named for historical reasons; it covers all three protocols); network.rs holds the resulting mapping handle alongside the iroh Endpoint — port mapping is an Endpoint concern, not a connection concern.
Startup flow
bind Endpoint → spawn portmapper Client (UDP) → wait up to 3s for first protocol response → bootstrap (TCP mapping fires in parallel for HTTP serving)
- Probe all three protocols in parallel: portmapper's background service fires UPnP-IGD discovery + NAT-PMP unicast + PCP unicast concurrently. First success wins; failures from the others are absorbed silently.
- - UDP mapping for QUIC: maps the local QUIC port to an external port. Required for direct inbound. Address feeds N+10 identification, InitialExchange, anchor registration, and peer address advertisements.
- - TCP mapping for HTTP: separate parallel attempt for HTTP serving (see Section 25). Independent of UDP — either can succeed alone. Phones with permissive NAT can serve HTTP directly to browser fetches as of v0.7.2.
+ - UDP mapping for QUIC: maps the local QUIC port to an external port. Required for direct inbound. Address feeds self-identification, InitialExchange, anchor candidacy, and peer address advertisements.
+ - TCP mapping for HTTP: separate parallel attempt for HTTP serving (see HTTP Content Delivery). Independent of UDP — either can succeed alone. Phones with permissive NAT can serve HTTP directly to browser fetches.
- Per-platform behavior: All three protocols on desktop. On Android, a WiFi/Ethernet gate skips probing on cellular (no UPnP/PCP gateway exposed by carriers) and a
WifiManager.MulticastLock is held for the lifetime of the mapping so UPnP-IGD's SSDP responses actually arrive. On iOS, PCP and NAT-PMP work without the multicast entitlement; UPnP-IGD silently fails until the entitlement is granted.
@@ -616,22 +672,25 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }
Network roams between UPnP-capable WiFi networks self-heal. Mobile devices never auto-anchor regardless — cellular IPs look public but sit behind CGNAT.
+ Port mapping drives anchor candidacy
+ A node with a successful mapping is directly reachable from the internet — which is the only thing that makes an anchor an anchor. Anchor candidacy under v0.8 is exactly this: stable direct reachability (the watcher above) + opt-in, not a peer-count threshold. When the mapping succeeds on an opted-in non-mobile node, is_anchor is set and the node begins offering the convection service (see Anchors); when the mapping is lost, it reverts to non-anchor and peers prune the stale entry via failed-probe pruning. Any desktop on a home network with a mapping-capable router becomes a potential entry point for the network, with no manual server deployment.
+
Shutdown
Explicitly release the mapping on clean shutdown. Routers have finite mapping tables — releasing is good citizenship.
Integration with existing address logic
- The UPnP external address is treated the same as any other address the node knows about. It feeds into:
+ The mapped external address is treated the same as any other address the node knows about. It feeds into:
- - N+10 identification: Included in self-identification so peers store a routable address
+ - Self-identification: Included when identifying ourselves to peers so they store a routable address
- InitialExchange: Advertised to new connections
- - Anchor registration: Included in bootstrap/recovery registration
- - Social routing: Available in social route address cache for follows/audience
+ - Anchor convection: The address handed to convection callers as a referral (see Anchors)
+ - Social routing: Available in the social route address cache for follows/audience
- Relay introduction results: Returned alongside hole-punch candidate addresses
- - Share link host lists: The UPnP external address, when mapped for TCP, determines whether this node includes itself in share link host lists (see Section 26). A node only self-includes if it has confirmed TCP reachability — either via UPnP TCP mapping or a known public IPv6 address.
+ - HTTP serving eligibility: A node only serves browser traffic (or lists itself as a redirect target) if it has confirmed TCP reachability — either via a TCP port mapping or a known public IPv6 address (see HTTP Content Delivery)
Why this matters for mobile
- Mobile devices on cellular networks cannot use UPnP (carrier NAT doesn't expose it). However, if the peers they're trying to reach (especially desktop nodes and anchors) have UPnP mappings, those peers become directly reachable from the phone without hole punching. The phone doesn't need UPnP — the other side does.
+ Mobile devices on cellular networks cannot use UPnP (carrier NAT doesn't expose it). However, if the peers they're trying to reach (especially desktop nodes and anchors) have port mappings, those peers become directly reachable from the phone without hole punching. The phone doesn't need a mapping — the other side does.
Honest limitations
@@ -639,81 +698,79 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }| UPnP disabled on router | Some ISPs ship routers with UPnP off. Mapping silently fails, fallback to hole punch. |
| Double NAT | ISP modem + user router: mapping reaches inner router but not outer. Partial help at best. |
| Cellular networks | No UPnP at all. This is purely a desktop/home-network feature. |
- | Carrier-grade NAT (CGNAT) | ISP shares one public IP across many customers. UPnP maps to the ISP's NAT, not the internet. Same as double NAT. |
+ | Carrier-grade NAT (CGNAT) | ISP shares one public IP across many customers. Mapping reaches the ISP's NAT, not the internet. Same as double NAT. |
- Design principle: UPnP is a best-effort enhancement that improves direct connection reliability for the common case. It is not a dependency. The hole punch + relay fallback chain already handles all failure cases — UPnP just reduces how often you fall back to them.
+ Design principle: Port mapping is a best-effort enhancement that improves direct connection reliability for the common case. It is not a dependency. The hole punch fallback chain already handles all failure cases — mapping just reduces how often you fall back to it.
-
- UPnP nodes are anchors
- A node with a successful UPnP mapping is directly reachable from the internet — which is the only thing that makes an anchor an anchor. When UPnP mapping succeeds, the node self-declares as an anchor (is_anchor = true). Other peers will add it to their known_anchors table, providing diverse bootstrap paths back into the network.
- When the UPnP mapping is lost (lease renewal fails, shutdown), the node reverts to non-anchor. Peers that stored it as an anchor will naturally age it out via last_seen — LIFO ordering means stale anchors drop to the bottom. The 7-day post-bootstrap cleanup probes stale entries and removes failures. No special cleanup needed beyond the existing anchor infrastructure.
- This means any desktop on a home network with UPnP-capable router becomes a potential bootstrap point for the network, dramatically increasing the number of available anchors without any manual server deployment.
-
- Implementation
- Crate: igd-next (async support, well-maintained fork of igd). Implementation lives in network.rs alongside the iroh Endpoint — UPnP mapping is an Endpoint concern, not a connection concern.
-
+
- 12. LAN Discovery
- Status: Complete
+ 11. LAN Discovery Planned (passive lookup Implemented)
- mDNS-based LAN discovery is integrated via iroh's built-in MdnsAddressLookupBuilder. It works automatically — peers on the same local network are discovered and connected without manual configuration. iroh's mDNS address lookup broadcasts peer presence on the local network via multicast DNS (service name "irohv1", backed by the swarm-discovery crate).
+ What exists today: passive mDNS address lookup
+ iroh's built-in MdnsAddressLookupBuilder is registered on the endpoint (service name "irohv1", backed by the swarm-discovery crate). It resolves addresses passively: when this node already knows a peer's NodeId and tries to connect, mDNS supplies the peer's LAN address if the peer is on the same network segment, and the connection rides the local link. There is no subscribe loop, no proactive session establishment, and no LAN-specific treatment beyond address resolution.
+
+ v0.8 change: earlier revisions of this document badged the full active flow as complete; an audit found only the passive lookup exists. The active design below is kept as a low-priority target.
+
- Discovery flow
+ Active discovery flow
- Hold the mDNS handle: Build
MdnsAddressLookup explicitly (not via the endpoint builder) so we retain a clone for subscribing.
- Spawn a LAN scan loop: Call
mdns.subscribe().await to get a stream of DiscoveryEvent::Discovered and DiscoveryEvent::Expired events.
- - On discovery: Extract NodeId + LAN addresses from the event. If not already connected, initiate a direct connection + initial exchange. Register as a LAN session (a keep-alive session tagged as local).
+ - On discovery: Extract NodeId + LAN addresses from the event. If not already connected, initiate a direct connection + initial exchange. Register as a LAN session (its own planned session class — see below).
- On expiry: Clean up the LAN session. Peer left the network or powered off.
LAN sessions
LAN peers are special: zero-cost bandwidth, sub-millisecond latency, and very likely someone you know (same household/office). They deserve their own treatment beyond regular mesh or session slots:
- - Automatic keep-alive: LAN sessions stay open as long as the peer is on the network (mDNS heartbeat). No idle timeout. Not counted against session slot limits.
- - Sync priority: Pull sync and push notifications go to LAN peers first — instant delivery over the local link.
- - Local relay: LAN peers can relay for each other to the wider internet. A phone behind carrier NAT can relay through the desktop's UPnP-mapped connection. Bandwidth is free (local network), so relay limits can be much more generous than over the internet.
+ - Automatic keep-alive: LAN sessions stay open as long as the peer is on the network (mDNS heartbeat). No idle timeout. Not counted against session slot limits. This is a deliberate, explicitly-scoped exception to the no-standing-sessions rule (see Update Cadence & Keep-Alive), justified by zero-cost local bandwidth.
+ - Sync priority: Update checks and uniques exchanges go to LAN peers first — instant delivery over the local link.
+ - Local relay: relaying traffic between LAN peers costs nothing, so limits on LAN-local relaying can be much more generous than over the internet. Relaying a LAN peer out to the wider internet — e.g., a phone behind carrier NAT riding the desktop's port-mapped connection — is session relay: it consumes the serving device's WAN bandwidth and remains gated by
relay.session_relay_enabled, opt-in and OFF by default (see Relay & NAT Traversal). A same-owner linked-device carve-out may be proposed later, but only as an explicit opt-in extension, never as automatic behavior.
- Blob transfer: Large blob transfers between LAN peers are essentially free. Prefer LAN peers as blob sources when available.
Design rationale
- Today, two distsoc devices on the same WiFi network can only find each other if they happen to share a peer that reports them in N2. This is absurd — they're on the same network segment. LAN discovery turns mDNS from a passive address resolver into an active peer source, exploiting the fact that local bandwidth is essentially unlimited.
- The keep-alive + relay pattern means a household with one well-connected desktop and several phones creates its own mini-mesh: the desktop provides anchor-like connectivity, the phones stay connected through it, and everyone syncs instantly over the LAN even when the internet connection drops.
+ Today, two devices on the same WiFi network only connect over the LAN if one already knows the other's NodeId. That's a narrow win — they're on the same network segment and should find each other unaided. Active LAN discovery turns mDNS from a passive address resolver into an active peer source, exploiting the fact that local bandwidth is essentially unlimited.
+ The LAN-session pattern means a household with one well-connected desktop and several phones creates its own mini-mesh: the desktop provides anchor-like connectivity (routing the phones' traffic through it requires the owner's session-relay opt-in), and everyone syncs instantly over the LAN even when the internet connection drops.
- Implementation note: iroh's MdnsAddressLookup::subscribe() returns a Stream<DiscoveryEvent>. The DiscoveryEvent::Discovered variant includes EndpointInfo with NodeId + IP addresses. Custom user_data can be set via endpoint.set_user_data_for_address_lookup() to embed distsoc-specific metadata (e.g., display name) in the mDNS TXT record.
+ Implementation note: iroh's MdnsAddressLookup::subscribe() returns a Stream<DiscoveryEvent>. The DiscoveryEvent::Discovered variant includes EndpointInfo with NodeId + IP addresses. Custom user_data can be set via endpoint.set_user_data_for_address_lookup() to embed app-specific metadata (e.g., display name) in the mDNS TXT record. Low priority relative to the v0.8 mesh work.
-
+
- 13. Worm Search
- Status: Complete
- Used at step 4 of connect_by_node_id, after N2/N3 resolution fails.
+ 12. Worm Search Implemented
+ The network's point-query mechanism: given a specific NodeId, post ID, or blob ID, find someone who has it. Used at step 4 of connect_by_node_id, after direct-address and N2/N3 resolution fail. Under v0.8 it is the fallback behind the uniques index — deep N1–N4 knowledge answers most "who can reach X" questions locally, and worm search covers the remainder.
+
+
+
No keyword search — by design. Worm search takes exact IDs only. A global keyword flood-search was considered and rejected for v0.8: in a 20-wide mesh it degenerates into hop-TTL flooding whose abuse and jamming limits would destroy its value. Person-discovery is handled by the opt-in registry instead (see
Discovery & First Contact).
+
Algorithm
- - Build needles: target NodeId + target's N+10 (up to 10 preferred peers from their profile/cached N+10)
- - Local check: Search own connections + N2/N3 for any of the 11 needles. Also check local storage, CDN downstream tree, and blob store for any requested post/blob content.
- - Burst (500ms timeout): Send
WormQuery{ttl=0} (0x60) to all mesh peers in parallel. Each peer checks their local connections + N2/N3, plus local storage and CDN tree for post/blob content.
- - Nova (1.5s timeout): Each burst response includes a random "wide referral" — an N2 peer. Connect to those referrals and send
WormQuery{ttl=1}. The referred peer does its own 101-burst (fans out to all its mesh peers with ttl=0). This reaches ~10K nodes with only ~202 relay hops, keeping network pressure low by expanding one hop at a time rather than flooding.
+ - Build needles: target NodeId + up to 10 of the target's recent peers (from their stored profile) — 11 needles total. Finding anyone who can reach any needle gives a path to the target.
+ - Local check: Search own connections + N2/N3 for any needle. Also check local storage, known
file_holders, and the blob store for any requested post/blob content.
+ - Burst (500ms timeout): Send
WormQuery{ttl=0} (0x60) to all mesh peers in parallel. Each peer runs the same local check against its own knowledge.
+ - Nova (1.5s timeout): Each burst response includes a random referral to one of the responder's other peers (historically called a "wide referral," after the retired Wide slot tier). Connect to those referrals and send
WormQuery{ttl=1}; the referred peer fans out to its own mesh with ttl=0. Expansion is one hop at a time rather than a flood, keeping network pressure low.
- Total timeout: 3 seconds for the entire search.
+ With a ~20-peer mesh the raw two-hop fan-out is a few hundred nodes — smaller than the old 101-mesh's reach — but under the v0.8 knowledge model Rework each queried peer will answer from much deeper knowledge: its N1–N4 uniques (tens of thousands of IDs at 4 bounces) plus its file_holders map. (Today peers answer from N2/N3 share lists.) Coverage shifts from "how many nodes did the query touch" to "how much does each touched node know".
Content search
WormQuery carries optional post_id and blob_id fields, enabling unified search for nodes, posts, and blobs in a single query. Each peer checks:
- - Posts: local storage (direct match), CDN downstream tree (
post_downstream — up to 100 known hosts per post)
- - Blobs: local blob store, CDN post ownership (
get_blob_post_id → post_downstream)
+ - Posts: local storage (direct match), then the flat
file_holders map (up to 5 known holders per post — see Content Propagation)
+ - Blobs: local blob store, then CDN post ownership (
get_blob_post_id → file_holders)
- WormResponse carries post_holder and blob_holder fields alongside the existing node search results. A content hit (post or blob holder found) is treated as a successful response even without a node match.
- The CDN layer is the key multiplier: each node's downstream tree can cover hundreds of posts across dozens of hosts, giving every peer thousands of "I know where that is" answers. Combined with social layer knowledge, even a 202-hop nova covers enormous content space.
+ WormResponse carries post_holder and blob_holder fields alongside the node search results. A content hit (post or blob holder found) is treated as a successful response even without a node match.
PostFetch (0xD4/0xD5)
- Lightweight single-post retrieval after worm search identifies a holder. Opens a bi-stream to the holder and requests one post by ID. Much lighter than full PullSync — no follow filtering, no batch processing, just the target post.
+ Lightweight single-post retrieval after worm search identifies a holder. Opens a bi-stream to the holder and requests one post by ID. Much lighter than a full pull exchange — no filtering, no batch processing, just the target post.
Dedup & cooldown
@@ -723,226 +780,156 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }
-
-
- 14. Preferred Peers
- Status: Complete
-
- Negotiation (MeshPrefer, 0xB3)
-
- - Bilateral: Requester sends
MeshPrefer{requesting: true}, responder accepts/rejects
- - Acceptance: Both sides persist to
preferred_peers table, upgrade slot to PeerSlotKind::Preferred
- - Rejection reasons: "not connected", "preferred slots full (N/M)"
-
-
- Properties
-
- - Eviction-protected: Never evicted during rebalance (only non-preferred peers can be evicted)
- - Priority reconnect: Reconnected first in rebalance (Priority 0), before any growth
- - Pruned after 7 days unreachable: If a preferred peer can't be reached for 7 days, the slot is released. The bilateral agreement is cleared — reconnection requires a new MeshPrefer handshake. A reconnect watcher persists for 30 days at low priority (daily check). This prevents churn from aggressive pruning while ensuring slots aren't held indefinitely for offline peers.
- - N+10 component: Your 10 preferred peers' NodeIds are included in your N+10 for all identification (see Section 3)
- - Preferred tree: Each social route caches a
preferred_tree (~100 NodeIds) — the target's preferred peers' preferred peers. Used for relay selection.
-
-
-
-
-
- 15. Social Routing
- Status: Complete
- Caches addresses for follows and audience members, separate from mesh connections.
-
- social_routes table
-
- | Field | Purpose |
|---|
- node_id | The social contact's NodeId |
- nplus10 | Their N+10 (NodeId + 10 preferred peers) |
- addresses | Their known IP addresses |
- peer_addresses | Their N+10 contacts (PeerWithAddress list) |
- relation | Follow / Audience / Mutual |
- status | Online / Disconnected |
- last_connected_ms | When we last connected |
- reach_method | Direct / Relay / Indirect |
- preferred_tree | ~100 NodeIds for relay tree |
-
-
- Wire messages
-
- | Code | Name | Stream | Purpose |
|---|
- 0x70 | SocialAddressUpdate | Uni | Sent when a social contact's address changes or they reconnect |
- 0x71 | SocialDisconnectNotice | Uni | Sent when a social contact disconnects |
- 0x72 | SocialCheckin | Bi | Keepalive with address + N+10 updates |
-
-
- Reconnect watchers
- reconnect_watchers table: when peer A asks about disconnected peer B, A is registered as a watcher. When B reconnects, A gets a SocialAddressUpdate notification. Watchers pruned after 30 days. Low priority — daily check frequency for watchers older than 7 days.
-
- Social route lifecycle
-
- - Follow → store their N+10, upgrade to Mutual (if audience)
- - Unfollow → downgrade/remove
- - Approve audience → Mutual/Audience
-
-
-
-
+
- 16. Keep-Alive Sessions
- Status: Planned
+ 13. Update Cadence & Keep-Alive Rework
Purpose
- When the mesh 101 doesn't provide <N4 access to all the nodes we need for social and file operations, keep-alive sessions bridge the gap. These are long-lived connections that participate in N2/N3 routing but are not part of the mesh 101.
-
- Social/File connectivity check (every 60s)
- Periodically check whether we can reach every node we need. A node is considered reachable if either:
+ One scheduled system drives all recurring traffic: content freshness checks, uniques-list refresh, and connection keep-alive. Every node runs a check cycle whose per-item frequency descends from minutes to days, computed from two axes:
- - We have <N4 access to their N+10 (within N1/N2/N3), or
- - There is an anchor within N2 of them — we can ask that anchor to matchmake on demand without maintaining a persistent connection
+ - Freshness — how recently the content saw a comment, update, or engagement. Fresh content is checked often; stale content decays toward daily checks.
+ - Relationship tier — how much the local user cares, in descending priority: actively searching/engaging right now → own content (posts of ours held elsewhere) → messaged → followed → pinned → vouched → vouched-by → reacted-to. Untiered content sits at the floor cadence.
- Only when neither condition is met do we open a keep-alive session. With UPnP auto-anchors (see Section 11) scattered throughout the network, the odds of an anchor being within N2 of any given peer increase significantly, reducing the number of keep-alive sessions needed.
- Nodes to check:
-
- - Nodes we DM'd in the last 4 hours
- - All follows
- - All audience members
- - All file upstream peers (for blobs we host)
- - All file downstream peers (for blobs we serve)
-
- For any node whose N+10 is NOT reachable within N3, open a keep-alive session to the closest available node in their N+10 (or to them directly if possible). This ensures we can always find and reach our social and file contacts without worm search.
+ The effective interval is the product of both axes: a followed author's post commented on an hour ago might check every few minutes; a reacted-to post untouched for a month checks daily.
- Keep-alive session behavior
+ What rides the cycle
- - N2/N3 routing: Keep-alive sessions exchange N1/N2 diffs and participate in routing, similar to mesh connections. They expand our network knowledge without consuming mesh slots.
- - Not counted in mesh 101: Keep-alive sessions are a separate pool. They don't affect mesh diversity scoring or slot management.
- - Capacity limit: Max 50% of total session capacity is reserved for keep-alive sessions. The other 50% remains available for interactive sessions (DMs, group activity).
- - Not idle-reaped: Unlike interactive sessions (5-min idle timeout), keep-alive sessions persist as long as the connectivity need exists.
- - Reevaluated periodically: The 60s connectivity check closes keep-alive sessions that are no longer needed (e.g., the target now appears in N3 via a mesh connection).
+ - Content update checks: fetch new comments/edits for held posts from their holders (CDN-level, no mesh involvement).
+ - Uniques-list refresh: the uniques announce (two pools: forwardable N0–N2 + terminal N3 — see Network Knowledge) with mesh peers piggybacks on the same scheduler — it doubles as the mesh keep-alive.
+ - Replication traffic: holder-count maintenance and the author-concealment push (see Content Propagation) are deliberately scheduled through the same shaped cycle.
- Practical ceilings
-
- | Platform | Ceiling | Binding constraint |
|---|
- | Desktop | ~300–500 | Routing diff broadcast overhead — NodeListUpdate to all sessions every 120s. Memory and connection count are not the bottleneck. |
- | Mobile | ~25–50 | Battery (radio wake-ups per heartbeat cycle) and OS background restrictions (iOS/Android will kill background sockets). |
-
+ Same-author dedup
+ A successful check against one of an author's posts is evidence about all of them: their other posts' checks are deprioritized (pushed later in the cycle) rather than fired back-to-back. This is the main lever for finding the lowest viable cadence — per-author, not per-post, contact frequency.
- Mobile priority stack
- When approaching the mobile ceiling, keep-alive sessions are prioritized:
-
- - DMs last 30 min — active conversations take highest priority
- - Follows — people you follow
- - Audience — people following you
- - File peers — upstream/downstream blob hosts
-
- Lower-priority sessions are closed first to make room.
+ Jitter as traffic shaping
+ Every interval carries random jitter, and checks are spread rather than batched at tier boundaries. The goal is that a node's outbound request stream looks like uniform network overhead: an observer watching traffic timing should not be able to infer which authors the user cares about, which posts are theirs, or where content originated. Request-timing correlation is the primary deanonymizer in a P2P network; the cadence system is the defense (adversary model in Graph Privacy).
- Hysteresis
- Don't open a keep-alive session for a contact who just barely fell outside N3. Wait for persistent unreachability — the contact must be absent from N1/N2/N3 for multiple consecutive connectivity checks (e.g., 3 checks = 3 minutes) before opening a keep-alive. This prevents churn from nodes that transiently appear and disappear at the N3 boundary.
+ Mobile
+ Mobile stretches all cadence bands and batches wakeups — the radio is the battery cost, not the bytes. Background-restricted platforms run the cycle only while the foreground service is alive.
- Reject + redirect
- When a node is at its keep-alive session capacity (50% of total sessions), it refuses new keep-alive requests with a redirect — offering a random N2 node that also has <N4 access to the target. Same pattern as mesh RefuseRedirect but for the keep-alive pool. The requester tries the suggested peer instead.
+
+ v0.8 change: this replaces the previously planned "keep-alive session pool" (a 60s connectivity check holding up to 50% of session capacity open to social/file contacts). Deep N1–N4 knowledge plus CDN holder search make standing reachability sessions unnecessary — sessions open on demand and close when idle; only the shaped check cycle recurs.
+
- Cross-layer benefit
- Keep-alive sessions from the social and file layers feed N2/N3 entries back into the mesh layer. A social keep-alive to a friend's preferred peer might provide N2 entries that help the mesh growth loop. Similarly, a file keep-alive to an upstream host might provide access to nodes the mesh has never seen. The three layers compound each other's reach.
+ Implementation status
+ The seed exists: a freshness-tiered engagement check in storage (get_posts_due_for_engagement_check) schedules per-post checks at 5 min / 1 h / 4 h / 24 h keyed on last-engagement age. The relationship-tier axis, same-author dedup, jittered shaping, and the uniques-refresh piggyback are not built — the current code is the freshness axis only.
-
-
+
- 17. Content Propagation
+ 14. Content Propagation
+
Intent
"Attention creates propagation": when you view something, you cache it. The cache is optionally offered for serving. Hot content spreads naturally through demand. Cold content decays unless intentionally hosted.
- The CDN vision: every file by author X carries an author manifest with the author's N+10 and recent post list. If you hold any file by author X, you passively know X's recent posts and can find X through their N+10.
+ Content travels through the network exclusively via the CDN: the mesh provides width (who exists, who can reach whom — the N1–N4 uniques index), the CDN provides depth (the actual posts, blobs, comments, and updates). A pull tells you who to talk to; the CDN holder model is how the bytes move.
- Status: Partial
+
+ v0.8 change: the flat holder model replaces the CDN hosting tree (1 upstream + 100 downstream per blob, relay-down-tree ManifestPush, BlobDeleteNotice healing). There are no upstream/downstream relationships anywhere — every holder is a peer of equal standing.
+
+
+ The flat holder model Implemented
+ Every file (blob CID) has a single flat set of known holders in the file_holders table, capped at 5 per file with LRU eviction — the 5 most recently interacted-with holders survive, older entries are dropped. A holder entry records the peer's NodeId, last-known addresses, last interaction time, and a direction flag:
+
+ | Direction | Meaning |
|---|
+ sent | We served this file to the peer — they likely hold it now |
+ received | We fetched this file from the peer — they definitely held it |
+ both | Traffic has flowed both ways (entry promoted on conflict) |
+
+ Holder knowledge accretes as a side effect of every transfer: serving a blob registers the requester, fetching a blob registers the source, replication acceptance registers both ends, and engagement propagation refreshes entries. No registration protocol exists — the table is a passive record of observed reality, which is why the LRU cap is safe: stale entries age out under real traffic.
+
+ Holder redirect at capacity Implemented
+ When a node serving a BlobRequest already knows 5 holders for the file, it still serves the bytes but declines to register the requester (cdn_registered: false) and instead returns its holder list as redirect candidates. The requester records those peers as additional sources for future lookups. This spreads lookup load across the holder set instead of concentrating it on whichever node happens to be well-known.
+
+ Manifest freshness Implemented
- BlobRequest/BlobResponse (0x90/0x91) for peer-to-peer blob fetch- AuthorManifest (ed25519-signed, 10+10 post neighborhood) travels with blob responses
- - CDN hosting tree (1 upstream + 100 downstream per blob)
- - ManifestPush propagates updates down the tree
- - BlobDeleteNotice for tree healing on eviction
- - Blob eviction with social-aware priority scoring
+ - ManifestPush (
0x94): when an author posts again, updated AuthorManifests for their blobs are pushed to the file's known holders; a receiving holder re-pushes newly-stored manifests to its own connected holders of the same CID, excluding the sender. No tree structure — propagation follows the flat holder set.
+ - ManifestRefreshRequest/Response (
0x92/0x93): pull-side counterpart — "has this manifest changed since updated_at?" Used by the update-cadence cycle (see Update Cadence & Keep-Alive) to pick a refresh source from file_holders.
+ - BlobHeaderDiff/BlobHeaderRequest (
0xD0/0xD1): engagement state (comments, reactions, encrypted slots, FoF ops) travels as header diffs between holders; a header can be fetched without retransferring blob data.
- Passive discovery via neighborhood diffs
- Passive file-chain propagation is enabled through BlobHeader neighborhood diffs. Every blob header carries the author's 25+25 post neighborhood (25 previous + 25 following). When a host receives a BlobHeaderDiff (0x96), they learn about the author's newer posts without explicit subscription. Hosts of old content are pulled toward new content by the same author naturally — attention creates propagation.
+ Passive discovery via manifest neighborhoods
+ Every AuthorManifest carries a neighborhood of the author's surrounding PostIds. Holding any file by author X therefore passively reveals some of X's other recent posts — hosts of old content are pulled toward new content by the same author. Attention creates propagation. In v0.8 the neighborhood is deliberately small: it is capped to the author's concealment budget of 2–5 publicly revealed posts (below), not a full recent-history window.
- Remaining gaps
-
- | Gap | Impact |
|---|
- | N+10 not yet in file headers | Blob headers should include author N+10, upstream N+10, and downstream N+10s. Currently only AuthorManifest travels with blobs. |
- | No "fetch from any peer who has it" | Blobs are fetched from specific peers. No content-addressed routing ("who has blob X?"). |
-
+ Active replication cycle Rework
+ Authors own durability for their own recent content. Every 10 minutes, a node checks its own posts younger than 72 hours; any with fewer than 2 known holders are offered to connected peers via ReplicationRequest. Target selection ranks peers by device role (Available desktops > Persistent anchors > Intermittent phones) plus advertised cache pressure. Accepting peers store the blob and both ends record each other in file_holders.
+
+ Known bug (fix required): the cycle queries "own posts" by network NodeId, but posts are authored under posting IDs (split in v0.6.1) — the under-replicated set is always empty and the cycle is currently inert. The machinery is correct; the author lookup must move to posting identities.
+
+
+ Author concealment & traffic shaping Planned
+ The replication cycle is also the privacy mechanism. An author's node publicly holds and reveals only 2–5 of its own posts at any time; the surplus is pushed out to holder sets on other nodes via replication requests, so the full catalog is served from behind public mesh IDs rather than from the author's device. Combined with the jittered update cadence, the goal is that update and replication traffic is indistinguishable from uniform network overhead — an observer watching traffic cannot tell where content originates or which node is the author. The adversary model is detailed in Network Graph Privacy.
+
+ Deletion
+ There is no delete notification message. Deletes are signed control posts that propagate through the same manifest/pull machinery as any other post; holders that process one drop the content and its holder entries, and everyone else's copy ages out via LRU eviction. Details in Delete Propagation & Expiry.
-
+
- 18. Files & Storage
+ 15. Files & Storage
- Blob storage Complete
+ Blob storage Implemented
| Property | Value |
|---|
| CID format | BLAKE3 hash of blob data (32 bytes, hex-encoded) |
| Filesystem path | {data_dir}/blobs/{hex[0..2]}/{hex} (256 shards) |
- | Metadata table | blobs (cid, post_id, author, size_bytes, created_at, last_accessed_at, pinned) |
+ | Metadata table | blobs (cid, post_id, author, size_bytes, mime_type, created_at, stored_at, last_accessed_at, pinned) |
| Max blob size | 10 MB |
| Max attachments per post | 4 |
+ | Delivery budget | Per-hour outbound serving cap set by device role; auto-resets hourly. Requests beyond budget answered found: false. |
Blob content immutability
- Blob data is BLAKE3-addressed — the CID is the hash of the content. This means blob content is immutable by definition. Any mutable metadata (neighborhood, host lists, signatures) MUST be stored separately in a BlobHeader. Inline mutable headers are architecturally incompatible with content addressing.
+ Blob data is BLAKE3-addressed — the CID is the hash of the content. This means blob content is immutable by definition. Any mutable metadata (engagement, holder knowledge, signatures) MUST be stored separately in a BlobHeader. Inline mutable headers are architecturally incompatible with content addressing.
- BlobHeader Complete
- Mutable structure stored and transmitted separately from blob data. Carries engagement state, CDN metadata, and encrypted slots for private posts.
+ BlobHeader Implemented
+ Mutable structure stored and transmitted separately from blob data. Carries engagement state and encrypted slots for private posts; mutations travel as BlobHeaderDiff ops between holders.
BlobHeader {
post_id, // PostId this header belongs to
- author, // Author NodeId
- reactions, // Vec of public reactions (emoji + reactor NodeId + timestamp)
+ author, // Author posting ID
+ reactions, // Vec of public reactions (emoji + reactor + timestamp)
comments, // Vec of comments (text + author + timestamp + signature)
- policy, // Author-controlled comment/react policy
+ policy, // Author-controlled CommentPolicy (incl. FriendsOfFriends)
updated_at, // Timestamp of last header update
thread_splits, // Linked thread posts when comments exceed 16KB
receipt_slots, // Encrypted delivery/read/react receipt slots (private posts)
comment_slots, // Encrypted comment slots (private posts)
+ prior_author, // Original author before cross-identity post merge (optional)
}
-
- - Post neighborhood: 25 previous + 25 following PostIds. Forward slots are empty at publish time and populate via
BlobHeaderDiff propagation as the author continues posting. Empty forward slots are not an error condition.
- - Downstream host count:
min(100, floor(170MB / blob_size_bytes)) — smaller blobs allow more downstream hosts, larger blobs reduce the count to cap per-host storage overhead.
- - BlobHeaderRequest: Lightweight header-only fetch — retrieve just the header without retransferring blob data. Useful for neighborhood updates and host discovery.
- - Self Last Encounter: Stored per-author, becomes the newer of what's stored and "file last update." Determines when to trigger pull sync.
-
+ Header diffs also carry the FoF lifecycle ops (FoFRevocation, FoFAccessGrant, FoFKeyBurn — see Friend-of-Friend Visibility) and, in the v0.8 target, each comment carries its expiry timestamp fixed at creation (see Delete Propagation & Expiry).
- Blob transfer flow (0x90/0x91)
+ Blob transfer flow (0x90/0x91) Implemented
- Requester sends
BlobRequest { cid, requester_addresses }
- - Host checks local BlobStore:
+
- Host checks local BlobStore and delivery budget:
- - Has blob: Return base64-encoded data + CDN manifest + file header (N+10s, recent posts). Try to register requester as downstream (max 100). If full, return existing downstream as redirect candidates.
- - No blob: Return
found: false
+ - Has blob, budget OK: Return base64-encoded data + CDN manifest (with current holder count). If the host knows fewer than 5 holders, register the requester (
cdn_registered: true); otherwise return the known holders as cdn_redirect_peers.
+ - No blob / budget exhausted: Return
found: false
- - Requester verifies CID, stores blob locally, records upstream in
blob_upstream table. Updates Self Last Encounter for the author based on file header.
+ - Requester verifies CID, stores blob locally, records the host in
file_holders (direction received), and stores any redirect peers as additional holder knowledge.
- CDN hosting tree Complete
+ Manifests Rework
- - AuthorManifest: ed25519-signed by post author, contains post neighborhood (25 previous + 25 following posts — see BlobHeader above), author N+10, author addresses
- - CdnManifest: AuthorManifest + hosting metadata (host NodeId/addresses, source, downstream count)
- - Tree structure: Each blob has 1 upstream source + up to 100 downstream hosts
- - ManifestPush (
0x94): Author/admin pushes updated manifests downstream, which relay to their downstream
- - ManifestRefreshRequest/Response (
0x92/0x93): Check if manifest has been updated since last fetch
- - BlobDeleteNotice (
0x95): Notify tree when blob is deleted; includes upstream info for tree healing
+ - AuthorManifest: ed25519-signed by the post author (posting key). Contains the anchor post, creation/update timestamps, and the author's post neighborhood. In v0.8 the neighborhood is capped to the 2–5 post concealment budget and the manifest carries no device addresses — readers reach the author's content through
file_holders search, never through addresses tied to the posting identity.
+ - CdnManifest: AuthorManifest + serving-host metadata. The v0.8 wire format drops the legacy tree fields (source, source addresses, downstream count) as part of the clean protocol break (see Sync Protocol).
+
+
v0.8 change (privacy fix): today's AuthorManifest ships the device's real addresses under a posting-identity author and a 10+10 post neighborhood. Both violate the v0.8 privacy rulings: posting identities must be location-anonymous, and the neighborhood must not reveal more than the concealment budget. This fix is a prerequisite for the discovery registry (
Discovery & First Contact).
+
Blob eviction Complete
- priority = pin_boost + share_boost + (relationship * heart_recency * freshness / (peer_copies + 1))
+ Blob eviction Implemented
+ priority = pin_boost + share_boost + (relationship × heart_recency × freshness / (peer_copies + 1))
| Factor | Calculation |
|---|
pin_boost | 1000.0 if pinned, else 0.0. Own blobs auto-pinned. |
- relationship | 5.0 (us), 3.0 (mutual follow+audience), 2.0 (follow), 1.0 (audience), 0.1 (stranger) |
- heart_recency | Linear decay over 30 days: max(0, 1 - age/30d) |
+ share_boost | 100.0 at 3+ known holders; 50.0 × n / 3 for 1–2 holders; 0 otherwise. Holder count comes from file_holders (max 5) — content with a healthy holder set is being actively served and stays cached longer. |
+ relationship | 5.0 (own content), 2.0 (followed author), 0.1 (stranger). The own-content tier currently compares against the network ID and never matches post-split; own blobs stay protected via auto-pin. Fix rides the ID-class sweep (see Identity Architecture). |
+ heart_recency | Linear decay over 30 days since last access: max(0, 1 - age/30d) |
freshness | 1 / (1 + post_age_days) |
- peer_copies | Known replica count (from post_replicas, only if < 1 hour old) |
- share_boost | +100.0 if 3+ downstream peers (shared link with healthy distribution), scaled linearly for 1–2 downstream peers (33.3 per peer). Keeps shared content cached longer. |
+ peer_copies | Known replica count from post_replicas, counted only if confirmed within the staleness window |
+ Eviction is also the passive half of delete propagation: unreferenced blobs (orphaned by a delete control post that hasn't reached this node) simply score low and age out.
Pin modes Planned
The CDN is delivery infrastructure, not storage. Authors own durability. Pinning extends content in the local delivery pool — it is not a network obligation.
@@ -953,12 +940,12 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }
-
+
- 18b. Erasure-Coded CDN Replication Planned
+ 16. Erasure-Coded CDN Replication Planned
Problem
- The existing CDN hosting tree (Section 18) replicates full blob copies to downstream peers. This works well when the replicating node chose to pull the content — a follow relationship or explicit action establishes user consent. But the ReplicationRequest (0xE1) protocol also pushes content to infrastructure nodes that never chose to host it. A node holding a full copy of content it never reviewed faces potential liability for that content.
+ Full-copy replication works well when the replicating node chose to pull the content — a follow relationship or explicit action establishes user consent. But the ReplicationRequest (0xE1) protocol also pushes content to infrastructure nodes that never chose to host it. A node holding a full copy of content it never reviewed faces potential liability for that content.
Encryption does not solve this for public posts: the content is plaintext by definition. A different mechanism is needed that makes it technically impossible for a CDN node to possess reconstructable content.
Approach: sub-threshold erasure shards
@@ -970,10 +957,13 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }Tier | Storage | Defense |
| Author’s node | Full copy | Publisher responsibility (content originator) |
| Pulled content (follows) | Full copy | User consent — explicit follow relationship |
- | Private auto-replication | Encrypted (CEK envelope, Section 20) | Replicating nodes are provably not keyring recipients — existing encryption architecture handles this |
+ | Private auto-replication | Encrypted (CEK envelope, Encryption Architecture) | Replicating nodes are provably not keyring recipients — existing encryption architecture handles this |
| Public auto-replication | Erasure-coded shards | Sub-threshold shard — reconstruction impossible from any single holder |
+ Composition with the flat holder model
+ Each shard is content-addressed like any other blob and tracked through the same file_holders machinery, keyed on the shard's own CID. The 5-holder LRU cap applies per shard, so the 10 slots of a 3-of-10 chain are 10 independent holder sets — the cap and the slot count do not interact. Eviction scoring, holder redirect, and manifest refresh work unchanged; sharding is an additional layer on the auto-replication path only, and full copies continue to flow through the flat model untouched.
+
Shard assignment
Slot assignment is deterministic from the PostId via DHT-style hashing, carried in the existing BlobHeader metadata — no additional discovery round required. Each node enforces single-slot acceptance: it only accepts shard push offers for its assigned slot, rejecting others. This prevents a bad actor from accumulating multiple shards toward the reconstruction threshold. Slot assignment is acceptance policy, not exclusivity — transient duplicate holders for the same slot are harmless and add redundancy.
@@ -985,129 +975,169 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }
Interaction with full copies
- As content gains followers, the follow graph naturally absorbs redundancy through full-copy pull sync. The shard layer can back off:
+ As content gains followers, the follow graph naturally absorbs redundancy through full-copy pulls. The shard layer can back off:
- - 2+ full copies in mesh: equivalent to ≥4 live shards → shard chain deprioritizes, may decay
+ - 2+ known full copies: equivalent to ≥4 live shards → shard chain deprioritizes, may decay
- 1 full copy: shard chain reformation trigger
- 0 full copies: shard chain is sole redundancy, maintain aggressively
This means popular content automatically shifts from CDN shard infrastructure to the social follow graph. The shard layer only works hard for content nobody has explicitly chosen to keep — exactly the content with the highest liability exposure.
Re-replication
- When a slot goes dark, a new shard holder is assigned via DHT. The new holder determines which chunks belong to its slot and pulls only those chunks from the live shard holders that have them. No shard holder ever reconstructs the full content — each node only ever possesses its own slot’s chunks. The pulling node identifies what it needs, requests those specific chunks, and aggressively refuses anything outside its assigned slot. The author’s node can go offline permanently once mesh replication is established.
+ When a slot goes dark, a new shard holder is assigned via DHT. The new holder determines which chunks belong to its slot and pulls only those chunks from the live shard holders that have them. No shard holder ever reconstructs the full content — each node only ever possesses its own slot’s chunks. The pulling node identifies what it needs, requests those specific chunks, and aggressively refuses anything outside its assigned slot. The author’s node can go offline permanently once shard replication is established.
Replication window
- Active shard push replication applies the same 72-hour window as full-copy replication (Section 19): only posts less than 72 hours old are actively pushed to shard holders. Beyond 72 hours, the shard chain relies on natural decay and pull-based replication only.
+ Active shard push replication applies the same 72-hour window as full-copy replication: only posts less than 72 hours old are actively pushed to shard holders. Beyond 72 hours, the shard chain relies on natural decay and pull-based replication only.
Exception — share link re-promotion: When a share link is added to a post’s BlobHeader, the 72-hour active replication window resets from that event. This ensures that content being actively shared gets CDN-prioritized delivery regardless of original post age. The re-promotion window is 72 hours from the share link addition, not from the original post timestamp.
Implementation path
- Extends the existing ReplicationRequest/ReplicationResponse (0xE1/0xE2) protocol. Shard slot metadata fits in the existing BlobHeader. The CDN hosting tree, downstream registration, and eviction scoring (Section 18) continue to work unchanged for full copies — sharding is an additional layer for the auto-replication path only.
+ Extends the existing ReplicationRequest/ReplicationResponse (0xE1/0xE2) protocol. Shard slot metadata fits in the existing BlobHeader. Flat holder tracking and eviction scoring continue to work unchanged for full copies. Shard traffic also serves the traffic-shaping goal: shard pushes are one more class of replication traffic that makes origin-tracing harder.
-
-
- 19. Sync Protocol
+ 17. Sync Protocol Rework
- Wire format
- [1 byte: MessageType] [4 bytes: length (big-endian)] [length bytes: JSON payload]
- Max payload: 16 MB. ALPN: itsgoin/3.
-
- Pull sync: social + file layers, not mesh
- v0.2.0 change: Pull sync pulls posts from social layer peers (follows, audience) and upstream file peers, NOT from mesh peers. Mesh connections exist for routing diversity, not content. This separates infrastructure from content flow.
+ v0.8 change: clean protocol break. v0.8 has no wire compatibility with v0.7.x or earlier. All v0.6.x receive-only compat handlers (DeleteRecord 0x51, VisibilityUpdate 0x52), the dual pull-matching path (have_post_ids fallback), and serde-default field shims are removed rather than maintained. There is no ALPN negotiation and never was — a node speaks exactly one protocol version.
- Self Last Encounter: For each peer we sync with, we track the timestamp of our last successful sync. When Self Last Encounter ages beyond 4 hours, a pull sync is triggered. Self Last Encounter is updated to the newer of: (a) what's currently stored, or (b) the "file last update" timestamp from file headers received during blob transfers. Since file headers include the author's recent post list, downloading a blob from any peer hosting that author's content can update Self Last Encounter for the author.
- Pull sync filtering
-
- - PullSyncRequest: Includes requester's follow list + post IDs they already have
- - PullSyncResponse: Sender filters posts through
should_send_post():
-
- - Author is requester → always send (own posts relayed back)
- - Public + author in requester's follows → send
- - Encrypted + requester in wrapped key recipients → send
- - Otherwise → skip
-
-
-
+ Wire format Implemented
+ [1 byte: MessageType] [4 bytes: length (big-endian)] [length bytes: JSON payload]
+ Max payload: 64 MB. ALPN: itsgoin/3. Every message travels on its own QUIC stream — uni-directional for fire-and-forget pushes, bi-directional for request/response pairs. Unknown message-type bytes fail the stream; unknown fields inside known payloads are ignored by serde.
+
+ Decided: the v0.8 break bumps the ALPN to itsgoin/4, so pre-v0.8 nodes are refused cleanly at the QUIC handshake rather than failing mid-conversation. (The constant also gets renamed — it is currently named ALPN_V2 while holding itsgoin/3.)
+
- Message types (49 total)
+ Message types (46 total) Implemented
| Hex | Name | Stream | Purpose |
|---|
- 0x01 | NodeListUpdate | Uni | Incremental N1/N2 diff broadcast |
- 0x02 | InitialExchange | Bi | Full state exchange on connect |
- 0x03 | AddressRequest | Bi | Resolve NodeId → address via reporter |
+ 0x01 | NodeListUpdate | Uni | Incremental knowledge-layer diff broadcast |
+ 0x02 | InitialExchange | Bi | Full state exchange on connect: N1/N2 IDs, profile, post IDs, peer addresses, NAT observations, device role, anchor address |
+ 0x03 | AddressRequest | Bi | Resolve NodeId → address via tagged reporter |
0x04 | AddressResponse | Bi | Address resolution reply |
0x05 | RefuseRedirect | Uni | Refuse mesh + suggest alternative |
- 0x40 | PullSyncRequest | Bi | Request posts filtered by follows |
- 0x41 | PullSyncResponse | Bi | Respond with filtered posts |
- 0x42 | PostNotification | Uni | Lightweight "new post" push to social contacts |
- 0x43 | PostPush | Uni | Direct encrypted post delivery to recipients |
- 0x44 | AudienceRequest | Bi | Request audience member list |
- 0x45 | AudienceResponse | Bi | Audience list reply |
+ 0x40 | PullSyncRequest | Bi | Pull exchange (v0.8: uniques-index exchange — see below) |
+ 0x41 | PullSyncResponse | Bi | Pull exchange reply |
0x50 | ProfileUpdate | Uni | Push profile changes |
- 0x51 | DeleteRecord | Uni | Signed post deletion |
- 0x52 | VisibilityUpdate | Uni | Re-wrapped visibility after revocation |
- 0x60 | WormQuery | Bi | Burst/nova search for nodes, posts, or blobs beyond N3 |
+ 0x51 | DeleteRecord | Uni | Receive-only compat; deletes are signed control posts (Delete Propagation). Removed in v0.8. |
+ 0x52 | VisibilityUpdate | Uni | Receive-only compat; visibility re-wraps propagate via control posts. Removed in v0.8. |
+ 0x60 | WormQuery | Bi | Point query for nodes, posts, or blobs beyond local knowledge (worm search) |
0x61 | WormResponse | Bi | Worm search reply (node + post_holder + blob_holder) |
0x70 | SocialAddressUpdate | Uni | Social contact address changed |
0x71 | SocialDisconnectNotice | Uni | Social contact disconnected |
- 0x72 | SocialCheckin | Bi | Keepalive + address + N+10 update |
+ 0x72 | SocialCheckin | Bi | Keepalive + address + peer-address update |
0x90 | BlobRequest | Bi | Fetch blob by CID |
0x91 | BlobResponse | Bi | Blob data + CDN manifest + file header |
0x92 | ManifestRefreshRequest | Bi | Check manifest freshness |
0x93 | ManifestRefreshResponse | Bi | Updated manifest reply |
- 0x94 | ManifestPush | Uni | Push updated manifests downstream |
- 0x95 | BlobDeleteNotice | Uni | CDN tree healing on eviction |
- 0xA0 | GroupKeyDistribute | Uni | Distribute circle group key to member |
- 0xA1 | GroupKeyRequest | Bi | Request group key for a circle |
- 0xA2 | GroupKeyResponse | Bi | Group key reply |
- 0xB0 | RelayIntroduce | Bi | Request relay introduction |
+ 0x94 | ManifestPush | Uni | Push updated manifests to known file holders |
+ 0xA1 | GroupKeyRequest | Bi | Orphaned: no live sender, handler has an ID-class bug. Removed in v0.8 — group seeds travel as encrypted posts (Encryption). |
+ 0xA2 | GroupKeyResponse | Bi | Orphaned reply pair of 0xA1. Removed in v0.8. |
+ 0xB0 | RelayIntroduce | Bi | Hole-punch introduction via mutual peer (always-on signaling, not session relay) |
0xB1 | RelayIntroduceResult | Bi | Introduction result with addresses |
- 0xB2 | SessionRelay | Bi | Splice bi-streams (own-device default) |
- 0xB3 | MeshPrefer | Bi | Preferred peer negotiation |
+ 0xB2 | SessionRelay | Bi | Splice bi-streams (opt-in; relay.session_relay_enabled OFF by default on every role, including anchors — see Relay & NAT Traversal) |
+ 0xB3 | MeshPrefer | Bi | Preferred-peer negotiation. Removed in v0.8 — the preferred tier is eliminated. |
0xB4 | CircleProfileUpdate | Uni | Encrypted circle profile variant |
- 0xC0 | AnchorRegister | Uni | Register with anchor (bootstrap/recovery only) |
- 0xC1 | AnchorReferralRequest | Bi | Request peer referrals from anchor |
+ 0xC0 | AnchorRegister | Uni | Register with anchor. Removed in v0.8 — connection convection replaces the register loop (Anchors). |
+ 0xC1 | AnchorReferralRequest | Bi | Request peer referrals (v0.8: the convection call) |
0xC2 | AnchorReferralResponse | Bi | Referral list reply |
0xC3 | AnchorProbeRequest | Bi | A → B → C: test cold reachability of address |
0xC4 | AnchorProbeResult | Bi | C → A (success) or C → B → A (failure) |
- 0xD0 | BlobHeaderDiff | Uni | Incremental engagement update (reactions, comments, policy, thread splits) |
+ 0xC5 | PortScanHeartbeat | Uni | Dormant: part of the disabled EDM scanner; no live senders or handlers. Fate rides the raw-UDP scanner refactor. |
+ 0xC6 | NatFilterProbe | Bi | NAT filtering classification probe via third party |
+ 0xC7 | NatFilterProbeResult | Bi | Filtering probe result |
+ 0xD0 | BlobHeaderDiff | Uni | Incremental engagement update (reactions, comments, policy, slots, FoF ops) |
0xD1 | BlobHeaderRequest | Bi | Request full engagement header for a post |
- 0xD2 | BlobHeaderResponse | Bi | Full engagement header response (JSON) |
- 0xD3 | PostDownstreamRegister | Uni | Register as downstream for a post (CDN tree entry) |
+ 0xD2 | BlobHeaderResponse | Bi | Full engagement header response (JSON, only if newer) |
+ 0xD3 | PostDownstreamRegister | Uni | Holder registration: “I now hold this post” → sender's file_holders (name is historical; no tree exists) |
0xD4 | PostFetchRequest | Bi | Request a single post by ID from a known holder |
0xD5 | PostFetchResponse | Bi | Single post response (SyncPost or not-found) |
0xD6 | TcpPunchRequest | Bi | Ask holder to punch TCP toward browser IP |
0xD7 | TcpPunchResult | Bi | Punch result + HTTP address for redirect |
0xE0 | MeshKeepalive | Uni | 30s connection heartbeat |
- 0xE1 | ReplicationRequest | Bi | Request peer to cache specific posts |
+ 0xE1 | ReplicationRequest | Bi | Request peer to cache specific posts (Content Propagation) |
0xE2 | ReplicationResponse | Bi | Accept/reject replication request |
+ Retired type bytes, never reused: 0x42–0x45 (PostNotification/PostPush/AudienceRequest/AudienceResponse, v0.6.2), 0x95 (BlobDeleteNotice — holders evict via LRU, no notice needed), 0xA0 (GroupKeyDistribute — group seeds travel as encrypted posts).
- Engagement propagation
- Reactions, comments, and policy changes propagate via BlobHeaderDiff (0xD0) through the CDN tree:
+ Pull = uniques-index exchange Rework
+ In v0.8 a pull is not a post transfer. A pull exchanges uniques lists: the set of unique IDs each side can help the other reach — mesh peers, direct contacts, CDN file authors, and CDN file peers, including throwaway/anonymous IDs (deliberate noise that protects real identities). The semantics are “if you want one of these IDs, talk to me and I'll help you find their CDN holders to connect to and search.” The union of these lists across the mesh forms a distributed search index — the same index that powers person-discovery for the registry. What a node knows and announces at each depth (N1–N4, termination rules, dedup, size budgets) is specified in Network Knowledge; this section covers only the wire exchange.
+ Content itself never rides the pull path: posts and blobs travel through the CDN (PostFetchRequest/BlobRequest against holders resolved through the index — see Content Propagation). The old “never pull from mesh peers” principle dissolves cleanly: the mesh exchanges the index, the CDN carries the content.
+ The exact request/response payload shape (announce diffs, Bloom compression for mobile N4) is still open — see Network Knowledge.
+
+
What it replaces: today
PullSyncRequest (0x40) carries a merged query list plus per-author
since_ms timestamps (Self Last Encounter), and the responder returns posts newer than each timestamp after
should_send_post() visibility filtering, plus visibility updates for own posts the requester already holds. The cycle ticks every 60s: a full pull from all connected peers on the first tick, then pulls only for authors stale beyond 4 hours. This per-author post-pull dies with v0.8; its staleness-tiering idea survives inside the
update cadence system.
+
Recipient matching (merged pull) Implemented
+ The query list a node sends with a pull merges its public follows with every posting identity it holds, so DMs addressed to any of its personas match. The responder's filter, should_send_post(), sends a post when:
+
+ - Author is the requester → always send (own posts relayed back)
+ - Public or
FoFClosed + author in the query list → send (FoFClosed carries no recipient IDs on the wire; it propagates by author like public content)
+ Encrypted + any wrapped-key recipient in the query list → sendGroupEncrypted + any known group member in the query list → send- Otherwise → skip
+
+ The network NodeId is never an author or recipient (persona split), so it never appears in query lists. This matching machinery survives the v0.8 pull redefinition as the visibility filter on CDN content fetches.
+
+ Manifest machinery Implemented
+ Each post's AuthorManifest carries the author, creation/update timestamps, a post neighborhood (up to 10 previous + 10 following ManifestEntry items), and an ed25519 signature over the canonical digest. It travels wrapped in a CdnManifest alongside blob responses and via ManifestPush (0x94). On receipt:
- - Push (real-time): On react/comment, the diff is sent to both downstream peers (CDN tree children) and all upstream peers (up to 3, who we got the post from). Each intermediate node re-propagates both directions, excluding sender. This flows the diff up to the author and down to all holders.
- - Auto downstream registration: Nodes that receive a post via pull sync or push notification automatically send
PostDownstreamRegister (0xD3) to the sender, ensuring bidirectional diff flow.
- - Pull (safety net): Tiered frequency based on content age: 5min (<72h), 1hr (3-14d), 4hr (14-30d), 24hr (>30d). Requests
BlobHeaderRequest (0xD1) with local header timestamp. Peers respond with full header only if newer. Additive merge — store_reaction upserts, store_comment inserts with ON CONFLICT DO NOTHING. Writes batched per chunk (single lock acquisition).
- - Tombstones: Deleted reactions and comments are not hard-deleted. Instead, a
deleted_at timestamp is set on the record. Tombstones propagate via pull sync headers. Prevents deleted engagement from being re-introduced by peers that haven't yet received the deletion.
- - Planned: Pull engagement from both upstream and downstream peers to catch missed diffs from either direction.
+ - Signature verified; unsigned or forged manifests dropped
+ - Newer-wins: stored only if
updated_at beats the local copy
+ - Holder tracking: the pusher is recorded in
file_holders (they evidently hold the file)
+ - Relay: the manifest is re-pushed to connected file holders of the same CID, excluding the sender
+ - Post discovery: neighborhood post IDs from followed authors that are missing locally are fetched via
PostFetch, capped at 10 per push to avoid storms
+ ManifestRefreshRequest/Response (0x92/0x93) let any holder check manifest freshness for a CID and receive the newer copy. This is how a post's neighborhood keeps growing on CDN copies after the author publishes newer posts.
+
+
v0.8 change: manifests lose all device-address fields.
AuthorManifest.author_addresses and the
CdnManifest host metadata (
host_addresses,
source,
source_addresses,
downstream_count) link a posting identity to device locations — a privacy bug — and the tree fields describe a topology that no longer exists. Holder resolution goes through the uniques index +
file_holders instead. The neighborhood is also capped to the 2–5 post concealment budget (see
Files & Storage).
+
Engagement security Complete
+ Engagement propagation via file holders Implemented
+ Reactions, comments, policy changes, slot writes, and FoF ops propagate as BlobHeaderDiff (0xD0) batches through the flat holder set — there is no tree:
+
+ - Push: on receipt of a diff, the sender is recorded as a holder, ops are verified and applied, the header JSON is rebuilt from the authoritative tables, and the diff is re-propagated to all known
file_holders of the post (flat set, max 5), excluding the sender.
+ - Holder registration: nodes that receive a post via pull or manifest discovery send
PostDownstreamRegister (0xD3) to the sender, which simply records them in the sender's file_holders — ensuring diffs flow both ways.
+ - Pull (safety net): tiered polling by engagement freshness — 5 min (<72h), 1 hr (3–14d), 4 hr (14–30d), 24 hr (older) — via a single SQL pass over
last_engagement_ms/last_check_ms. BlobHeaderRequest (0xD1) carries the local header timestamp; peers respond with the full header only if theirs is newer. In v0.8 this cadence folds into the update cadence system with jitter as traffic shaping.
+ - Tombstones: deleted reactions and comments keep a
deleted_at timestamp instead of being hard-deleted, and propagate through header rebuilds — preventing re-introduction by peers that missed the deletion.
+
+
+ | Diff op | Purpose |
|---|
+ AddReaction / RemoveReaction | Signed reaction add; removal by reactor or post author |
+ AddComment / EditComment / DeleteComment | Signed comment lifecycle, gated by CommentPolicy |
+ SetPolicy | Author-controlled comment/react permissions + blocklist |
+ ThreadSplit | Register a comment-overflow post |
+ WriteReceiptSlot / WriteCommentSlot / AddCommentSlots | Encrypted slot writes (below) |
+ FoFRevocation / FoFAccessGrant / FoFKeyBurn | FoF key lifecycle ops, author-signed (Friend-of-Friend Visibility) |
+ Unknown | serde catch-all: ops from newer versions are silently skipped |
+
+
+ Engagement security Implemented
Engagement operations are cryptographically verified on receipt to prevent forgery and unauthorized modification:
- - Reaction signatures: Each reaction carries an ed25519 signature over
BLAKE3(reactor || post_id || emoji || timestamp_ms). Verified before storing. Unsigned reactions from older nodes accepted for backward compatibility (#[serde(default)] on signature field).
- - Comment signatures: Comments carry an ed25519 signature (existing field).
verify_comment_signature() now called on receipt via BlobHeaderDiff. Forged comments rejected.
- - Reaction removal: Only the reactor (original author of the reaction) or the post author can remove a reaction. Sender verified against
reactor and payload.author.
- - Comment edit/delete: Only the comment author can edit; comment author or post author can delete. Sender identity verified via QUIC transport authentication (iroh ed25519).
- - BlobHeader author: When rebuilding headers after engagement diffs, the author is verified against the stored post’s actual author, not trusted from the payload. Prevents relay nodes from misattributing headers.
- - BlobHeader source of truth: The
reactions and comments tables are authoritative. The blob_headers JSON is a derived snapshot rebuilt after each engagement operation. When they diverge (e.g., after a BlobHeaderResponse with a newer snapshot), the next engagement op rebuilds from tables.
+ - Reaction signatures: each reaction carries an ed25519 signature over
BLAKE3(reactor || post_id || emoji || timestamp_ms), verified before storing.
+ - Comment signatures:
verify_comment_signature() runs on every comment received via BlobHeaderDiff; forged comments are rejected. FoF-gated comments additionally pass the CDN-level pub_post_set gate (FoF).
+ - Reaction removal: only the original reactor or the post author can remove a reaction; the QUIC-authenticated sender is checked against both.
+ - Comment edit/delete: only the comment author can edit; comment author or post author can delete. Sender identity comes from QUIC transport authentication (iroh ed25519).
+ - Header author: rebuilt headers take the author from the stored post, never from the payload — relay nodes cannot misattribute.
+ - Source of truth: the
reactions and comments tables are authoritative; blob_headers JSON is a derived snapshot rebuilt after each engagement op.
+
+ v0.8 change: the empty-signature acceptance path for reactions (a v0.6.x #[serde(default)] compat shim) is removed with the clean break — signatures become mandatory.
+
- Device roles & bandwidth budgets Complete
- Each node advertises its device role in InitialExchange, which determines its bandwidth budgets for replication (pulling posts to cache) and delivery (serving requests from peers):
+ Encrypted receipt & comment slots Implemented
+ Private posts (Friends, Circle, Direct) carry encrypted slots in their BlobHeader for delivery receipts, read receipts, reactions, and private comments. The CDN propagates these as opaque bytes — only participants with the post's CEK can decrypt them. (FoFClosed posts skip this path; their wrap-slot mechanism covers both reads and comments.)
+
+ - Pre-filled noise: all slots are filled with random bytes at post creation; writing replaces noise with same-size ciphertext, so writes are indistinguishable from creation to observers.
+ - Slot key:
slot_key = BLAKE3_derive_key("itsgoin/slot/v1", CEK) — only participants who can decrypt the post can read or write slots.
+ - Receipt slots (64 bytes each): one per participant, index = position in the NodeId-sorted participant list. Decrypted:
[1B state][8B timestamp LE][23B emoji/padding]; states empty/delivered/seen/reacted. The author may pre-feed their own slot with a reaction at creation.
+ - Comment slots (256 bytes each):
ceil(participants / 3) initial slots. Decrypted: [32B author][8B timestamp LE][216B UTF-8 + padding]. First-available selection; when exhausted, any participant appends new noise-filled slots via AddCommentSlots.
+ - CDN-safe: relay nodes store and forward slot bytes without decryption; slots travel inside
BlobHeaderDiff, no dedicated messages.
+
+ DM conversations render these as delivery indicators: sent → delivered → seen checkmarks, emoji on react. Opening a conversation auto-marks incoming messages as seen.
+
+ Device roles & bandwidth budgets Implemented
+ Each node advertises its device role in InitialExchange, which sets hourly budgets for replication (pulling posts to cache) and delivery (serving requests from peers):
| Role | Replication / hour | Delivery / hour |
|---|
| Intermittent (phones) | 100 MB | 1 GB |
@@ -1115,201 +1145,53 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }Persistent (anchors) | 200 MB | 1 GB |
- - Budgets auto-reset every hour
- - Role is self-declared based on device type and advertised to peers in
InitialExchange
- - Peers respect advertised budgets when selecting replication targets
+ - Budgets auto-reset every hour; anchors reserve delivery headroom for browser-facing HTTP serving
+ - Role is self-declared from device type; peers respect advertised budgets when selecting replication targets (Content Propagation)
- Active CDN replication Complete
- All devices proactively replicate recent under-replicated posts to peers, not just passively serve on request:
-
- - 10-minute cycle: All devices initiate replication checks every 10 minutes
- - Target prioritization: Desktops > anchors > phones, scored by available bandwidth budget and connection quality
- - Selection criteria: Posts less than 72 hours old with fewer than 2 downstream replicas are selected for replication
- - Protocol:
ReplicationRequest (0xE1) asks a peer to cache specific posts; ReplicationResponse (0xE2) accepts or rejects based on available budget and storage
- - Graceful degradation: In small networks with few peers, the cycle runs but finds few or no viable targets — no wasted bandwidth. As the network grows, replication naturally increases.
-
-
- Connection rate limiting
+ Connection rate limiting Implemented
Incoming QUIC connections that fail authentication are rate-limited per source IP to prevent CPU exhaustion from rogue or stale nodes:
- First 3 failures: logged normally, connection attempts proceed
- 4+ failures: silently dropped with exponential backoff (2n-3 seconds, capped at ~256s)
- Successful connection: clears the failure count for that IP
- - Cleanup: stale entries removed every 60 seconds (pruned after 5 minutes of inactivity)
+ - Cleanup: stale entries pruned after 5 minutes of inactivity
- N2/N3 freshness
+ Knowledge freshness Implemented
+ Gossiped knowledge decays unless refreshed. Today's timers, which carry forward under the N1–N4 uniques model (Network Knowledge):
- - TTL: N2/N3 entries expire after 5 hours (pruned during rebalance cycle)
- - Full state broadcast: Every 4 hours, nodes re-broadcast their complete N1/N2 state (not just diffs) to catch any missed incremental updates
- - Disconnect cleanup: When a mesh peer disconnects, all N2/N3 entries they reported are immediately removed (
clear_peer_n2/clear_peer_n3)
- - Startup sweep: On boot, all N2/N3 and mesh_peers entries are cleared — they're stale from the previous session and will be rebuilt as connections establish
+ - TTL: N2/N3 entries expire after 5 hours (pruned during the rebalance cycle)
+ - Full state broadcast: every 4 hours, nodes re-broadcast complete state (not just diffs) to catch missed incrementals
+ - Disconnect cleanup: when a mesh peer disconnects, everything they reported is immediately removed (
clear_peer_n2/clear_peer_n3)
+ - Startup sweep: on boot, all gossiped entries and mesh-peer rows are cleared — stale from the previous session, rebuilt as connections establish
- Bootstrap isolation recovery
+ Bootstrap isolation recovery Implemented
Prevents network segments from becoming permanently isolated from the main network:
- - 24-hour check: Starting 24 hours after boot, nodes verify the bootstrap anchor is within their N1/N2/N3 reach
+ - 24-hour check: starting 24 hours after boot, nodes verify the bootstrap anchor is within their knowledge reach
- If absent: reconnect to bootstrap, request referrals, connect to referred peers
- - Sticky N1: The bootstrap is added to a sticky N1 set for 24 hours, so mesh peers learn about it via routing diffs and can independently bridge back to the main network
- - Self-limiting: Once the bootstrap is in N3, the check passes and no action is taken. Goes silent as the network grows.
+ - Sticky N1: the bootstrap joins a sticky N1 set for 24 hours, so mesh peers learn about it via routing diffs and can independently bridge back
+ - Self-limiting: once the bootstrap is within reach, the check passes silently; goes quiet as the network grows
- Schema versioning
+ Schema versioning Implemented
SQLite databases track their schema version via PRAGMA user_version. On startup:
- If the database version is older than
MIN_MIGRATABLE_VERSION, the database is reset (preserving identity key)
- If older than the current version, data migrations run once and the version is bumped
- - Schema-level changes (new tables, columns) are handled by
init_tables() (CREATE TABLE IF NOT EXISTS) and migrate() (column-level ALTER TABLE checks)
+ - Schema-level changes are handled by
init_tables() (CREATE TABLE IF NOT EXISTS) and migrate() (column-level ALTER TABLE checks)
- Upstream tracking (post_upstream)
- Each post tracks which peer it was received from in the post_upstream table (post_id → peer_node_id). Set during pull sync and push notification. Used for:
-
- - Engagement diff propagation toward the post author (hop-by-hop upstream)
- - Future: N+10 identification in blob headers (upstream source's N+10)
-
-
- IPv6 HTTP address advertisement
- Nodes with public IPv6 addresses advertise their actual routable address (from endpoint.addr().ip_addrs()) paired with their bound port, rather than the bind address (0.0.0.0). This enables direct browser-to-node HTTP serving for share links. Unroutable addresses (0.0.0.0, 127.x) are filtered out in the tiered web serving redirect path.
-
- Encrypted receipt & comment slots
- Private posts (Friends, Circle, Direct) carry encrypted slots in their BlobHeader for delivery receipts, read receipts, reactions, and private comments. The CDN propagates these as opaque bytes — only participants with the post’s CEK can decrypt them.
-
- Design principles
-
- - Pre-filled noise: All slots are filled with random bytes on post creation. Writing to a slot replaces noise with encrypted content of the same size, making writes indistinguishable from creation to observers.
- - Slot key derivation:
slot_key = BLAKE3_derive_key("itsgoin/slot/v1", CEK). Only participants who can decrypt the post can read/write slots.
- - CDN-safe: Relay nodes store and propagate slot bytes without decryption. No new protocol messages needed — slots travel via
BlobHeaderDiff.
-
-
- Receipt slots (64 bytes each)
-
- - Allocation: 1 per participant (including author)
- - Slot assignment: Participants sorted by NodeId; slot index = position in sorted list
- - Decrypted format:
[1 byte: state][8 bytes: timestamp_ms BE][23 bytes: emoji/padding]
- - States: 0=empty/noise, 1=delivered, 2=seen, 3=reacted
- - Author pre-feed: Author can write their own slot with a reaction emoji at creation time
-
-
- Comment slots (256 bytes each)
-
- - Allocation:
ceil(participants / 3) initial slots, expandable via AddCommentSlots diff op
- - Decrypted format:
[32 bytes: author_node_id][8 bytes: timestamp_ms BE][216 bytes: UTF-8 content + zero padding]
- - Slot selection: First available (all-zero content after decryption = available)
- - Growth: When all comment slots are used, any participant can append new noise-filled slots
-
-
- Wire operations
-
- | Op | Purpose |
|---|
- WriteReceiptSlot | Update a receipt slot (state change: delivered → seen → reacted) |
- WriteCommentSlot | Write encrypted comment to a slot |
- AddCommentSlots | Append new noise-filled comment slots when capacity is exhausted |
-
-
- UI
- DM conversations display delivery indicators: single checkmark (sent), double checkmark (delivered/on device), blue double checkmark (seen), emoji (reacted). Opening a conversation auto-marks incoming messages as seen. Messages have a react button for emoji responses.
+ IPv6 HTTP address advertisement Implemented
+ Nodes with public IPv6 addresses advertise their actual routable address (from endpoint.addr().ip_addrs()) paired with their bound port, rather than the bind address (0.0.0.0). This enables direct browser-to-node HTTP serving for share links. Unroutable addresses (0.0.0.0, 127.x) are filtered out in the tiered web-serving redirect path (HTTP Delivery).
-
- Protocol v4: Header-Driven Sync Complete
- Major sync protocol revision that replaces the current pull-everything-from-everyone model with header-driven discovery, per-author tracking, and tiered engagement polling. Reduces bandwidth by ~90% for established nodes.
-
- Core principle: headers as notification
- The AuthorManifest already carries a post neighborhood (20 previous + 20 following PostIds). When this neighborhood is pushed via ManifestPush (0x94) or travels with blob responses, receiving nodes can diff their local post list against the neighborhood to discover new posts without a full pull sync. The CDN tree becomes the notification system.
-
- Self Last Encounter (per-author sync tracking)
- Implements the v0.2.0 design intent: track last_sync_ms per followed author. Pull sync triggers only when now - last_sync_ms > check_interval. Updated on:
-
- - Successful pull sync response containing that author's posts
- - ManifestPush/BlobHeaderDiff received for that author
- - Blob transfer that includes that author's file header
-
-
- Slim PullSyncRequest
- Current: { follows: Vec<NodeId>, have_post_ids: Vec<PostId> } — sends ALL post IDs every request.
- v4: { follows: Vec<NodeId>, since_ms: HashMap<NodeId, u64> } — per-author timestamps. Response contains only posts newer than the requester's timestamp for each author. Drops request size from O(posts) to O(follows).
-
- Tiered pull sync frequency
- Instead of pulling from ALL mesh peers every 5 minutes, pull is driven by Self Last Encounter staleness:
-
- - Pull only triggers for authors where
last_sync_ms is stale AND no header updates were received in the interval
- - Default check interval: 4 hours (reduced from current 5-minute blanket pull)
- - Pull targets known CDN sources first (peers in
post_upstream / post_downstream for that author), falling back to mesh peers only if CDN sources are unavailable
- - Serial, not parallel — one peer at a time, stop when delta is received
-
-
- Tiered engagement check rates
- Engagement header polling (BlobHeaderRequest 0xD1) frequency scales with content age and activity:
-
- | Content age / last engagement | Check interval |
|---|
- | < 72 hours | 5 minutes |
- | 3–14 days | 1 hour |
- | 14–30 days | 4 hours |
- | > 30 days | 24 hours |
-
- DB tracks last_engagement_ms and last_check_ms per post. A single SQL query filters posts due for check:
- SELECT post_id FROM posts
-WHERE last_check_ms < now_ms - CASE
- WHEN last_engagement_ms > now_72h THEN 300000
- WHEN last_engagement_ms > now_14d THEN 3600000
- WHEN last_engagement_ms > now_30d THEN 14400000
- ELSE 86400000
-END
- Checks are serial (one peer at a time). A single “no new engagement” response is treated as authoritative — if that peer missed an update, it would have been replicated to them by now.
- On connect/wake: if since_last_check > check_rate for any post, an automatic check runs immediately.
-
- Multi-upstream (3 max)
- Extend post_upstream from 1 entry to up to 3 per post:
-
- - Primary upstream: First source of the post (current behavior)
- - Backup upstreams: Additional sources discovered via CDN tree or replication
- - Upstream unavailable: If primary is unreachable, promote backup. If no backups, use the CDN node map to find the author’s upstream chain and request connection — accepted or directed to an alternate via RefuseRedirect
- - Unregistration:
BlobDeleteNotice (0x95) already handles “I no longer hold this.” Extend to upstream: “I’m no longer your upstream for this post.”
- - Overhead is minimal: 3 rows per post vs 100 rows per post for downstream
-
-
- Auto-prefetch followed authors
- When a post by a followed author appears in any header (PullSyncResponse, ManifestPush, BlobHeaderDiff neighborhood), prefetch the post and its blobs if:
-
- - Post age < 90 days
- - Post not already in local storage
- - Within replication budget
-
- Encrypted posts without key: Store the encrypted post in DB marked as “not-for-us.” The node contributes to CDN availability for those who CAN decrypt it. Normal cache decay handles cleanup — these posts are given low eviction priority (stranger relationship score of 0.1) and will be evicted when space is needed.
-
- Header-driven post discovery flow
-
- - Author creates post → updates their AuthorManifest neighborhood (20 before + 20 after)
- - ManifestPush (0x94) propagates to downstream CDN tree
- - Each receiving node diffs the neighborhood against local post list
- - New post IDs discovered → fetch via PostFetch (0xD4/0xD5) from the peer that sent the manifest
- - Blobs prefetched for followed authors (within budget)
- - Node registers as downstream for the new post
- - Self Last Encounter updated for the author → suppresses pull sync for that author
-
- This makes the CDN tree the primary content distribution channel, with pull sync serving only as a safety net for missed headers.
-
- Migration path
- v4 is backward-compatible via ALPN negotiation. Nodes running v3 continue to work with the existing pull model. v4 nodes detect peer capability during InitialExchange and use the optimized paths when both sides support them. The v3 pull cycle remains as a fallback for mixed-version networks.
-
- New DB columns required
-
- | Table | Column | Purpose |
|---|
- posts | last_engagement_ms | Timestamp of most recent reaction/comment |
- posts | last_check_ms | Timestamp of last engagement check |
- follows | last_sync_ms | Self Last Encounter per followed author |
- post_upstream | (expand to 3 rows) | Multi-upstream with priority |
-
-
-
-
-
+
- 20. Encryption
+ 18. Encryption
- Envelope encryption (1-layer) Complete
+ Envelope encryption (1-layer) Implemented
- Generate random 32-byte CEK (Content Encryption Key)
- Encrypt content:
ChaCha20-Poly1305(plaintext, CEK, random_nonce)
@@ -1317,71 +1199,77 @@ END
- For each recipient (including self):
- X25519 DH:
our_ed25519_private (as X25519) * their_ed25519_public (as montgomery)
- - Derive wrapping key:
BLAKE3_derive_key("distsoc/cek-wrap/v1", shared_secret)
+ - Derive wrapping key:
BLAKE3_derive_key("itsgoin/cek-wrap/v1", shared_secret)
- Wrap CEK:
ChaCha20-Poly1305(CEK, wrapping_key, random_nonce) → 60 bytes per recipient
+ Recipients are posting identities (personas), never network identities. The wrapping key derivation runs against the recipient's posting pubkey; the sender wraps with the authoring persona's posting secret.
Visibility variants
| Variant | Overhead | Audience limit |
|---|
Public | None | Unlimited |
- Encrypted { recipients } | ~60 bytes per recipient | ~500 (256KB cap) |
+ Encrypted { recipients } | ~60 bytes per recipient | ~500 (design target; not enforced in code) |
GroupEncrypted { group_id, epoch, wrapped_cek } | ~100 bytes total | Unlimited (one CEK wrap for the group) |
- FoFClosed v0.7.0 | ~154 bytes per admitted V_x, padded | Bucketed (8/16/32/64/128/256, then +128 steps) |
+ FoFClosed | 98-byte wrap slot + 32-byte pub_post_set entry per admitted V_x, bucket-padded | Bucketed (8/16/32/64/128/256, then +128 steps) |
PostId integrity
PostId = BLAKE3(Post) covers the ciphertext, NOT the recipient list. Visibility is separate metadata. This means visibility can be updated (re-wrapped) without changing the PostId.
- Group keys (circles) Complete
+ Group keys (circles) Implemented
- Each circle gets its own ed25519 keypair
group_id = BLAKE3(initial_public_key) — permanent identifier- Group seed wrapped per-member via X25519 DH (KDF domain:
"distsoc/group-key-wrap/v1")
+ - Group seed wrapped per-member via X25519 DH (KDF domain:
"itsgoin/group-key-wrap/v1")
- Epoch rotation: On member removal, generate new keypair, increment epoch, re-wrap for remaining members
- - Wire:
GroupKeyDistribute (0xA0), GroupKeyRequest/Response (0xA1/0xA2)
+ - Distribution: the group seed travels as a normal encrypted post (
VisibilityIntent::GroupKeyDistribute) authored by the admin's persona, with each member's posting ID as a recipient. Members receive it via the ordinary CDN / pull paths and unwrap with their posting secret. No dedicated wire push exists — there is no wire-level signal that one endpoint is coordinating group membership with another.
+
+ v0.8 change: the orphaned GroupKeyRequest/Response (0xA1/0xA2) pair is deleted — its handler compares network IDs against posting-ID member lists and can never grant (ID-class bug family), and no sender was ever wired. Post-based distribution is the only path.
+
Three-tier access revocation
Three levels of revocation, chosen based on threat level:
-
Tier 1: Remove Going Forward (default)
+
Tier 1: Remove Going Forward (default) Implemented
Revoked member is excluded from future posts automatically. They retain access to anything they already received. This is the default behavior when removing a circle member — no special action needed.
When to use: Normal membership changes. Someone leaves a group, you unfollow someone. The common case.
Cost: Zero. Just stop including them in future recipient lists.
-
Tier 2: Rewrap Old Posts (cleanup)
-
Same CEK, re-wrap for remaining recipients only. The revoked member can no longer unwrap the CEK even if they later obtain the ciphertext. Propagate updated visibility headers via VisibilityUpdate (0x52).
+
Tier 2: Rewrap Old Posts (cleanup) Rework
+
Same CEK, re-wrap for remaining recipients only. The revoked member can no longer unwrap the CEK even if they later obtain the ciphertext. The updated visibility propagates as a signed control post authored by the post's persona — peers verify the control post's signature against the target's author before applying, so only the author can rewrite visibility.
When to use: Revoked member never synced the post (common with pull-based sync — encrypted posts only sent to recipients). You want to clean up access lists.
Cost: One WrappedKey operation per remaining recipient, no content re-encryption.
+
Required fix: the entry-point guard compares post.author (a posting ID) against the node's network ID — always different since the identity split — so every revoke currently bails before doing anything, and the rewrap is keyed with the wrong identity pair. Same ID-class bug family as the group-key handler. The control-post propagation machinery underneath is correct.
-
Tier 3: Delete & Re-encrypt (nuclear)
-
Generate new CEK, re-encrypt content, wrap new CEK for remaining recipients, push delete for old post ID, repost with new content but same logical identity. Well-behaved nodes honor the delete.
+
Tier 3: Delete & Re-encrypt (nuclear) Rework
+
Generate new CEK, re-encrypt content, wrap new CEK for remaining recipients, issue a control-post delete for the old post ID (see Delete Propagation), repost with new content but same logical identity. Well-behaved nodes honor the delete.
When to use: Revoked member already has the ciphertext and could unwrap the old CEK. Only for content that poses an actual danger/risk if the revoked member retains access. Recommended against in most cases.
Cost: Full re-encryption + delete propagation + new post propagation. Heavy.
+
Required fix: shares the Tier-2 entry-point author-check bug.
Trust model: The app honors delete requests from content authors by default. A modified client could ignore deletes, but this is true of any decentralized system. For legal purposes: the author has proof they issued the delete and revoked access.
- Private profiles (Phase D-4) Complete
+ Private profiles Implemented
Different profile versions per circle, encrypted with the circle/group key. A peer sees the profile version for the most-privileged circle they belong to. CircleProfileUpdate (0xB4) wire message. Public profiles can be hidden (public_visible=false strips display_name/bio).
-
+
- 20a. Friend-of-Friend Visibility v0.7.0
+ 19. Friend-of-Friend Visibility Implemented
-
Distinct from directory vouches. The "FoF vouch" described here is a
cryptographic primitive for post readership and comment gating (per-persona symmetric key
V_me). It is unrelated to the
directory vouch system in
section 27, which governs discovery-layer trust and bot-ring resistance. The two share vocabulary but operate at different layers.
+
Distinct from directory vouches. The "FoF vouch" described here is a
cryptographic primitive for post readership and comment gating (per-persona symmetric key
V_me). It is unrelated to the
directory vouch system in the
Directory Trust Layer, which governs discovery-layer trust and bot-ring resistance. The two share vocabulary but operate at different layers.
The problem
@@ -1394,31 +1282,32 @@ END
| Public | All readers (unchanged) |
| Friends-only | Personas you have vouched for |
| Friends-of-Friends | Your vouchees + every vouchee of anyone who vouched for you (emergent FoF) |
- | Custom v2 | Author-selected subset of held vouch keys |
+ | Custom Planned | Author-selected subset of held vouch keys |
Vouches are NOT delivered via DM. Instead, the voucher publishes anonymous HPKE-sealed wrappers (one per recipient) inside their bio post. HPKE (RFC 9180) provides recipient anonymity — wrapper ciphertext reveals nothing about the recipient's pubkey. Each wrapper is 48 bytes (32-byte sealed V_me + 16-byte AEAD tag); one shared ephemeral pubkey per batch.
Vouches are NOT delivered via DM. Instead, the voucher publishes anonymous sealed wrappers (one per recipient) inside their bio post. The sealing is an HPKE-style construction — per-batch ephemeral X25519 keypair, ECDH against the recipient's persona key, BLAKE3-derived wrapping key and nonce (domain itsgoin/vouch-grant/v1, bound to the bio post's ID), ChaCha20-Poly1305 seal. It is not RFC 9180 HPKE, but provides the same recipient anonymity property: wrappers carry no recipient identifier, the KDF context is recipient-free, and every wrapper in a batch shares one ephemeral pubkey. Each wrapper is 48 bytes (32-byte sealed V_me + 16-byte AEAD tag).
Readers auto-scan bio posts of accounts they follow, trial-decrypting each wrapper against each of their personas. Cost is ~60µs per wrapper per persona on mobile — a 200-wrapper bio scanned against 3 personas is ~36 ms. The bio's VouchGrantBatch is padded with dummy wrappers and shuffled on every publish so observers see neither vouch-set size nor change-targets.
Any failure → drop, do not forward. This kills the bandwidth-amplification attack that a single admitted-but-malicious FoF member could otherwise mount: their forgeries cannot pass the propagation gate.
Persona-wide V_me rotation
@@ -1464,202 +1356,290 @@ END
Body encryption, wrap slots, and HKDF/HMAC are all symmetric — PQ-safe. Comment signing uses Ed25519 today; the spec shape is algorithm-agnostic so ML-DSA-65 (~2 KB pubkey, ~3.3 KB signature) can substitute, optionally with a Merkle-commit variant on pub_post_set to keep header size bounded.
Implementation
-
Full crypto-level byte layouts, data models, wire-format additions, ship criteria, and integration tests are specified in docs/fof-spec/. The implementation is layered for bottom-up shipping:
+
Full crypto-level byte layouts, data models, wire-format additions, ship criteria, and integration tests are specified in docs/fof-spec/. All five layers shipped in v0.7.0:
| Layer | Scope | Status |
|---|
- | 1 | Vouch primitive (V_x keys, keyring, bio-post HPKE wrappers, scan policy) | v0.7.0 |
- | 2 | Mode 2: public posts with FoF-gated comments, CDN-level verification | v0.7.0 |
- | 3 | Mode 1: FoFClosed body + wrap slots + anonymous prefilter | v0.7.0 |
- | 4 | Rotation, revocation, key lifecycle (grandfather + cascade + key-burn) | v0.7.0 |
- | 5 | Unlock cache + prefilter optimization (perf-critical at scale) | v0.7.0 |
+ | 1 | Vouch primitive (V_x keys, keyring, bio-post sealed wrappers, scan policy) | Implemented |
+ | 2 | Mode 2: public posts with FoF-gated comments, CDN-level verification | Implemented |
+ | 3 | Mode 1: FoFClosed body + wrap slots + anonymous prefilter | Implemented |
+ | 4 | Rotation, revocation, key lifecycle (grandfather + cascade + key-burn) | Implemented |
+ | 5 | Unlock cache + prefilter optimization (perf-critical at scale) | Implemented |
-
+
- 21. Delete Propagation
- Status: Complete
+ 20. Delete Propagation & Comment Expiry
- Delete records
- DeleteRecord { post_id, author, timestamp_ms, signature } — ed25519-signed by author. Stored in deleted_posts table (INSERT OR IGNORE). Applied: DELETE from posts table WHERE post_id AND author match.
+ Deletes are signed control posts Implemented
+ A delete is a control post: a small public post with VisibilityIntent::Control, signed by the target's authoring persona, naming the PostId to delete. It propagates exactly like any other post — it enters the author's neighbor manifests (following_posts lists on their other posts) and rides manifest diffs out to file_holders; anyone following any of the author's posts picks it up via the normal CDN / pull paths.
+ On receipt, a node verifies the control post's signature against the target post's author before applying. Application is atomic with the control-post insert: record in deleted_posts (INSERT OR IGNORE), then DELETE the target from posts. The deleted_posts table also guards against re-ingesting the deleted post from a stale holder later.
- Propagation paths
+
+ v0.8 change: the direct-push delete paths are gone entirely — DeleteRecord (0x51) uni-stream push and BlobDeleteNotice (0x95) were retired in v0.6.2, and the receive-only 0x51/0x52 compat handlers die with the v0.8 clean protocol break. Control posts via manifests are the only delete path.
+
+
+ Blob cleanup on delete Implemented
- - InitialExchange: All delete records exchanged on connect
- - DeleteRecord message (
0x51): Pushed via uni-stream to connected peers on creation
- - PullSync: Included in responses for eventual consistency
+ - Locally: blob metadata and CDN records for the post's blobs are removed, blob files deleted from disk.
+ - Remotely: no notice is sent. Copies held by other
file_holders become orphans — nothing references them, nothing re-shares them — and are evicted naturally by the CDN's LRU eviction cycle (see Content Propagation).
- CDN cascade on delete
-
- - Send BlobDeleteNotice to all downstream hosts (with our upstream info for tree healing)
- - Send BlobDeleteNotice to upstream
- - Clean up blob metadata, manifests, downstream/upstream records
- - Delete blob from filesystem
-
+ Comment expiry: rand(30–365 days) TTL Planned
+ Every comment gets an expiry timestamp at creation: created_at + rand(30–365 days). The expiry is fixed at creation, signed, and carried with the comment — it is part of the comment's identity, not a per-holder policy. Every holder in the network expires the comment at (near) the same moment.
+
+ - Why carried, not local: if each holder chose its own retention, staggered disappearance would fingerprint individual holders and their policies. A carried timestamp makes expiry a network-wide event that reveals nothing about who holds what.
+ - Why random: a fixed TTL would let observers date a comment's creation from its expiry. A 30–365-day uniform draw breaks that inference.
+ - Identity hygiene: throwaway persona IDs used for greeting comments (Discovery & First Contact) exist in the network only through their comments. When the comments expire, the IDs vanish from uniques lists entirely — the network forgets them by construction.
+ - No silent extension: the TTL cannot be edited — re-posting the content is a new comment with a new ID and a fresh draw.
+
-
+
- 22. Social Graph Privacy
- Status: Complete
+ 21. Social Graph Privacy & Traffic Shaping
+
+ What is never shared
- - Follows are never shared in gossip or profiles
- - N1 share merges mesh peers + social contacts into one list (indistinguishable)
- - No addresses ever shared in routing updates
- - N3 is never shared outward (search-only)
+ - Follows are never shared in gossip or profiles. Implemented
+ - Uniques announcements carry bare IDs — except anchors. The uniques announce (see Network Knowledge) ships NodeIds without addresses; the sole exception is anchor-flagged entries, whose addresses ride the pools by design (self-declared reachable infrastructure — publishing them links nothing to any person). Rework (today's N1/N2 share lists are fully address-free; the uniques format adds the anchor exception)
+ - N4 is never re-announced — knowledge terminates at the fourth bounce, so an observer M hops away cannot enumerate your neighborhood through transitive gossip. Planned
+ - Posting identities never map to device addresses on the wire. Addresses attach to network identities only; the persona split (see Identity Architecture) keeps the two namespaces unlinkable. Rework — today's
AuthorManifest.author_addresses ships the device's real addresses under a posting-identity author. This is a privacy bug; the v0.8 manifest carries no device addresses, and the fix must land before or with the discovery registry (location-anonymity dependency).
+
+
+ Where addresses do travel, and why
+ Routing needs somewhere to connect. Addresses appear in exactly these payload classes, all scoped to network identities:
+
+ - Social-routing peer addresses: the initial exchange includes addresses of our currently connected mesh peers. These are network endpoints an observer at that vantage could already see us talking to — sharing them reveals mesh topology (public by design), not social links.
+ - Point lookups: worm address-requests and lookup responses return addresses for a specific queried network ID.
+ - Connection convection referrals: anchors hand out addresses of recent callers, and
RelayIntroduce coordination carries both ends' addresses for hole-punching (see Anchors).
+
+
+ Uniques lists as cover traffic
+ The N1 uniques list deliberately merges mesh peers, social contacts, CDN file authors, CDN file peers, and throwaway/anonymous IDs into one flat set. An observer diffing your announcements over time is looking at a list dominated by high-churn CDN-derived entries and deliberate throwaway-ID noise; the stable social core does not stand out the way it did when the share list was just mesh-plus-contacts. Throwaway IDs additionally age out of the network via comment TTL (above), so the noise floor is self-renewing rather than monotonically accumulating. Planned (today's N1 share merges only mesh peers + social contacts)
+
+ The timing adversary Planned
+ With content encrypted and recipient lists off the wire, the remaining deanonymizer is request-timing correlation: an observer who can watch traffic sees who fetches what, when. If your device checks an author's CDN seconds after every new post, you are that author's reader; if fresh content always radiates outward from one node first, that node is the author. The v0.8 defenses are behavioral, not cryptographic:
+
+ - Jittered, tiered update cadence: CDN update checks run on a descending-frequency schedule (minutes → days) keyed on freshness × relationship tier, with jitter on every timer (see Update Cadence). No check fires because a post appeared; checks fire because a schedule slot came up.
+ - Replication shaped as overhead: replication pushes are paced and jittered so an author seeding their own content is indistinguishable from a holder performing routine redundancy maintenance. The goal: update + replication traffic reads as uniform network overhead, with no observable "origin burst".
+ - 2–5 visible posts per author: the public CDN surface reveals at most 2–5 posts per author at any time; the surplus is pushed out to holders via replication requests and reached through the uniques index. This caps what enumeration of any single node's public holdings discloses about any author's corpus — and hides the author's own node behind ordinary public mesh IDs.
+ - Comment TTL: long-lived comment trails are the easiest correlation anchors across time; the 30–365-day expiry bounds how long any trail exists.
- Known temporary weakness: An observer who diffs your N1 share over time can infer your social contacts (they're the stable members while mesh peers rotate). This will be addressed when CDN file-swap peers are added to N1, making the stable set larger and harder to distinguish.
+ Honest scope: this defeats passive observers correlating timing and public holdings. A global adversary who can watch every link, or an adversary who compromises your own device, is out of scope — as in every friend-to-friend design.
-
-
- 23. Identity Management
+ 22. Identity Management
- Multi-identity per device Complete
+ Multi-identity per device Implemented
A single device can hold multiple identities, each with its own ed25519 keypair, database, blob store, follows, and posts. One identity is active at a time — switching performs a hot-swap (Node teardown + rebuild, ~3-5 seconds).
- Directory structure:
itsgoin-data/identities/{node_id_hex}/ — each identity gets its own subdirectory with identity.key, itsgoin.db, blobs/, and meta.json
- - Legacy migration: Flat
itsgoin-data/ layout auto-migrates to per-identity subdirectories on first launch
- Create, import, switch, delete via Settings UI
- Key permissions:
identity.key files written with 0600 permissions (Unix)
- Multi-device identity Planned
- Multiple devices share the same identity key (ed25519 keypair, same NodeId). All devices ARE the same node from the network’s perspective. Posts from any device appear as the same author.
+ Multi-device: posting-key bundles Implemented
+ Devices are linked by sharing posting keys, never network keys. Each device keeps its own network key (its own QUIC NodeId); what travels between your devices is the export bundle — a ZIP containing the identity key, all posting identities, posts, blobs, follows, and settings. Transfer happens out-of-band between your own devices; the network sees no linking message.
- - Setup: Export
identity.key from one device, import on another using the identity management UI
- - Device identity: Each device generates a unique device keypair for self-routing and conflict resolution (planned)
- - Own-device relay: Route traffic through your own devices (planned)
+ - Export: Settings → Export produces the ZIP bundle. All posting keys held by the identity are included (
posting_identities.json), so every persona restores on the target device.
+ - Import as personas (the multi-device path): the bundle’s posting keys are added to the current identity’s persona set and its posts are inserted as-authored — original PostIds, authors, and signatures intact. Content encrypted to any imported key becomes decryptable because the device now holds those secrets. Idempotent; duplicates are skipped.
+ - Import as new identity: alternatively, the bundle can be imported as a separate identity on the device (creates the identity subdirectory from the bundle’s key; data restores on first switch).
+ - QR / file-share linking flows for pairing a new device without manual file transfer Planned
+
+ Raw key import restores the network key only. Pasting a bare identity.key hex string creates an identity around that network key — it carries no posting identities, posts, or follows. To move personas between devices, use the ZIP bundle.
+
+
+
v0.8 change: the earlier “same identity key on all devices” multi-device design is deleted — it predates the network/posting key split and contradicted it. Devices never share a network key; see
Identity Architecture.
+
Post import & merge Planned
- Import posts from another identity into the current one. Public posts imported directly. Encrypted posts require the original identity’s key for decryption, then re-encrypted under the current identity. Merge creates new posts (new PostId, new author) with original timestamps preserved and prior author noted in BlobHeader.
+ Post import & merge Implemented
+ Two ways to bring another identity’s posts into the current one, both wired into the import wizard:
+
+ - Import public posts: public posts from the bundle are re-created under the current identity with new PostIds.
+ - Merge with key: supply the original identity’s key; encrypted posts are decrypted with it, then re-created under the current identity. Posts the original identity could not read are skipped. Original timestamps are preserved and the prior author is recorded in the BlobHeader for provenance.
+
-
- 24. Phase 2: Reciprocity (Reconsidered)
- Status: Reconsidered
+ 23. Reciprocity (Reconsidered) Planned
The original Phase 2 design centered on hosting quotas (3x rule), chunk audits, and tit-for-tat QoS. On reflection, the attention-driven delivery model makes quota enforcement unnecessary. The CDN is a delivery amplifier, not a storage system — hot content propagates through demand, cold content decays. Authors are responsible for their own content durability.
Tit-for-tat QoS solves the wrong problem: it optimizes for fairness in a storage-obligation model that no longer exists. What matters is that the delivery network functions efficiently, which it does through natural attention dynamics.
If reciprocity mechanisms are needed at scale, they should address delivery quality (bandwidth, latency, uptime) rather than storage quotas. This remains an open design area.
-
+
+ 24. Identity Architecture
+ ItsGoin separates network identity (per-device routing/connection key) from posting identity (the face/persona authoring content). This is the architectural foundation for multi-device, multi-persona, and DM-level traffic-graph privacy — and it fully survives into v0.8: the uniques index treats posting IDs (including throwaway ones) as first-class searchable entries, and the registry + greeting-comment first-contact design (Discovery & First Contact) builds directly on the split.
+
+ Two layers of identity Implemented
+ Each device has ONE network key — used for QUIC connections, endpoint ID, mesh routing. It’s never linked on the wire to any posting key.
+ Each user can hold MANY posting keys simultaneously — no “active” persona, no switching. Each posting key is a persona (Public, Private, Work, Family, etc). Posts are signed with the posting key chosen at compose time. The two key layers are always distinct: fresh installs generate an independent default posting identity, and the v0.6.1 migration rotates the network key of any install whose default posting key still equaled it. No node’s network key ever doubles as a posting key.
+ Privacy invariant: peers cannot determine which network IDs belong to a given posting ID, which posting IDs belong to the same network ID, or which posting IDs belong to the same user. These associations are private to the device owner.
+
+
Known violation — manifest addresses Rework: today’s AuthorManifest ships the device’s real addresses under a posting-identity author, linking persona to location on the wire. The v0.8 target removes device addresses from manifests entirely (holders are found via the CDN holder search, not via addresses embedded in author-signed data). This fix must land before or with the registry — location-anonymous discovery depends on it. Details in
Files & Storage.
+
Persona types Implemented
+
+ - Public posting IDs — main persona(s), openly associated with “you”
+ - Private posting IDs — smaller-context personas for close contacts or specific groups
+ - Contextual posting IDs — per-relationship or per-group; user creates one per context and sticks with it
+ - Throwaway posting IDs — single-purpose IDs (e.g., the outer signer of a greeting comment). Deliberately kept in uniques lists as network noise; retired naturally by comment TTL expiry (see Deletes & Expiry)
+
+
+ Required fixes: network-ID/posting-ID confusion Rework
+ The key split left four code paths comparing the device’s network NodeId where posting IDs now live. Since the two are always distinct, each path silently fails. All four are required v0.8 fixes (first item on the roadmap):
+
+ - Visibility revocation: the author check compares
post.author (posting ID) against the network ID, so every revoke bails with “not the author”; the rewrap path also pairs the default posting secret with the network ID.
+ - Group key granting: the GroupKeyRequest handler checks the requester’s network ID against a posting-ID member list and wraps with the network key — it can never grant. (No send side exists; the message pair is retired in the v0.8 protocol.)
+ - Replication cycle: under-replicated own posts are queried by network ID, so the set is always empty and the cycle is inert.
+ - Blob-eviction relationship tier: eviction scoring compares
candidate.author (a posting ID) against the network ID, so the 5.0 own-content tier never matches. Masked in practice because own blobs are auto-pinned (pin boost 1000), but the tier is inert.
+
+ The fix class is uniform: every self.node_id comparison or query against post/persona authors must use posting identities (all of them, not just the default). A full sweep for further instances is part of the fix.
+
+ Multi-device is a special case Implemented
+ “Two devices holding the same posting key” is a trivial case of the multi-key model. Link happens out-of-band between the user’s own devices via the export/import bundle (Identity Management) — the export includes all posting identities so they restore on the second device, which keeps its own network key. The network sees no cross-device message announcing the relationship; each device pulls content for its posting IDs via the normal CDN. The fact that two network nodes hold the same posting key is only discoverable if an observer has private knowledge (which they shouldn’t).
+ In-app QR / file-share flows for easy linking between an existing device and a new one are not wired yet — today it’s manual export-then-import.
+
+ Ephemeral rotating IDs for DM threads Planned
+ The intent: DM threads and group messages use per-thread unique posting IDs that rotate per message. Each encrypted message would include a handshake field — the next posting ID to use. Observer sees a stream of distinct posting IDs with no cryptographic tie between them, defeating thread-level traffic correlation. Desync recovery via a sliding window of the last N accepted IDs. Message history kept in a local encrypted-to-self archive that replicates across linked devices via self-follow.
+ v0.8 removes most of the old blockers. The uniques index deliberately carries throwaway/anonymous posting IDs (they double as network noise), so an ephemeral author is findable through the same distributed search as any other ID; comment TTL expiry retires spent IDs without leaving a permanent trail; and the greeting-comment machinery provides the cold-contact channel. What still needs a crisp design is freshness: how a just-generated posting ID becomes reachable fast enough to receive a reply before the next rotation.
+
+ CDN: per-file holder sets Implemented
+ Each file (post, blob, manifest) has its own flat holder set — up to 5 most-recent peers, LRU-capped. Once a file already has 5 holders, further requesters are redirected to existing holders instead of being registered. Eviction scoring is relationship-tiered (own content > followed > other). There is no tree rooted at the author and no upstream/downstream structure anywhere.
+ When a new post is created, the creator’s updated AuthorManifests (whose neighborhoods now reference the new post) are pushed to each prior post’s holder set via ManifestPush; recipients see the reference and fetch the new post via PostFetch. Notifications thus route via network-ID peers who happen to hold related files — content always arrives via pull, never pushed directly. Full mechanics in Files & Storage and Sync Protocol.
+
+ DM privacy model Implemented
+ Three mechanisms eliminate the “A messaged B” traffic signal:
+
+ - CDN-only propagation. There is no direct push for encrypted posts. All encrypted posts propagate via the file-holder CDN, indistinguishable on the wire from any other encrypted content.
+ - Merged pull + recipient-match. Content queries are a uniform list of IDs; a serving peer returns posts matching
author ∈ query_list OR wrapped_key.recipient ∈ query_list. The client always includes every one of its own posting identities alongside follows, so there is no distinguishable “searching for DMs” traffic fingerprint. This matching machinery survives the v0.8 pull redefinition — it runs at the CDN content-fetch step (see Sync Protocol).
+ - Greeting comments for cold contact Planned. First contact from outside the graph arrives as an HPKE-sealed comment on a bio post, signed by a throwaway outer ID — see Discovery & First Contact.
+
+
+ What the user sees
+
+ - One merged incoming feed (all content to all personas), with filter-by-persona pills Implemented
+ - Per-post “(you) as <persona>” label on own posts authored by a non-default persona Implemented
+ - Persona picker at post-creation time, auto-hidden with a single persona Implemented
+ - Settings → Personas to create, delete, and set default Implemented
+ - Reply/comment default persona = whichever key decrypted the post Planned
+ - Contextual compose defaults (e.g., posting to a circle auto-picks that circle’s last-used persona) Planned
+ - DMs to different personas as distinct inbox threads Planned
+
+
+ Key/collision safety
+ Posting keys and network keys are ed25519 seeds (256 bits of entropy). Birthday paradox reaches 50% collision at ~2128 keys generated — not a concern even at aggressive rotation rates across a global userbase. The operational risk is weak RNG during key generation; we rely on the platform CSPRNG everywhere.
+
+ Status
+ The persona split is fully implemented: key-layer plumbing, multi-persona storage and UX, CDN-only encrypted propagation, and merged-pull recipient matching are all live. Ephemeral rotating DM IDs remain planned. The network-ID/posting-ID bug family above is the one required rework before v0.8 features (registry, concealment budget) can sit on this foundation.
+
+
- 25. HTTP Post Delivery
- Status: Complete
- Direct peer-to-browser HTTP serving is implemented. For share link delivery, this is now part of the tiered web serving strategy (redirect → TCP punch → QUIC proxy) described in Section 26.
+ 25. HTTP Post Delivery Implemented
+
Intent
- Every ItsGoin node that is publicly reachable can serve its cached public posts directly to browsers over HTTP — no extra infrastructure, no additional dependencies, no new binary. The same QUIC UDP port used for app traffic is accompanied by a TCP listener on the same port number. UDP goes to the QUIC stack as always. TCP goes to a minimal raw HTTP/1.1 handler baked into the binary.
- This makes every publicly-reachable node a browser-accessible content endpoint, enabling share links that deliver content peer-to-browser without routing any post bytes through itsgoin.net.
+ Every ItsGoin node that is publicly reachable can serve its cached public posts directly to browsers over HTTP — no extra infrastructure, no additional dependencies, no new binary. The same port number used for QUIC (UDP) carries a TCP listener for a minimal raw HTTP/1.1 handler baked into the binary. This makes every publicly-reachable node a browser-accessible content endpoint, enabling share links that deliver content peer-to-browser without routing post bytes through itsgoin.net.
Dual listener architecture
- <port>/UDP → QUIC (existing app protocol)
-<port>/TCP → HTTP/1.1 (new, read-only, single route)
- Both listeners bind on the same port. The OS routes UDP and TCP to separate sockets — no conflict, no protocol ambiguity.
+ <port>/UDP → QUIC (app protocol)
+<port>/TCP → HTTP/1.1 (read-only, two routes)
+ Both listeners bind the same port number. The OS routes UDP and TCP to separate sockets — no conflict, no protocol ambiguity. The handler is raw tokio::net::TcpListener, no HTTP crate, zero new dependencies.
- HTTP handler
- The handler is intentionally minimal — implemented with raw tokio::net::TcpListener, no HTTP crate, no new dependencies. Approximately 150–200 lines of Rust.
- Single valid route: GET /p/<postid_hex> HTTP/1.1
+ Routes
+
+ | Route | Serves |
|---|
+ GET /p/<postid_hex> | Post rendered as a minimal HTML page (static compiled-in footer, no template engine) |
+ GET /b/<blob_hex> | Raw attachment bytes (images/video referenced from the post page via /b/ URLs) |
+
- postid_hex must be exactly 64 lowercase hex characters (BLAKE3 hash)- Any other path, method, or malformed request: hard close with no response (not even a 400). Do not be helpful to malformed requests.
- - Post must be public (
PostVisibility::Public). Encrypted posts are never served over HTTP regardless of whether the node holds the content.
+ - Both hex arguments must be exactly 64 lowercase hex characters (BLAKE3 hash). Anything else — any other path, method, or malformed request — is a hard close with no response. Do not be helpful to malformed requests.
+ - Posts must be
PostVisibility::Public. Blobs are served only if they are an attachment of a public post. Encrypted content is never served over HTTP regardless of what the node holds.
+ - Connections are keep-alive: one connection serves sequential requests (the post page, then its attachments) inside its slot.
- Response: Minimal HTML page containing the post content with a small footer:
- <footer>
- This post is on the ItsGoin network — content lives on people's devices,
- not servers. <a href="https://itsgoin.com">Get ItsGoin</a>
-</footer>
- The footer HTML is a static string constant compiled into the binary (~2KB). No template engine, no dynamic footer generation.
+
+ Connection budget
+ The server runs a two-tier slot budget instead of a flat connection cap:
+
+ | Budget | Value | Purpose |
|---|
+ | Content slots | 5 | Full service: post pages + blob bytes |
+ | Redirect slots | 15 | 302-only service when content slots are full |
+ | Per-IP cap | 1 | One connection per client IP across both tiers |
+ | Header timeout | 5 s | Slow-request defense; exceeded → hard close |
+
+ A new connection tries to acquire a content slot first; if those are full it gets a redirect slot, where post requests are answered with a 302 to another holder (below) and blob requests are hard-closed. Over-budget connections are closed immediately — no queue, no wait.
Security constraints
| Concern | Mitigation |
|---|
- | Connection exhaustion | Hard cap: 20 concurrent HTTP connections. New connections over the cap are immediately closed. No queue, no wait. |
- | Slow HTTP attacks | 5-second read timeout for complete request headers. Exceeded → hard close. |
- | Content enumeration | Identical response (hard close) for “post not found” and “post not public.” No timing oracle, no distinguishable error codes. |
- | Malformed requests | Hard close only. No error response. |
- | Encrypted content | Never served. Public visibility check is mandatory before any response. |
+ | Connection exhaustion | 5+15 slot budget, 1 per IP, immediate close over budget |
+ | Slow HTTP attacks | 5-second read timeout for request headers |
+ | Content enumeration | Identical response (hard close) for “not found” and “not public.” No timing oracle, no distinguishable error codes. |
+ | Malformed requests | Hard close only. No error response, not even a 400. |
+ | Encrypted content | Never served. Public-visibility check is mandatory on both routes. |
Which nodes serve HTTP
- A node serves HTTP only if it is publicly TCP-reachable:
+ A node starts the HTTP listener only if it is publicly TCP-reachable, meaning any of:
- - IPv6 public address — serves directly
- - IPv4 + UPnP mapping — serves if TCP is included in the UPnP mapping (see Section 11 update)
- - IPv4 behind NAT without UPnP — cannot serve HTTP, but can still appear as a host in share links for app-protocol delivery. The CDN tree and itsgoin.net redirect handler route around unreachable nodes automatically.
+ - Active TCP port mapping via UPnP-IGD / NAT-PMP / PCP (see Port Mapping). Since v0.7.2 this includes mobile: phones on WiFi/Ethernet behind a permissive NAT serve HTTP too.
+ - Public IPv6 address — serves directly (desktop, anchor, or mobile).
+ - Explicit public bind (servers started with
--bind).
+ Nodes behind a NAT that refuses all three mapping protocols cannot serve HTTP but can still appear as holders for app-protocol delivery; the share-link tiers (see Share Links) route around them. A serving node advertises its HTTP capability and address to peers so they can be selected as redirect targets. This is the same direct-reachability signal that drives anchor candidacy (see Anchors & Connection Convection) — in practice the HTTP-serving population and the anchor-candidate population are the same nodes.
- 302 load shedding via CDN tree
- When a node is overwhelmed (at the 20-connection cap) or chooses to redirect:
+ 302 load shedding via file holders
+ A redirect-slot request is answered from the flat holder model:
- - Query
post_downstream table for the requested postid
- - Filter downstream hosts to those with a known public address (IPv6 or UPnP-mapped IPv4)
- 302 → http://[their_address]:<port>/p/<postid>- Verify the post exists locally and is public (else hard close).
+ - Query
file_holders for the post (flat set, cap 5 — see Content Propagation).
+ - Filter each holder's known addresses to publicly-routable ones.
+ - TCP-probe candidates with a 200 ms timeout;
302 → http://<addr>:<port>/p/<postid> to the first live one.
+ - All dead → hard close.
- The receiving node applies the same logic recursively if needed. This mirrors the app-layer CDN tree behavior at the HTTP layer — the same attention-driven propagation model, the same tree structure, now accessible to browsers.
-
- Binary size impact
- Zero new dependencies. Negligible compiled size delta (~10–20KB). No App Store size concerns. No install size impact for existing users.
+ The receiving node applies the same logic recursively if its own content slots are full. Load spreads across whichever holders are publicly reachable, mirroring the attention-driven holder model at the HTTP layer.
+
+ v0.8 change: redirect candidates come from the flat file_holders set; the old post_downstream tree (and the “CDN tree” framing of HTTP load shedding) is gone.
+
-
+
- 26. Share Links
+ 26. Share Links Implemented
+
Intent
Every public post can be shared as a URL that works for both app users and browser users:
- - App installed: OS intercepts the URL via Universal Links (iOS) / App Links (Android) before the browser loads. App opens directly to the post, fetched via QUIC. Zero browser involvement.
- - No app: Browser loads itsgoin.net, which searches the ItsGoin network for the post and redirects the browser to a live node serving it over HTTP. The share link becomes a product demo and install opportunity.
+ - App installed: OS intercepts the URL via Universal Links (iOS) / App Links (Android) before the browser loads. App opens directly to the post, fetched via QUIC. Zero browser involvement. (Planned — see below.)
+ - No app: Browser loads itsgoin.net, which finds the post on the network and delivers it through the tiered strategy below. The share link becomes a product demo and install opportunity.
- URL format
- https://itsgoin.net/p/<postid_hex>/<author_nodeid_hex>
+ URL format Implemented
+ https://itsgoin.net/p/<postid_hex>
- postid_hex: 64 hex characters (BLAKE3 post hash)author_nodeid_hex: 64 hex characters (author's ed25519 public key). Enables direct QUIC connection as fast path; worm search handles the case where author is unreachable.postid_hex: 64 hex characters (BLAKE3 post hash). That is the whole payload — no author ID, no host hints, no binary encoding. The anchor resolves holders itself.- Older
/p/<postid>/<author> URLs (pre-v0.7.2) still parse; the trailing author hex is used as an extra holder hint for the search rather than encoded into new links.
+ - Only public posts generate share links.
- Example: https://itsgoin.net/p/3a7f...c921/b4e2...f817
- Simple, human-inspectable, no binary encoding needed. Author ID in the URL is sufficient for the tiered serving strategy below.
+
+ v0.8 change: the former Phase 2/3 roadmap (holder-hint URLs, base64url hostlist encoding) is dropped. The bare postid is the final format — holder resolution is the anchor's job, and embedding device addresses in share links would undercut the location-anonymity goals of the v0.8 privacy model.
+
- Tiered web serving
- When a browser visits a share link, itsgoin.net attempts three tiers to deliver the post:
+ Tiered web serving Implemented
+ When a browser visits a share link, the itsgoin.net web handler attempts three tiers:
- - 302 redirect: If a publicly-reachable holder is known (IPv6, UPnP TCP), redirect the browser directly to that node's HTTP endpoint. Zero post bytes flow through itsgoin.net.
- - TCP punch: If a holder is connected but not publicly reachable, send
TcpPunchRequest (0xD6) asking the holder to punch TCP toward the browser's IP. On success, TcpPunchResult (0xD7) returns the HTTP address for a redirect.
- - QUIC proxy: If neither redirect nor punch works, itsgoin.net fetches the post on-demand via PostFetch (0xD4/0xD5), renders HTML, and serves directly to the browser.
+ - 302 redirect: holders advertising HTTP capability with a known public address (and still connected or in session) — redirect the browser straight to that node's HTTP endpoint. Zero post bytes flow through itsgoin.net.
+ - TCP punch: holders behind endpoint-independent (EIM) NAT that are connected but not publicly reachable — the anchor sends
TcpPunchRequest (0xD6) asking the holder to punch TCP toward the browser's IP; on success TcpPunchResult (0xD7) returns the HTTP address for a redirect.
+ - QUIC proxy: serve from local storage if the post is already held; otherwise a worm point-query (see Worm Search) finds a holder and
PostFetchRequest/Response (0xD4/0xD5) pulls the post (15 s budget), which is rendered as HTML and served directly. Posts fetched this way are cached, not permanently hosted — the proxy is a convenience service, not a data store.
-
- itsgoin.net QUIC proxy handler
- Route: GET /p/<postid_hex>/<author_nodeid_hex>
- 1. Check local storage (fast path — post already fetched recently)
-2. Connect to author via QUIC (connect_by_node_id cascade) → PostFetch
-3. Content search (extended worm with post_id) → find holder → PostFetch
-4. Found? Store temporarily + render HTML + serve to browser
-5. Not found? Serve "unavailable" page
- The itsgoin.net server acts as a QUIC proxy — it fetches posts on-demand from the peer network and renders them as HTML for the browser. Posts are not permanently stored on the server. This is a marketing/convenience service, not a data store. The proxy model works because most nodes are behind NAT and can't serve HTTP directly to browsers.
- Scalable via additional instances: 1.itsgoin.net, 2.itsgoin.net, etc. Each runs its own ItsGoin node with its own mesh connections, increasing total search coverage.
+ If all three tiers fail, the browser gets the unavailable page below.
itsgoin.net node
- itsgoin.net runs a permanent, well-connected ItsGoin node (--bind 0.0.0.0:4433 --daemon --web 8080). This serves two purposes:
-
- - Content search — when a share link is visited, the server-side node uses the extended worm search (with
post_id) to find any peer holding the post. The CDN tree means each peer knows about hundreds of posts across their downstream hosts, making hits highly likely even for content the server has never synced.
- - Anchor — a permanently-online, high-uptime node that bootstrap peers can rely on. Strengthens the network’s anchor infrastructure without any special protocol — it’s just a well-connected peer that happens to also serve the web handler.
-
-
- Share link format
- https://itsgoin.net/p/<postid_hex>/<author_nodeid_hex>
-
- - PostID: 64 hex chars (32 bytes, BLAKE3 content hash)
- - Author NodeID: 64 hex chars (32 bytes, ed25519 public key)
- - Author ID enables direct QUIC connection as fast path; worm search handles the case where author is unreachable
- - Only public posts can generate share links
-
+ itsgoin.net runs a permanent, well-connected ItsGoin node with the web handler enabled. It is an anchor in the v0.8 sense — a directly-reachable, opted-in node serving connection convection to joining peers (see Anchors & Connection Convection) — and its mesh position plus uniques knowledge is what makes share-link holder resolution likely to succeed even for content it has never synced. Scalable via additional instances (1.itsgoin.net, 2.itsgoin.net, …), each with its own mesh connections increasing total search coverage.
Unavailable page
⬡ This content isn't currently reachable.
@@ -1670,80 +1650,124 @@ END
[ Install ItsGoin to find it when it resurfaces ]
This is not a 404. It communicates the honest model: content lives on devices, not servers. Cold content decays. The install CTA is the honest answer to “how do I get this.”
- Universal Links / App Links
- Same URL — itsgoin.net/p/... — intercepts to the native app for users who have ItsGoin installed. No separate URL scheme, no app:// links.
- Required static files on itsgoin.net:
+ Universal Links / App Links Planned
+ Same URL — itsgoin.net/p/... — intercepts to the native app for users who have ItsGoin installed. No separate URL scheme, no app:// links. Requires two static JSON files on itsgoin.net:
/.well-known/apple-app-site-association (iOS Universal Links)
/.well-known/assetlinks.json (Android App Links)
- Both are static JSON deployed once, pointing to the ItsGoin app package ID for the path pattern /p/*.
- App-side handling: Register the URL pattern in Tauri config. On receipt of itsgoin.net/p/<postid>/<hostlist>, parse the postid, decode the hostlist, fetch via QUIC from the hostlist peers. If the post is already in local SQLite, render immediately. Universal Links intercept before the browser loads — itsgoin.net sees zero traffic for app users.
- iOS caveat: Universal Links require the app to have been opened manually at least once before OS interception activates. First-time tap from a link goes to the browser fallback. All subsequent taps open the app directly. The browser fallback is the full loading screen experience — first-time users see the product demo, which is the right outcome anyway.
+ App-side: register the URL pattern in Tauri config; on receipt, parse the postid and fetch via QUIC (render immediately if already in local SQLite). Universal Links intercept before the browser loads, so itsgoin.net sees zero traffic for app users. Until this ships, share links always traverse itsgoin.net's tiered serving even on app-installed devices.
+ iOS caveat: Universal Links require the app to have been opened manually once before interception activates. A first-time tap goes to the browser fallback — which is the full product-demo page, the right outcome for a first-time user anyway.
QR codes
- Share links are also valid QR codes. At ~127 chars base64url for 5 IPv6 hosts plus postid and domain, total URL length stays well under 200 characters — comfortably scannable at low error correction.
- QR codes for share links use the same Universal Links / App Links interception path. A generic phone camera scanning the QR sees an itsgoin.net URL and offers to open it — either in the app (if installed) or in the browser.
- No custom QR scheme needed. The HTTPS URL is the QR payload.
+ Share links are valid QR payloads as-is: the full URL is under 90 characters, comfortably scannable at low error correction. A phone camera sees an itsgoin.net URL and offers to open it in the app (once link interception ships) or the browser. No custom QR scheme.
Chrome HTTPS (October 2026)
- Chrome 154 (October 2026) enables “Always Use Secure Connections” by default, warning before HTTP sites. This does not affect the share link architecture:
-
- - itsgoin.net is HTTPS — no warning, Universal Links work normally
- - The 302 redirect takes the browser off the HTTPS page before content loads, so no mixed-content issue
- - Node HTTP endpoints are raw IP:port addresses, which Chrome treats as private network addresses (exempt from the public-site warning requirement)
- - The redirect itself is a header only — no content flows through itsgoin.net
-
- No architecture changes needed before or after October 2026.
+ Chrome 154 enables “Always Use Secure Connections” by default, warning before HTTP sites. This does not affect the architecture: itsgoin.net is HTTPS (Universal Links work normally); the 302 redirect is a header only, taking the browser off the HTTPS page before any content loads; and node HTTP endpoints are raw IP:port addresses, which Chrome exempts from the public-site warning requirement. No changes needed before or after the rollout.
-
+
+
+ 27. Discovery & First Contact Planned
+
+ The problem
+ The mesh-CDN architecture deliberately has no global feed and no keyword flood-search (see Worm Search for why flooding is rejected). That leaves two gaps: discovery — how do you find a person you don't already have a graph path to? — and first contact — how does a stranger send you an initial message when every messaging rail (DMs, FoF comments) requires an existing key relationship? This section is the designed answer: an opt-in registry for discovery, and greeting comments for first contact. Both compose with machinery that already exists (bio posts, CDN comment verification, HPKE-sealed wrappers, RevocationEntry, comment TTL).
+
+ The registry: opt-in discoverable bio posts
+ Registration is a flag on a public bio post: “this persona wants to be findable.” A discoverable bio post links a public persona (display name, bio text, avatar) to a public author ID (a posting identity) — and nothing else. There is no directory server; the registry is the set of discoverable-flagged bio posts, replicated as ordinary CDN content.
+
+ - Location-anonymous by construction: a registry entry maps name → author ID only. Resolving the author ID to actual content goes through normal CDN holder search — never through an address in the entry. This depends on the AuthorManifest privacy fix (manifests must not carry device addresses — see Files & Storage); the fix must land before or with the registry.
+ - Replication set: flagging a bio discoverable is explicit consent to wide replication. Discoverable bios form their own replication set that opted-in holders spread beyond the 2–5-post concealment budget that applies to ordinary content — registration trades concealment for reach, deliberately and per-persona.
+ - Search: a searching node fetches the registry post's comment chain (below) via an ordinary pull, then queries locally. No flood queries; search cost lands on the searcher, not the network.
+ - Consent gradient: registered personas are searchable by anyone. Unregistered personas are reachable only through the graph — someone must encounter their author ID via the uniques index (see Network Knowledge), a comment thread, or a vouch. Nothing about an unregistered persona is indexed.
+
+ The v0.6.2 Discover tab — a local list of named, public-visible profiles already held in storage — is the UI seed for this: the registry generalizes it from “profiles I happen to hold” to a replicated, searchable set.
+
+ The registry post: rendezvous & enumeration Planned — tester-critical
+ Discoverable bios alone leave a gap: how does a searcher enumerate them? The answer is a well-known “registrations here” post whose comment chain is the registry index. A registration is a signed public comment on that post — structurally the same open-slot comment as a greeting (see below), aimed at a well-known target instead of a stranger's bio. One implementation serves both features.
+
+ - The post: published at network genesis by the default anchor's persona; its ID is derived from a well-known seed the app ships with (
shard(seed, k) — shard count starts at 1, and the derivation scheme exists from day one so splitting by hash(persona_id) later is a threshold change, not a migration). It replicates as ordinary CDN content; anchors hold it by durability, not appointment.
+ - The entry: a registration comment carries {display name, search keywords, persona ID / public author ID} signed by the registering persona's posting key — self-certifying: any holder can verify the entry against the key it names, no authority consulted.
+ - Self-cleaning: registration comments carry a fixed 30-day TTL (unlike ordinary comments' randomized TTL — that randomness serves anonymity, which a deliberately-public registration doesn't want). Staying listed means re-signing a fresh entry; holders keep one active entry per persona ID (newest wins). The chain self-cleans and is never more than 30 days stale.
+ - Delete / modify: a delete request signed by the same persona key is honored by every holder (verifiable from the entry itself — the network agreement is checkable, not honor-system). Modify = delete + re-register; no separate primitive needed.
+ - Moderation hook: the registry post's author retains the standard per-post
RevocationEntry power — manual spam removal for the alpha era, at the acknowledged cost that the author could censor entries. Acceptable while the author is the project's own anchor; the trust layer (see Directory Trust Layer) is the eventual replacement.
+ - Anti-bot pricing: registrations carry the slot-declared PoW stamp (a few CPU-seconds), fit a single small size bucket, and are rate-capped per holder. This prices flooding without preventing it — a bot willing to mint personas and burn CPU can still pollute the chain.
+
+
+
Honest threat model: personas are deliberately near-free in this system (throwaway IDs are a feature), so no per-key or proof-of-work measure changes the spam economics against a
motivated adversary — and this network must expect motivated adversaries with an interest in making decentralized social feel broken. The alpha registry survives casual spam only. Two things make that acceptable: (1)
bounded blast radius — searches are local and the registry is an opt-in convenience layer, so a fully-flooded registry blocks only
new stranger discovery while every existing relationship, follow, vouch, and sync continues untouched; griefing spends real resources to inconvenience only people who haven't met anyone yet. (2)
Vouch-gated registration (see
Directory Trust Layer) is the durable fix once utilization supports vouch introductions — the registry post's
format survives that transition, only the acceptance rule tightens. Sharding is deliberately deferred to the same moment: by the time volume demands shards, registration demands vouches.
+
Greeting comments: sealed first contact
+ First contact rides the comment rails. A bio post that opts into contact includes a greeting slot in its pub_post_set (see Friend-of-Friend Visibility) whose signing keypair is published — derivable by anyone from material in the bio post header. A stranger's greeting comment signed with the greeting key passes the CDN-level comment-verification gate that otherwise drops non-member comments, so it propagates like any FoF comment.
+
+ - Sealed content: the greeting body is HPKE-sealed to the bio author's posting key. Inside: the sender's real persona ID and a return path. Outside: an opaque ciphertext authored by a throwaway outer ID. Observers learn that the bio received a greeting, not from whom.
+ - Blends in: FoF Mode 2 already makes encrypted comment blobs commonplace on public posts. Greetings are one more ciphertext among many — if everyone's anonymous, anonymity doesn't stand out.
+ - Off-switch: the author revokes the greeting
pub_x via the existing per-post RevocationEntry sweep — propagation nodes delete stored greetings and stop accepting new ones. No new revocation machinery.
+ - Retirement: comments expire at a random TTL fixed at creation (see Delete Propagation & Comment Expiry), so throwaway greeting IDs eventually vanish from the network — and from everyone's uniques lists — entirely.
+
+
+ Spam controls
+
+ | Control | Mechanism |
|---|
+ | Opt-in | No greeting slot in the bio → no greetings possible. Default is off. |
+ | Size bucket | Greetings fit a single small padded bucket — no attachments, no long-form spam payloads. |
+ | Holder-side count caps | Nodes holding a bio cap how many unexpired greetings they store and forward per bio. |
+ | PoW stamp (optional) | The greeting slot may declare a proof-of-work difficulty a valid greeting must carry — the BitMessage lesson: make bulk greeting-spam computationally expensive. |
+
+
+ Known limit: the greeting slot's pub_x_index is observable, so greetings are identifiable as a class (never by sender). Mitigation: friends can route innocuous chatter through the open greeting key as padding, so greeting traffic on a bio is not automatically stranger traffic.
+
+
+ The composed flow
+
+
Find a persona via registry search (or encounter their author ID through the graph) → greet with a sealed greeting comment on their bio post, real identity and return path inside → the author unseals it, decides, and vouches back via the normal bio-post HPKE wrapper batch → the relationship moves onto standard FoF rails (readable posts, gated comments, DMs). The throwaway greeting ID expires with its comment; nothing permanent links the approach to the relationship that followed.
+
+
+
- 27. Directory Service (Planned)
+ 28. Directory Trust Layer (Future) Planned
-
The directory is an opt-in convenience layer for discovery and creator protection. It is not node access — losing directory presence does not disconnect anyone from the network or from their existing connections. This asymmetry is load-bearing: humans with mature relationships shrug off directory loss; bots and content thieves depend on it entirely.
+
The trust layer is an opt-in convenience layer for curated discovery and creator protection. It is not node access — losing trust-layer standing does not disconnect anyone from the network or from their existing connections. This asymmetry is load-bearing: humans with mature relationships shrug off directory loss; bots and content thieves depend on it entirely.
-
Distinct from FoF cryptographic vouches. The "vouch" described in this section is a
directory-layer trust signal governing discovery and bot-ring resistance. It is unrelated to the
cryptographic vouch (
V_me) in
section 20a, which gates post readership and commenting via per-persona symmetric keys. The two share vocabulary but operate at different layers.
+
Distinct from FoF cryptographic vouches. The "vouch" described in this section is a
directory-layer trust signal governing discovery and bot-ring resistance. It is unrelated to the
cryptographic vouch (
V_me) in
Friend-of-Friend Visibility, which gates post readership and commenting via per-persona symmetric keys. The two share vocabulary but operate at different layers.
+
+
v0.8 change: the directory no longer defines its own entry/listing mechanics. The
registry is the entry point — anyone can register a discoverable bio. The trust layer, when built, curates
standing over registry entries: vouch-weighted visibility, moderation, and creator protection on top of the flat registry. Everything below is a future layer beyond the v0.8 scope.
Scope
- - Whitelist track — discoverability, vouch-based entry, graph-scoped visibility.
+ - Whitelist track — vouch-weighted discoverability and graph-scoped visibility over registry entries.
- Blacklist track — community-flagged accounts and content; voluntary node-level replication refusal.
- - Out of scope — node access, message delivery, post sync, existing follows. All continue to work without directory membership.
-
- Because directory loss is a low-cost outcome for real humans and a high-cost outcome for bad actors, enforcement thresholds can be deliberately aggressive without meaningful false-positive risk.
-
- Entry
- Two paths to directory listing, both yielding equal discoverability but different trust-building capacity:
-
- - Vouch entry — 1 vouch from a directory member with remaining vouch capacity.
- - Paid entry — fee in lieu of vouch. Grants directory presence only; vouch capacity starts at zero and grows only from received human vouches. Prevents ring-bootstrapping: a bad actor cannot pay their way to vouch power.
+ - Out of scope — node access, message delivery, post sync, existing follows, and bare registry presence. All continue to work without trust-layer standing.
+ Because trust-layer loss is a low-cost outcome for real humans and a high-cost outcome for bad actors, enforcement thresholds can be deliberately aggressive without meaningful false-positive risk.
Vouch capacity
A member's outbound vouch capacity is derived from received vouches:
- Base rate: 1 outbound vouch per 30 days per received vouch, up to a hard cap of 5 outbound vouches per 30 days.
- - A member with zero received vouches (paid-entry only) has zero outbound capacity.
+ - A registered persona with zero received vouches has zero outbound capacity — presence in the registry grants findability, never vouch power.
- Capacity regenerates; unused capacity does not accumulate beyond the monthly window.
Trust signals are internal, not purchasable. Aged off-platform accounts (FB, X, etc.) can be bought cheaply; in-system tenure crossed with graph density cannot. Vouch weight derives from the voucher's in-system graph depth and their history of non-revoked vouches — not account age.
Cascade punishment
- A "bad vouch" is a vouch later determined to have been extended to a bot, impersonator, content thief, or other directory-removable actor. Punishment is asymmetric — mild for humans who miscalled a single vouch, devastating for botnets whose strength is their dense internal vouch graph.
+ A "bad vouch" is a vouch later determined to have been extended to a bot, impersonator, content thief, or other removable actor. Punishment is asymmetric — mild for humans who miscalled a single vouch, devastating for botnets whose strength is their dense internal vouch graph.
| Offense | Immediate consequence | Recovery requirement |
|---|
| 1st bad vouch | All given and received vouches invalidated (member remains listed, but cannot vouch) | 2 NEW vouches received before outbound vouching resumes |
- | 2nd bad vouch | Removed from directory | 1 new vouch to relist + 2 additional new vouches before vouching again |
- | 3rd bad vouch | Removed from directory; 1-year outbound vouch freeze | 4 new vouches to relist; no outbound vouching for 12 months regardless of received vouches |
+ | 2nd bad vouch | Removed from trust-layer visibility | 1 new vouch to relist + 2 additional new vouches before vouching again |
+ | 3rd bad vouch | Removed from trust-layer visibility; 1-year outbound vouch freeze | 4 new vouches to relist; no outbound vouching for 12 months regardless of received vouches |
"NEW" is strictly defined: a voucher who has never previously vouched this member and is not within the first-degree graph of any prior voucher for this member. This blocks ring members from cycling each other through as "recovery" vouchers.
Cascade radius. Invalidation of received vouches propagates the effective penalty upward: the bad actor's vouchers lose the credibility granted by that downstream relationship. One verified human assertion against a single botnet node can cascade through the entire dense cluster because bots' own topology carries the invalidation. They build the weapon that points back at them.
Graph-relative visibility
- The directory is not global. A viewer sees directory entries N hops from their own social graph (N tunable; start with N=3). Bots at the fringe of the graph are structurally invisible to most humans. Visibility is further rate-limited (Y new profiles per viewer per day, Y tunable) to make harvesting economically unattractive without affecting normal discovery.
+ Trust-layer discovery is not global. A viewer sees vouched entries N hops from their own social graph (N tunable; start with N=3). Bots at the fringe of the graph are structurally invisible to most humans. Visibility is further rate-limited (Y new profiles per viewer per day, Y tunable) to make harvesting economically unattractive without affecting normal discovery.
This scoping is also why the vouch system's bot-ring problem is bounded: even a successful ring has no harvest value if it cannot be seen by real users.
Verification (circuit breaker)
@@ -1759,14 +1783,14 @@ END
The reporting pipeline is lightweight, member-facing, and feeds a single review queue with multiple severity tracks:
- Impersonation — a member reports an account copying their (or someone they know's) identity. High-confidence signal: verifiable via profile comparison and content signatures.
- - Bad vouch — a member flags a directory entry they vouched for, or vouches in their extended graph. Triggers cascade punishment on confirmation.
+ - Bad vouch — a member flags an entry they vouched for, or vouches in their extended graph. Triggers cascade punishment on confirmation.
- Content theft — a creator reports unauthorized reposting. Evidence is the signed original versus the repost.
- Automated topology detection — graph analysis flags ring structures, abnormal clustering coefficients, and low-bridging clusters. Feeds the same queue as human reports.
Reports from verified accounts carry higher weight. A confirmed report against one member in a tight cluster opens review on the whole cluster.
Blacklist
- Blacklist is a higher-severity tier than directory suspension. Two states:
+ Blacklist is a higher-severity tier than trust-layer suspension. Two states:
- Under review — enough independent reports to warrant scrutiny; member still discoverable but flagged.
- Confirmed blacklisted — entry persists at the identity level. A bad actor creating a new identity starts from zero; the old identity's blacklist entry remains as a reputational cost that outlasts the account.
@@ -1774,10 +1798,10 @@ END
Escalation paths:
- Confirmed impersonation → direct blacklist.
- - Vouch violations → directory suspension first, blacklist only on repeated/escalated offenses.
+ - Vouch violations → trust-layer suspension first, blacklist only on repeated/escalated offenses.
- Content theft → voluntary replication refusal (below); blacklist for repeat offenders.
- The blacklist must be slower and more evidence-bound than directory suspension. Suspension costs a bot everything but costs a human almost nothing; blacklist has real network consequences (below) and must not be a censorship weapon.
+ The blacklist must be slower and more evidence-bound than suspension. Suspension costs a bot everything but costs a human almost nothing; blacklist has real network consequences (below) and must not be a censorship weapon.
Voluntary network compliance
Nodes may configure policies to decline replication or delivery of blacklisted content and accounts. This is opt-in, not forced. The effect is architectural starvation: stolen or abusive content cannot sustain replication when hosts collectively decline to be its infrastructure, while the legitimately signed original continues propagating normally.
@@ -1787,7 +1811,7 @@ END
The platform does not insert or intermediate ads. Creators may embed ads directly in their content or feed, as part of the signed post.
- Ads inside signed content cannot be stripped without breaking the ed25519 signature — tampering is detectable.
- - A repost that strips ads produces an unsigned-or-differently-signed copy, which compliance rules (below) treat as non-compliant.
+ - A repost that strips ads produces an unsigned-or-differently-signed copy, which compliance rules treat as non-compliant.
- A repost that preserves ads intact monetizes the original creator automatically.
- Ad revenue therefore follows the legitimate copy by construction, not by enforcement.
@@ -1805,16 +1829,16 @@ END
Most platforms have a structural conflict of interest with creators: fakes inflate engagement metrics, thieves generate content volume, both drive ad revenue. ItsGoin's incentives are inverted by design. Fakes degrade the vouch system; thieves attack the network's most valuable users; bots pollute the replication layer the network depends on. Every bad actor makes ItsGoin worse as software, not just ethically. The enforcement mechanisms above are therefore load-bearing, not policy theater.
Implementation status
- Designed, not implemented. Requires:
+ Designed, not implemented — and deliberately beyond the v0.8 scope. Requires:
- - Directory storage schema (vouch records, blacklist records, report queue, graph-density caches)
+ - Trust-layer storage schema (vouch records, blacklist records, report queue, graph-density caches)
- New protocol messages (vouch, revoke, report, verify challenge/response)
- Graph-topology analysis job (automated ring/cluster detection)
- - Client UX for discovery, vouching, reporting, verification
+ - Client UX for vouching, reporting, verification
- Node-side replication policy config (blacklist honoring)
- Review-queue tooling for contested cases
- Recommended staging: minimum viable slice = invite-tree directory + single-vouch entry + impersonation reports → manual review queue. Defer cascade math, automated topology detection, verification override, and repost compliance until real graph data exists to calibrate thresholds.
+ Recommended staging: minimum viable slice = registry (see Discovery & First Contact) + single-vouch trust entries + impersonation reports → manual review queue. Defer cascade math, automated topology detection, verification override, and repost compliance until real graph data exists to calibrate thresholds.
Tunable parameters
@@ -1822,7 +1846,7 @@ END
| DIRECTORY_VOUCH_INTERVAL | 30 days | Time between outbound vouches per received vouch |
| DIRECTORY_VOUCH_CAP | 5 | Hard cap on outbound vouches per 30-day window |
| DIRECTORY_GRAPH_HOPS | 3 | Visibility radius for discovery |
- | DIRECTORY_DISCOVERY_RATE | TBD / day | New profiles visible per viewer per day |
+ | DIRECTORY_DISCOVERY_RATE | ~10 / day (provisional) | New profiles visible per viewer per day; tune against real graph data |
| RECOVERY_VOUCHES_1ST | 2 | NEW vouches to restore voucher status after 1st bad vouch |
| RECOVERY_VOUCHES_2ND | 1 + 2 | Relist + additional vouches after 2nd bad vouch |
| RECOVERY_VOUCHES_3RD | 4 | Relist vouches after 3rd bad vouch |
@@ -1830,105 +1854,88 @@ END
| REPOST_DISCUSSION_RATIO | 1:4 | Max embed duration relative to original (discussion track) |
-
-
-
- 28. Identity Architecture Mostly Shipped (v0.6.0)
- v0.6.0 separates network identity (per-device routing/connection key) from posting identity (the face/persona authoring content). This is the architectural foundation for multi-device, multi-persona, and DM-level traffic-graph privacy. Five of the original six phases shipped as v0.6.0; ephemeral rotating DM IDs (§28.4) are deferred pending connection-model design work.
-
- 28.1 Two layers of identity Shipped
- Each device has ONE network key — used for QUIC connections, endpoint ID, mesh routing. It's never linked on the wire to any posting key.
- Each user can hold MANY posting keys simultaneously — no "active" persona, no switching. Each posting key is a persona (Public, Private, Work, Family, etc). Posts are signed with the posting key chosen at compose time. On first launch after upgrading, the existing identity becomes the default posting key (same bytes as the network key for upgraders, so all prior signed content keeps verifying).
- Privacy invariant: peers cannot determine which network IDs belong to a given posting ID, which posting IDs belong to the same network ID, or which posting IDs belong to the same user. These associations are private to the device owner.
-
- 28.2 Persona types Shipped
-
- - Public posting IDs — main persona(s), openly associated with "you"
- - Private posting IDs — smaller-context personas for close contacts or specific groups
- - Contextual posting IDs — per-relationship or per-group; user creates one per context and sticks with it
-
-
- 28.3 Multi-device is a special case Partial
- "Two devices holding the same posting key" is a trivial case of the multi-key model. Link happens out-of-band between the user's own devices via the existing identity export/import bundle — the export includes all posting identities so they restore on the second device. The network sees no cross-device message announcing the relationship; each device pulls content for its posting IDs via the normal CDN. The fact that two network nodes hold the same posting key is only discoverable if an observer has private knowledge (which they shouldn't).
- In-app QR / file-share flows for easy linking between an existing device and a new one are not wired yet — today it's manual export-then-import.
-
- 28.4 Ephemeral rotating IDs for DM threads Deferred
- The intent: DM threads and group messages use per-thread unique posting IDs that rotate per message. Each encrypted message would include a handshake field — the next posting ID to use. Observer sees a stream of distinct posting IDs with no cryptographic tie between them, defeating thread-level traffic correlation. Desync recovery via a sliding window of the last N accepted IDs. Message history kept in a local encrypted-to-self archive that replicates across linked devices via self-follow.
- Why deferred: the current connection model is per-network-ID (peers establish QUIC sessions keyed by stable NodeId; holder-set learning assumes durable addressable authors). Ephemeral authors that come into existence per-thread need a crisper design for how a freshly generated posting ID becomes reachable fast enough to receive replies before the next rotation. The pieces needed to implement rotation (per-file holder sets, merged pull with recipient-match, per-posting-ID signing) are all in place — what's missing is the cold-contact discovery semantics for ephemeral recipients.
-
- 28.5 CDN: per-file holder sets Shipped
- The earlier upstream/downstream tree (which assumed a single author network endpoint) is replaced by a flat per-file holder set with header-diff propagation.
- Each file (post, blob, manifest) has its own holder set (up to 5 most-recent peers, LRU-capped). When a new post is created, the creator updates the headers of recent prior posts (their manifests now reference the new post), then pushes the header diff to each updated prior post's holder set. Recipients apply the header diff, see the reference to the new post, and pull it through normal sync.
- Notifications thus route via network-ID peers who happen to hold related files — not via any tree rooted at the author. Content always arrives via pull, never pushed directly. Four legacy tables (post_upstream, post_downstream, blob_upstream, blob_downstream) were dropped in v0.6; a one-way migration seeds file_holders from them on first launch.
-
- 28.6 DM privacy model Shipped
- Two of the three originally-planned mechanisms shipped in v0.6.0 and eliminate the "A messaged B" traffic signal for follower-to-follower and cold-contact-via-pull scenarios:
-
- - CDN-only propagation. Direct
PostPush for encrypted posts is removed. All encrypted posts propagate via the file-holder CDN, indistinguishable on the wire from any other encrypted content.
- - Merged pull + recipient-match. Pull sync's query is a uniform list of NodeIds; server returns posts matching
author ∈ query_list OR wrapped_key.recipient ∈ query_list. Client always includes its own NodeId alongside follows. No distinguishable "searching for DMs" traffic fingerprint — the query looks identical to a routine follow-pull.
- - Comment-as-introduction for cold contact. Any public post with open comments serves as a message-request surface. The machinery exists (engagement diffs flow through the post's holder network); a dedicated UX affordance for "start a conversation by commenting" is still on the todo list.
-
-
- 28.7 What the user sees
-
- - One merged incoming feed (all content to all personas), with filter-by-persona pills Shipped
- - Per-post "(you) as <persona>" label on own posts authored by a non-default persona Shipped
- - Persona picker at post-creation time, auto-hidden with a single persona Shipped
- - Settings → Personas to create, delete, and set default Shipped
- - Reply/comment default persona = whichever key decrypted the post Not yet
- - Contextual compose defaults (e.g., posting to a circle auto-picks that circle's last-used persona) Not yet
- - DMs to different personas as distinct inbox threads Not yet
-
-
- 28.8 Key/collision safety Shipped
- Posting keys and network keys are ed25519 seeds (256 bits of entropy). Birthday paradox reaches 50% collision at ~2128 keys generated — not a concern even at aggressive rotation rates across a global userbase. The operational risk is weak RNG during key generation; we rely on the platform CSPRNG everywhere.
-
- 28.9 Rollout status
-
- - Phase 1 Shipped v0.6.0 — Direct
PostPush for encrypted posts removed. Encrypted DMs propagate via ManifestPush like any other content.
- - Phase 2 Shipped v0.6.0 — File-holder model + header-diff propagation replaces upstream/downstream. Legacy tables dropped with a one-way seed migration.
- - Phase 3 Shipped v0.6.0 — Merged pull + recipient-match search. DM search indistinguishable from follow-pull.
post_recipients index added.
- - Phase 4 Shipped v0.6.0 — Posting-key / network-key split plumbing.
posting_identities table; seed migration copies network key in as default persona for upgraders.
- - Phase 5 Shipped v0.6.0 — Multi-persona UX (Settings page, compose picker, feed filter pills, per-post persona labels).
- - Phase 6 Deferred — Ephemeral rotating IDs for DM threads + local self-archive. See §28.4 for the open design question.
-
- v0.6 is a hard network fork from v0.5. The two versions do not interoperate. This was an explicit simplification once the user base was small enough to migrate everyone directly — no dual-write tables, no legacy message handlers, no mixed-network testing. Users cross via the existing export/import identity bundle; see the download page for the upgrade path.
-
-
-
+
- Appendix A: Timeout Reference
+ Appendix A: Timeout & Interval Reference
+ Every constant below exists in code today (cited per row in source comments) or comes from a v0.8 ruling (marked Planned). Constants tied to retired subsystems (preferred peers, register loop, keep-alive pool) are gone; the two registration-model referral constants remain, badged Rework, until the convection window replaces them.
| Constant | Value | Purpose |
|---|
- | MESH_KEEPALIVE_INTERVAL | 30s | Ping to prevent zombie detection |
- | ZOMBIE_TIMEOUT | 600s (10 min) | No activity → dead connection |
- | SESSION_IDLE_TIMEOUT | 300s (5 min) | Reap idle interactive sessions (NOT keep-alive) |
- | SELF_LAST_ENCOUNTER_THRESHOLD | 14400s (4 hours) | Trigger pull sync when last encounter exceeds this |
+
+ | MESH_KEEPALIVE_INTERVAL | 30s | Ping to prevent zombie detection + NAT timeout |
+
+ | ZOMBIE_TIMEOUT | 600s (10 min) | No stream activity → dead connection |
+
+ | SESSION_IDLE_TIMEOUT | 300s (5 min) | Reap idle sessions (referral-list peers and known anchors exempt) |
+
| QUIC_CONNECT_TIMEOUT | 15s | Direct connection establishment |
- | HOLE_PUNCH_TIMEOUT | 30s | Overall hole punch window |
- | HOLE_PUNCH_ATTEMPT | 2s | Per-address attempt within window |
- | RELAY_INTRO_TIMEOUT | 15s | Relay introduction request |
- | RELAY_PIPE_IDLE | 120s (2 min) | Relay pipe idle before close |
- | RELAY_COOLDOWN | 300s (5 min) | Per-target relay cooldown |
- | RELAY_INTRO_DEDUP | 30s | Dedup intro forwarding |
+
+ | SESSION_KEEPALIVE_PROBE | 3s | Per-peer keepalive send timeout; failure marks the connection dead |
+
+ | HOLE_PUNCH_TIMEOUT | 30s | Overall hole punch window (parallel over all known addresses) |
+ | HOLE_PUNCH_ATTEMPT | 2s | Per-attempt timeout before retry within the window |
+
+ | RELAY_INTRO_TIMEOUT | 15s | Relay introduction request round-trip |
+
+ | RELAY_PIPE_IDLE | 120s (2 min) | Session-relay pipe idle before close (opt-in serving only) |
+ | RELAY_MAX_BYTES | 50 MB | Max bytes relayed per pipe before close (opt-in serving only) |
+
+ | RELAY_COOLDOWN | 300s (5 min) | Per-target relay cooldown after a failed introduction |
+
+ | RELAY_INTRO_DEDUP | 30s (cap 500) | Dedup window for forwarded introductions (seen_intros) |
+
| WORM_TOTAL_TIMEOUT | 3s | Entire worm search |
| WORM_FAN_OUT_TIMEOUT | 500ms | Per-peer fan-out query |
- | WORM_BLOOM_TIMEOUT | 1.5s | Bloom round to wide referrals |
+ | WORM_BLOOM_TIMEOUT | 1.5s | Nova round: follow-up queries to burst-response referrals (historically "wide referrals") |
| WORM_DEDUP | 10s | In-flight worm dedup |
| WORM_COOLDOWN | 300s (5 min) | Miss cooldown before retry |
- | REFERRAL_DISCONNECT_GRACE | 120s (2 min) | Anchor keeps peer in referral list after disconnect |
- | N2/N3_STALE_PRUNE | Immediate on disconnect + 7 day fallback | Remove reach entries tagged to disconnected peers; age-based fallback for stragglers |
- | N2/N3_STARTUP_SWEEP | On boot | Remove all N2/N3 entries tagged to peers not in current mesh |
- | PREFERRED_UNREACHABLE_PRUNE | 7 days | Release preferred slot (must re-negotiate MeshPrefer on reconnect) |
- | RECONNECT_WATCHER_EXPIRY | 30 days | Low-priority reconnect awareness; daily check after 7 days |
- | GROWTH_LOOP_TIMER | 60s | Periodic growth loop check |
- | CONNECTIVITY_CHECK | 60s | Social/file <N4 access check for keep-alive sessions |
- | DM_RECENCY_WINDOW | 14400s (4 hours) | DM'd nodes included in connectivity check |
- | UPNP_DISCOVERY_TIMEOUT | 2s | Gateway discovery on startup (do not block) |
- | UPNP_LEASE_RENEWAL | 2700s (45 min) | Refresh port mapping before TTL expiry |
- | ANCHOR_PROBE_INTERVAL | 1800s (30 min) | Periodic re-probe while anchor-declared |
- | ANCHOR_PROBE_TIMEOUT | 15s | Cold connect attempt by witness |
- | ANCHOR_STALE_THRESHOLD | 7 days | Post-bootstrap cleanup probes known_anchors older than this |
+
+ | REFERRAL_DISCONNECT_GRACE | 120s (2 min) | Rework Current-code registration model: anchor keeps a disconnected caller referable during this grace; retired by the convection window (see Anchors) |
+ | REFERRAL_LIST_CAP | 50 | Rework Current-code registration model: soft cap on the anchor's registration list, driving per-entry use tiering; retired by the convection window (replacement window size still open) |
+
+ | KNOWLEDGE_STALE_PRUNE | Immediate on disconnect + 5h fallback | Remove reach entries tagged to disconnected reporters; age fallback for stragglers. Boot sweep clears all non-mesh entries. |
+
+ | RECONNECT_WATCHER_EXPIRY | 30 days | Reconnect-notification watchers pruned after expiry |
+
+ | GROWTH_LOOP | Signal-driven (no timer) | Runs on disconnects, knowledge additions, rebalance backstop; 3-failure backoff, 500ms pause between attempts |
+
+ | REBALANCE_CYCLE | 600s (10 min) | Dead/zombie sweep, knowledge prune, session reap, growth backstop |
+
+ | ROUTING_DIFF_CYCLE | 120s | Incremental knowledge diff broadcast to connected peers |
+ | FULL_STATE_BROADCAST | 4h | Full knowledge-state broadcast (diff-loss repair) |
+
+ | RECOVERY_DEBOUNCE | 2s | Recovery loop debounce after the mesh < 2 signal |
+
+ | STICKY_N1_TTL | 24h | Bootstrap peer advertised as sticky N1 (isolation recovery) |
+
+ | PULL_CYCLE_TICK | 60s | Pull cycle tick; first tick full pull. v0.8: replaced by the update-cadence scheduler. |
+ | SELF_LAST_ENCOUNTER | 4h | Per-author staleness threshold before a pull fires |
+
+ | ENGAGEMENT_CHECK_TIERS | 5 min / 1h / 4h / 24h | Freshness-keyed engagement re-check (active 72h / recent 14d / aging 30d / cold). Seed of the v0.8 update-cadence scheduler. |
+
+ | REPLICATION_CYCLE | 600s (120s initial delay) | Own-post replication check: posts <72h old with <2 known holders. Currently inert (author-lookup bug, Roadmap item 1). |
+
+ | PORTMAP_DISCOVERY_WAIT | 3s | Watch-channel wait for first router response across UPnP-IGD / NAT-PMP / PCP (never blocks startup; renewal is internal to portmapper) |
+
+ | ANCHOR_REACHABILITY_LOSS | 300s (5 min) | Clear is_anchor after sustained port-mapping loss; restored on recovery |
+
+ | ANCHOR_PROBE_INTERVAL | 30 min | Anchor self-verification re-probe spacing. Rework: today's gate also requires ≥50 connections + 2h uptime (dying peer-count model); v0.8 re-keys candidacy to the reachability watcher. |
+
+ | BOOTSTRAP_BATCH_SIZE | 3 | Concurrent anchor probes in flight |
+ | BOOTSTRAP_BATCH_STAGGER | 2s | Delay between probe batch dispatches |
+ | BOOTSTRAP_PER_ANCHOR_TIMEOUT | 10s | Per-anchor probe timeout during batched bootstrap |
+ | ANCHOR_STALE_THRESHOLD | 3 days | Failed probe to an anchor with last_seen older than this deletes the known_anchors row |
+
+ | NAT_FILTER_PROBE_TIMEOUT | 10s | Third-party NAT filtering probe at startup anchor contact |
+
+ | HTTP_HEADER_TIMEOUT | 5s | HTTP request header read timeout |
+ | HTTP_REDIRECT_PROBE | 200ms | TCP liveness probe of a holder before issuing a 302 |
+
+ | CONVECTION_ACTION | stochastic, per-disconnect | Planned On each mesh disconnect: random {nothing | anchor introduction | mesh-peer introduction}; weights adaptive = local anchor-density prior + refusal feedback |
+ | CONVECTION_CLASS | entry / top-up (1 bit) | Planned Entry (<2 connections) always served; top-up refused cheaply under load — refusal feeds the adaptive weights |
+
+ | COMMENT_TTL | rand(30–365 days) | Planned Comment expiry, fixed at creation and carried with the comment |
+
+ | UPDATE_CADENCE | minutes → days | Rework Descending-frequency check scale keyed on freshness × relationship tier, jittered (seed: ENGAGEMENT_CHECK_TIERS above; see Update Cadence & Keep-Alive) |
@@ -1937,203 +1944,256 @@ END
Appendix B: Design Constraints
| Constraint | Value | Notes |
|---|
- | Visibility metadata cap | 256 KB | Applies to WrappedKey lists in encrypted posts |
- | Max recipients (per-recipient wrapping) | ~500 | 256KB / ~500 bytes JSON per WrappedKey |
+
+ | Mesh slots | ~20 (Desktop) / 15 (Mobile) | Single pool, no tiers. Rework — current code runs 101/15 across a Preferred/Local/Wide split |
+
+ | Temp referral slots | +4 to +10 above the cap | Planned Introduction facilitation only; no uniques exchange; graduate or expire |
+
+ | Knowledge depth | 4 bounces (N4 terminates) | Planned Mobile may hold 3 bounces or Bloom-compress N4 |
+
+ | Session slots | 20 (Desktop) / 5 (Mobile) | Oldest-idle evicted at capacity |
+
+ | Relay pipes | 10 (Desktop) / 2 (Mobile) | Session relay is opt-in, OFF by default on every role including anchors |
+
+ | Max payload (wire) | 64 MB | Length-prefixed JSON framing |
+
| Max blob size | 10 MB | Per attachment |
| Max attachments per post | 4 | |
- | Public post encryption overhead | Zero | No WrappedKeys, no sharding, unlimited audience |
- | Max payload (wire) | 16 MB | Length-prefixed JSON framing |
- | Mesh slots | 101 (Desktop) / 15 (Mobile) | Preferred + non-preferred, no local/wide distinction |
- | Keep-alive session cap | 50% of session capacity | Ensures interactive sessions remain available |
- | Keep-alive ceiling (desktop) | ~300–500 | Binding constraint: routing diff broadcast overhead |
- | Keep-alive ceiling (mobile) | ~25–50 | Binding constraint: battery + OS background restrictions |
- mesh_blacklist table | { node_id } | Targeted mutual stranger relationships for testing/diversity |
- known_anchors table | { node_id, addresses, last_seen } | LIFO ordered, 7-day stale cleanup via probe |
+
+ | File holders per content | 5 (LRU) | Flat file_holders set; registration past 5 answers with a holder redirect |
+
+ | Publicly revealed posts per author | 2–5 | Planned Concealment budget; surplus pushed out via replication (see Social Graph Privacy) |
+
+ | Comment TTL | rand(30–365 days) | Planned Fixed at creation, carried with the comment; retires throwaway greeting IDs |
+
+ | Max recipients (per-recipient wrapping) | ~500 / 256 KB | Design target for WrappedKey lists; not enforced in code |
+ | Public post encryption overhead | Zero | No WrappedKeys, unlimited audience |
+
+ known_anchors table | { node_id, addresses, last_seen, success_count } | Ranked by success_count, capped at 5; failed probes to rows older than 3 days self-prune |
+
+ | Wire compatibility | None | v0.8 is a clean protocol break from ≤v0.7.x (zero-users ruling). ALPN bumps itsgoin/3 → itsgoin/4 so old nodes are refused at handshake (see Sync Protocol) |
-
+
Appendix C: Implementation Scorecard
+ Code status as of v0.7.3, measured against the v0.8 target. Implemented = code matches the target today. Rework = built, but differently; migration required. Planned = not built.
| Area | Status |
|---|
- | Mesh connection architecture (101 slots, preferred/non-preferred) | Complete |
- | N1/N2/N3 knowledge layers | Complete |
- | Growth loop (60s timer + reactive on N2/N3) | Partial (timer exists, reactive trigger needs update) |
- | Preferred peers + bilateral negotiation | Complete |
- | N+10 identification | Partial (preferred peers exist, N+10 not in all headers) |
- | Worm search (nodes + content search for posts/blobs) | Complete |
- | Relay introduction + hole punch | Complete |
- | Session relay (own-device default) | Partial (relay works, own-device restriction not implemented) |
- | Social routing cache | Complete |
- | Three-layer architecture (Mesh/Social/File) | Partial (layers exist conceptually, pull sync still uses mesh) |
- | Keep-alive sessions | Planned |
- | Self Last Encounter sync trigger | Planned |
- | Algorithm-free reverse-chronological feed | Complete |
- | Envelope encryption (1-layer) | Complete |
- | Group keys for circles | Complete |
- | Three-tier access revocation | Partial (Tier 1+2 work, Tier 3 crypto exists but no UI) |
- | Private profiles per circle | Complete |
- | Pull-based sync with follow filtering | Complete |
- | Push notifications (post/profile/delete) | Complete |
- | Blob storage + transfer | Complete |
- | CDN hosting tree + manifests | Complete |
- | Blob eviction with priority scoring | Complete |
- | Anchor bootstrap + referrals | Complete |
- | Delete propagation + CDN cascade | Complete |
- | Multi-device identity | Planned |
- | UPnP port mapping (desktop) | Complete |
- | NAT type detection (STUN) + hard+hard skip | Complete |
- | Advanced NAT traversal (role-based scanning + filter probe) | Complete |
- | LAN discovery (mDNS scan + auto-connect) | Complete |
- | Content propagation via attention | Partial |
- | BlobHeader separation from blob content | Complete |
- | 25+25 neighborhood with HeaderDiff propagation | Partial (engagement diffs work, neighborhood diffs planned) |
- | BlobHeaderDiff message (engagement) | Complete |
- | Reactions (public + private encrypted) | Complete |
- | Comments + author policy enforcement | Complete |
- | Engagement sync via BlobHeaderRequest after pull sync | Complete |
- | Notification settings (messages/posts/nearby) | Complete |
- | Tiered DM polling (recency-based schedule) | Complete |
- | Auto-sync on follow | Complete |
- | Post CDN tree (post_downstream) | Complete |
- | Anchor self-verification (reachability probe) | Complete |
- | Mutual mesh blacklist | Planned |
- --max-mesh flag (test affordance) | Planned |
- | Audience sharding | Planned |
- | Custom feeds | Planned |
- | HTTP post delivery (TCP listener, single route, load shedding) | Complete |
- | Share link generation (postid + author NodeId) | Complete |
- | itsgoin.net QUIC proxy handler (on-demand fetch + render) | Complete |
- | PostFetch (0xD4/0xD5) single-post retrieval | Complete |
- | Universal Links / App Links (itsgoin.net/p/*) | Planned |
- | itsgoin.net ItsGoin node (anchor + web handler) | Complete |
- | UPnP TCP port mapping alongside UDP | Complete |
+
+ | Mesh & knowledge |
+ | Single ~20-slot mesh pool (tiers eliminated) | Rework — code runs Preferred/Local/Wide, 101 desktop / 15 mobile |
+ | Preferred-peer elimination (slots, MeshPrefer, prunes, watchers) | Rework — deletion pending (Roadmap item 2) |
+ | Temp referral slots (+4..+10, no knowledge exchange) | Planned |
+ | Uniques announce, payload N1–N3 (N4 stored but terminates, dedup at store + announce) | Rework — code exchanges N1–N3 share lists |
+ | Pull as uniques-index exchange (distributed search index) | Rework — code pulls posts via since_ms |
+ | Growth loop (signal-driven, diversity scoring, stochastic anchor path) | Rework — reactive core + scoring shipped; whole-pool counting and the per-disconnect stochastic action are target |
+ | Worm search (point queries: posts, blobs, nodes; no keyword flooding) | Implemented |
+ | Social routing cache + checkins + reconnect watchers | Implemented — preferred_tree field retires with preferred peers |
+ | Update-cadence scheduler (freshness × relationship, jitter, dedup) | Rework — freshness-only engagement tiers exist |
+
+ | Anchors & connectivity |
+ | Reachability-based anchor candidacy (portmapper watcher, bidirectional) | Implemented |
+ | Anchor opt-in gate | Planned — reachable desktops auto-anchor today |
+ | Connection convection (2-before/2-after rolling referrals, per-disconnect stochastic trigger) | Rework — register loop + tiered referral list exist; convection semantics are target |
+ | Batched bootstrap probes + 3-day stale-anchor self-prune | Implemented |
+ | Anchor self-verification probe (3rd-party witness) | Implemented — scheduling re-keys to the watcher in v0.8 |
+ Port mapping (UPnP-IGD + NAT-PMP + PCP via portmapper, all platforms) | Implemented |
+ | Relay introduction + hole punch (single quick + 30s parallel) | Implemented |
+ | NAT type detection (STUN) + filter probe | Implemented |
+ | EDM port scanner via raw UDP (bypassing iroh's path store) | Planned — disabled body preserved as edm_port_scan_disabled_v0_7_3 |
+ | Session relay (opt-in byte pipe, OFF by default, both sides gated) | Implemented |
+ | Connection rate limiting (per-IP failure backoff) | Implemented |
+ | LAN discovery — passive mDNS address lookup | Implemented |
+ | LAN discovery — active flow (subscribe, LAN sessions, sync priority) | Planned — low priority |
+
+ | Identity & personas |
+ | Network ↔ posting identity split + v0.6.1 key rotation | Implemented |
+ | Multi-persona per device (create/switch/import/export) | Implemented |
+ | Posting-key ZIP bundles: export, import-as-personas, post import & merge | Implemented |
+ | ID-class bug fixes (revocation, group-key grant, replication lookup, eviction own-content tier) | Rework — Roadmap item 1 |
+ | Ephemeral rotating DM identity | Planned |
+ | Throwaway-ID retirement via comment TTL | Planned |
+
+ | Encryption & visibility |
+ Envelope encryption (X25519 + ChaCha20-Poly1305, itsgoin/cek-wrap/v1) | Implemented |
+ | Group keys as encrypted posts + epoch rotation | Implemented |
+ | Tier-2/3 revocation (control-post propagation) | Rework — machinery built; entry-point author-check bug blocks every revoke |
+ | Private profiles per circle | Implemented |
+ | FoF visibility Layers 1–5 (vouch, gated comments, FoFClosed, rotation/burn, unlock cache) | Implemented |
+
+ | Sync & content |
+ | Wire protocol (length-prefixed JSON, 64 MB payload) | Rework — 46 types today; six (0x51/0x52/0xA1/0xA2/0xB3/0xC0) deleted in the clean-break purge (Roadmap item 2) |
+ | Merged pull with recipient matching (all posting identities) | Implemented — payload semantics change with the uniques redefinition |
+ | Manifest machinery (push, refresh, header diffs) | Implemented |
+ | AuthorManifest privacy (no device addresses, 2–5 neighborhood) | Rework — code ships addresses + 10+10 neighborhood |
+ Engagement propagation via flat file_holders + signature verification | Implemented |
+ | Deletes as signed control posts (+ LRU blob orphaning) | Implemented |
+ | Comment TTL expiry (rand 30–365d, fixed at creation) | Planned |
+ | Author concealment (2–5 visible posts) + traffic-uniformity shaping | Planned |
+ | Push notifications, reactions, comments, tombstones, auto-sync on follow | Implemented |
+ | Audience sharding (large-audience optimization) | Planned |
+
+ | Files & CDN |
+ | Blob storage (BLAKE3, 256-shard layout) + transfer with delivery budgets | Implemented |
+ | Flat holder model (cap 5, LRU, redirect at capacity) | Implemented |
+ | Blob eviction (pin / share-boost / relationship-tier scoring) | Implemented |
+ | Active replication cycle (10 min, 72h window, <2 holders) | Rework — cycle ships but is inert (author-lookup bug); becomes the concealment push-out path |
+ | Erasure-coded CDN replication (3-of-10 sub-threshold shards) | Planned |
+ | Pin modes (anchor pin / fork pin / personal vault) | Planned |
+
+ | Web, share links & discovery |
+ HTTP post delivery (/p/ + /b/, 5+15 slot budget, holder redirect, mobile serving) | Implemented |
+ Share link Phase 1 (itsgoin.net/p/<postid>, anchor resolves holders) | Implemented |
+ | itsgoin.net QUIC proxy + PostFetch + TCP punch | Implemented |
+ | Universal Links / App Links | Planned |
+ | Registry (registrations-here post + discoverable bios, location-anonymous) | Planned — tester-critical, roadmap #2 |
+ | Greeting comments (HPKE-sealed first contact via published slot key) | Planned |
+ | Directory trust layer (vouch graph, cascade punishment, blacklist) | Planned |
- Appendix D: Critical Path Forward
- The highest-impact items, in priority order:
+ Appendix D: Roadmap — v0.8 Critical Path
+ The path from today's code to the v0.8 target, in order. Items 1–2 clear the ground; 3–7 are the architecture shift; 8–10 build on it.
+
-
1. Three-layer separation (pull sync from social/file, not mesh)
-
Implement Self Last Encounter tracking and move pull sync to social + upstream file peers. This is the foundation for the layered architecture.
+
1. ID-class bug fixes
+
One bug family: the v0.6.1 persona split left code comparing the network NodeId where posting IDs now live. Four known instances: Tier-2/3 visibility revocation (author check bails on every revoke), the GroupKeyRequest handler (checks the wrong ID class, wraps with the wrong key — can never grant), the replication cycle (queries own posts by network ID, so the under-replicated list is always empty), and the blob-eviction relationship tier (own-content boost compares the network ID; masked by auto-pin). Fix all four, then sweep every self.node_id comparison against post/persona authors for further instances.
-
2. N+10 in all identification
-
Add N+10 (NodeId + 10 preferred peers) to self-identification, post headers, blob headers, and social routes. Dramatically improves findability.
+
2. Registry + greeting comments — tester-critical
+
The app is eminently useful only if people can find each other — this is what makes it usable for testers, so it jumps the queue. Ship the “registrations here” post (open-slot signed registration comments, fixed 30-day TTL, one entry per persona, self-certifying signed deletes, PoW stamp) and greeting comments (HPKE-sealed first contact via the published greeting-slot key) — one open-slot comment implementation serves both. Prerequisite folded in: remove device addresses from AuthorManifest first (registered author IDs must be location-anonymous). Known-bounded: survives until deliberate bot-flooding; the trust layer (item 10) is the durable fix.
-
3. Keep-alive sessions
-
Implement social/file connectivity check and keep-alive sessions for peers not reachable within N3. Cross-layer N2/N3 routing from keep-alive sessions.
+
3. Cruft purge
+
The zero-users ruling removes all wire-compat obligations. Delete: v0.6.x receive-only compat handlers (0x51/0x52) and the dual pull-matching path; the preferred-peer subsystem (slots, MeshPrefer negotiation, 7-day prune, 30-day watcher plumbing); dead senders and constants (hostlist encoder, no-op renewal cycle, orphaned GroupKeyRequest/Response); and legacy tree fields in wire structs. Clean protocol break — bump ALPN to itsgoin/4 here.
-
4. UPnP port mapping
-
Best-effort NAT traversal for desktop/home networks. Makes nodes directly reachable without hole punching. External address feeds into N+10 and all peer advertisements. Especially impactful for mobile-to-desktop connectivity.
+
4. N1–N4 uniques knowledge layer
+
Narrow the mesh to ~20 slots (single pool), add temp referral slots (+4..+10, no knowledge exchange), and replace N1–N3 share lists with the two-pool uniques announce: forwardable N0–N2 + terminal N3, received entries stored shifted one bounce deeper, N4 held but never re-announced. Anchor entries carry addresses (the pools double as the anchor directory); everything else is bare IDs. Redefine pulls as uniques-index exchange. Device-tiered depth and Bloom-compressed N4 for mobile as needed.
-
5. Growth loop reactive trigger
-
Fire growth loop immediately on N2/N3 receipt until 90% full. Currently only timer-based.
+
5. Anchor convection
+
Retire the register loop. Anchor candidacy = reachability watcher + opt-in (opt-in gate is low priority — IPv6-abundant anchors are unlikely to be overwhelmed); the anchor service becomes a rolling referral window (caller gets 2 recent prior callers, is referred to the next 2), with RelayIntroduce coordination when both ends are live, and entry/top-up request classes (entry always served; top-up refused cheaply under load). Clients act stochastically on each mesh disconnect — random {do nothing | anchor introduction | mesh-peer introduction} with adaptive weights (anchor-density prior + refusal feedback). Anchors are mined from the uniques pools; known_anchors recedes to a bootstrap cache.
-
6. Multi-device identity
-
Same identity key across devices with device-specific identity for self-discovery and own-device relay.
+
6. Post concealment (2–5 budget)
+
Cap the public post neighborhood to the 2–5 concealment budget (the AuthorManifest address removal already landed with item 2). Surplus posts are pushed out via replication so update/replication traffic hides content origin.
-
7. File-chain propagation
-
Make AuthorManifest with N+10 and recent posts work passively. Enable discovery of new content from any blob holder.
+
7. Update-cadence system
+
Replace the fixed pull tick with the descending-frequency scheduler: check intervals from minutes to days keyed on content freshness × relationship tier, same-author dedup, jittered timing so update and replication traffic reads as uniform network overhead. Uniques-list refresh rides these exchanges. Comment TTL (rand 30–365d) lands with this work.
-
8. Share links + HTTP post delivery
-
The viral growth mechanism. Every share becomes a product demo for non-app users and opens natively for app users. Dependencies in order:
-
- - UPnP TCP mapping (small addition to existing UPnP code)
- - Raw TCP HTTP listener (150–200 lines, zero new dependencies)
- - Host list generation at share time (query post_downstream, encode, embed in URL)
- - itsgoin.net redirect handler + known_good DB (server-side, independent of app releases)
- - itsgoin.net loading screen
- - Universal Links / App Links registration (static JSON files + Tauri config)
- - itsgoin.net ItsGoin node (run the binary, configure as anchor)
-
-
Steps 4–7 are itsgoin.net infrastructure, deployable independently of app releases. Steps 1–3 ship in the app. Step 6 requires an app store release to activate but can be deployed to itsgoin.net ahead of time.
+
8. Raw-UDP EDM refactor
+
Revive the disabled port scanner by replacing per-probe endpoint.connect() with raw socket.send_to() on the endpoint's bound UDP socket, so probes never enter iroh's path store (the v0.7.3 DoS-grade outbound cause). The preserved edm_port_scan_disabled_v0_7_3 body is the starting point.
-
9. Own-device relay restriction
-
Restrict relay pipes to own-device by default, opt-in for relaying for others.
+
9. Erasure-coded CDN replication
+
Replace full-copy auto-replication on the public tier with 3-of-10 sub-threshold shards, each shard a content-addressed blob with its own holder set. Removes the liability profile of infrastructure nodes holding reconstructable copies of content they never reviewed.
+
+
10. Directory trust layer
+
Vouch capacity, cascade punishment, graph-relative visibility, verification circuit-breaker, blacklist — built on top of the registry as the entry point. Minimum viable slice: single-vouch listing + impersonation reports → manual review. Defer cascade math until real graph data exists to calibrate thresholds.
-
-
- Appendix E: Features Designed But Not Built
-
- | Feature | Source | Status |
|---|
- | Three-layer pull sync (social/file, not mesh) | v0.2.0 design | Planned |
- | N+10 in all identification & headers | v0.2.0 design | Planned |
- | Keep-alive sessions | v0.2.0 design | Planned |
- | Multi-device identity | v0.2.0 design | Planned |
- | Own-device relay restriction | v0.2.0 design | Planned |
- | Self Last Encounter sync trigger | v0.2.0 design | Planned |
- | Anchor pin vs Fork pin distinction | project discussion.txt | Planned |
- | Audience sharding for groups > 250 | ARCHITECTURE.md | Planned |
- | Repost as first-class post type | project discussion.txt | Planned |
- | Custom feeds (keyword/media/family rules) | project discussion.txt | Planned |
- | Bounce routing (social graph as routing) | ARCHITECTURE.md | Planned |
- | Reactions (public + private encrypted) | v0.2.11 | Complete |
- | RefuseRedirect handling (retry suggested peer) | protocol.rs | Partial (send-only) |
- | Profile anchor list used for discovery | ARCHITECTURE.md | Partial (field exists) |
- | File-chain propagation (passive post discovery) | Design | Partial (manifest exists) |
- | Anchor-to-anchor gossip/registry | Observed gap | Planned |
- | BlobHeader as separate mutable structure | v0.2.11 | Complete |
- | BlobHeaderDiff incremental propagation (engagement) | v0.2.11 | Complete |
- | Post export/backup tooling (author durability) | v0.2.4 design | Planned |
- | Anchor reachability probe (self-verification) | v0.2.6 | Complete |
- | Mutual mesh blacklist | v0.2.4 design | Planned |
- --max-mesh flag (test topology control) | v0.2.4 design | Planned |
- | Relay-assisted port scanning (advanced NAT traversal) | v0.2.6 | Complete |
-
-
-
-
+
- Appendix F: File Map
+ Appendix E: File Map
+ As of v0.7.3 code. Use this as a navigation map, not an exhaustive line count.
crates/core/
src/
- lib.rs — module registration, parse_connect_string, parse_node_id_hex
- types.rs — Post, PostId, NodeId, PublicProfile, PostVisibility, WrappedKey,
- VisibilityIntent, Circle, PeerRecord, Attachment
- content.rs — compute_post_id (BLAKE3), verify_post_id
- crypto.rs — X25519 key conversion, DH, encrypt_post, decrypt_post, BLAKE3 KDF
- blob.rs — BlobStore, compute_blob_id, verify_blob
- storage.rs — SQLite: posts, peers, follows, profiles, circles, circle_members,
- mesh_peers, reachable_n2/n3, social_routes, blobs, group_keys,
- preferred_peers, known_anchors; auto-migration
- protocol.rs — MessageType enum (39 types), ALPN (itsgoin/3),
- length-prefixed JSON framing, read/write helpers
- connection.rs — ConnectionManager + ConnHandle/ConnectionActor (actor pattern):
- mesh QUIC connections (MeshConnection), session connections,
- slot management, initial exchange, N1/N2 diff broadcast,
- pull sync, relay introduction. All external access via ConnHandle.
- network.rs — iroh Endpoint, accept loop, connect_to_peer,
- connect_by_node_id (7-step cascade), mDNS discovery
- node.rs — Node struct (ties identity + storage + network), post CRUD,
- follow/unfollow, profile CRUD, circle CRUD, encrypted post creation,
- startup cycles, bootstrap, anchor register cycle
- web.rs — itsgoin.net web handler: QUIC proxy for share links,
- on-demand post fetch via content search, blob serving
- http.rs — HTML rendering for shared posts (render_post_html)
+ lib.rs — module registration, parse_connect_string, parse_node_id_hex,
+ DEFAULT_ANCHOR_POSTING_ID
+ types.rs — Post, PostId, NodeId, PublicProfile, PostVisibility (incl. FoFClosed),
+ WrappedKey, VisibilityIntent, Circle, DeviceProfile slot tables,
+ AuthorManifest/CdnManifest, BlobHeader + diff ops, DeviceRole budgets
+ content.rs — compute_post_id (BLAKE3), verify_post_id
+ crypto.rs — X25519 conversion, CEK wrap (itsgoin/cek-wrap/v1), group-key wrap,
+ encrypt/decrypt_post, FoF wrap slots + prefilter, slot keys
+ blob.rs — BlobStore (256-shard layout), compute_blob_id, verify_blob,
+ delivery budget tracking
+
+ identity.rs — IdentityManager: multi-identity dirs, create/list/switch/delete,
+ network vs posting identity (v0.6.0 split)
+ profile.rs — profile posts (signed, content-addressed profile updates)
+ control.rs — control posts: audience removal, delete control posts,
+ persona admin ops
+ fof.rs — Friend-of-Friend visibility: V_x wrap slots, anonymous prefilter,
+ pub_post_set, CDN-level comment verification,
+ V_me rotation / cascade-revoke / KeyBurnDiff
+ group_key_distribution.rs — circle group key distribution as encrypted posts + epoch rotation
+ activity.rs — in-memory ring buffer (200 events) of node activity
+ (growth/rebalance/recovery/anchor/connection/relay) for the UI
+ status panel — NOT engagement storage
+ announcement.rs — release announcement post handling
+
+ storage.rs — SQLite (auto-migration): posts, peers, follows, profiles,
+ circles/circle_members/circle_profiles, deleted_posts,
+ post_replicas, peer_neighbors, mesh_peers, reachable_n2/n3,
+ social_routes, reconnect_watchers, blobs, blob_headers,
+ cdn_manifests, file_holders (flat CDN, cap 5), post_hosts,
+ reactions/comments/comment_policies/thread_meta,
+ seen_engagement/seen_messages, group_keys/group_member_keys/
+ group_seeds, known_anchors, ignored_peers, relay_cooldowns,
+ worm_cooldowns, settings, post_recipients, posting_identities,
+ preferred_peers (retires in v0.8), + 10 vouch/FoF tables
+ (vouch_keys_own/received, vouch_bio_scan_cache, own_vouch_targets,
+ fof_revocations, own_post_slot_provenance, vouch_unlock_cache,
+ vouch_unreadable_posts, own_fof_post_ceks, fof_key_burns).
+ post/blob_upstream+downstream tables dropped in v0.6.1-beta.
+ protocol.rs — MessageType enum (46 types), ALPN (itsgoin/3; bumps to itsgoin/4 in v0.8),
+ length-prefixed JSON framing, InitialExchangePayload
+ connection.rs — ConnectionManager + ConnHandle/ConnectionActor (actor pattern):
+ mesh + session slots, initial exchange, N1/N2 diff broadcast,
+ pull sync, worm search, relay introduction, referral list,
+ anchor self-probe, engagement diff handling. EDM scanner
+ preserved as edm_port_scan_disabled_v0_7_3 (#[allow(dead_code)]).
+ session_relay_enabled gate (opt-in OFF by default).
+ network.rs — iroh Endpoint, accept loop, connect_to_peer, growth loop,
+ connect_by_node_id support, mDNS passive address lookup,
+ bidirectional anchor reachability watcher, per-IP rate limiting
+ node.rs — Node struct, post CRUD, follow/unfollow, profile/circle CRUD,
+ encrypted post creation, startup cycles, bootstrap with
+ probe_anchors_batched + stale-anchor prune, pull/diff/rebalance/
+ recovery/replication cycles, delete control posts,
+ connect_by_node_id resolution cascade, blob eviction scoring
+ web.rs — itsgoin.net web handler: QUIC proxy for share links, tiered
+ serving (redirect/TCP punch/proxy), on-demand post fetch
+ http.rs — raw-TCP HTTP listener: /p/ + /b/ routes, 5+15 slot budget,
+ holder 302 redirect with 200ms probe, render_post_html
+ upnp.rs — port mapping: UPnP-IGD + NAT-PMP + PCP via portmapper crate;
+ PortMapping holds Client for service lifetime; watch_external()
+ stun.rs — STUN probes (NAT type detection)
+ android_wifi.rs — Android-only JNI bridge: WifiManager.MulticastLock,
+ WiFi/Ethernet detection, NodeService.stopFromNative
+ export.rs / import.rs — posting-key ZIP bundles: export, import-as-personas,
+ import-as-new-identity, post import & merge (merge_with_key)
crates/cli/
- src/main.rs — interactive REPL + anchor mode (--bind, --daemon, --web)
+ src/main.rs — interactive REPL + anchor/daemon mode (--bind, --daemon, --web)
crates/tauri-app/
- src/lib.rs — Tauri v2 commands (38 IPC handlers), DTOs
+ src/lib.rs — Tauri v2 commands (DTOs + IPC handlers): persona CRUD, identity
+ import/export, exit_app (Android NodeService stop), session-relay toggle
+ gen/android/ — Android shell: NodeService.kt foreground service with
+ companion stopFromNative() static
frontend/
- index.html — single-page UI: 5 tabs (Feed / My Posts / People / Messages / Settings)
+ index.html — single-page UI: 5 tabs (Feed / My Posts / People / Messages / Settings),
+ close-app button with inline SVG power icon, session-relay toggle
app.js — Tauri invoke calls, rendering, identicon generator, circle CRUD
- style.css — dark theme, post cards, visibility badges, transitions
+ style.css — dark theme, post cards, visibility badges, transitions
+
+website/
+ design.html — this document (canonical design reference)
+ download.html — release notes
+ index.html, tech.html, contribute.html
+ style.css — shared site styling
8. Social Routing Implemented
-Status: Complete (with gaps)
+Caches addresses for follows and audience members, separate from mesh connections. Social contacts are consented relationships, so — unlike the uniques exchange — these payloads do carry addresses: that is the point of the layer (see Graph Privacy for the exact scoping). Every social route is also an N1 unique, announced source-merged like any other.
-When anchors are used
+
+social_routestablenode_idaddressespeer_addressesrelationstatuslast_connected_mslast_seen_msreach_methodpreferred_treecolumn and all N+10 semantics are retired with preferred peers (see Connections).peer_addressessurvives as plain known-peer referrals with no preferred-tier meaning. +Wire messages Implemented
+0x700x710x72Reconnect watchers Implemented
+
+ +reconnect_watcherstable: when peer A asks about disconnected peer B, A is registered as a watcher. When B reconnects, A gets aSocialAddressUpdatenotification and the watch is cleared. Watchers are pruned after 30 days.Social route lifecycle
-- Bootstrap: First startup with empty peers table. Connect to anchor, request referrals and matchmaking, persist on referral list while growing mesh.
- - Recovery: When mesh drops to 0 connections. Same flow as bootstrap.
- - Not ongoing: Nodes do NOT register with anchors on a loop. Anchors are for forming initial connections, not for ongoing presence.
- - itsgoin.net node: A permanent, well-connected ItsGoin node runs on itsgoin.net as part of the share link redirect infrastructure (see Section 26). This node participates in the network as a standard anchor — it bootstraps new nodes, accepts referral requests, and is included in
+ - Follow → store their addresses + referrals, upgrade to Mutual (if audience)
+ - Unfollow → downgrade/remove
+ - Approve audience → Mutual/Audience
-known_anchorsby peers that connect through it. It is not special-cased in the protocol. Its value as an anchor comes from permanent uptime and high mesh connectivity, not from any privileged role.Anchor referral mechanics
-When a bootstrapping node connects, the anchor provides referrals from its mesh and referral list. The node persists on the anchor's referral list until released at the referral count limit. During this time, the anchor can matchmake — introducing the new node to other peers requesting referrals.
- -Anchor selection order
--- Hardcoded default anchor(s) — only if
-
-known_anchorstable —ORDER BY last_seen DESC(LIFO). The most recently seen anchor is most likely still reachable, particularly given short-lived home desktop anchors.known_anchorsis empty or exhausted. A brand-new node hits hardcoded anchors once on first bootstrap, populatesknown_anchorsfrom that session, and the hardcoded list recedes to pure fallback.No scoring, no success counting, no prediction. Attempt, move to next on failure. The
- -known_anchorstable stores only:node_id,addresses,last_seen.Anchor self-verification Complete
-Nodes with UPnP-mapped IPv4 or IPv6 public addresses cannot self-certify as anchors — they need external verification that they are genuinely reachable by cold direct connect. A node is a viable anchor only if a complete stranger can connect to it directly with no introduction, no hole punch, and no relay.
- -Witness selection
-Node A (candidate anchor) selects a witness from its own N2 table entries NOT present in its N1. These are genuine strangers — no prior connection, no cached address, no warm path. A selects one (call it C) and knows C's address via the N1 reporter (call it B) who reported C in their N1 share.
- -Probe message flow
- -Asymmetric return path: If cold connect fails, by definition there is no direct path from C to A. C reports failure through B (who has a live connection to A). On success, C has a fresh direct connection and uses it. The
-return_viafield tells C which node to route failure through.Why bypass the cascade: The normal
- -connect_by_node_idcascade has 7 steps including hole punch and relay. If C uses the full cascade, a successful result via relay is a false positive. The probe handler must be a special code path: raw QUIC connect only.Anchor candidacy checklist
- - -Probe refresh schedule
-Session fallback for full anchors
-When an anchor's mesh is full (101/101), new nodes fall back to a session connection for matchmaking. The anchor accepts referral requests over session connections, not just mesh.
- -Remaining gaps
-anchorsfield but it's not consulted during address resolutionRole under the v0.8 model
+Social routing is the warm path for the people you actually talk to: step 0 of the address resolution cascade, the delivery route for DMs, and the freshness signal that drives the top tiers of the update cadence (see Update Cadence & Keep-Alive — messaged, followed, and vouched relationships check most often). Content from social contacts still arrives via the CDN like everything else; the social layer only carries the addresses and the relationship state.