feat: v0.8 Iteration C — narrow mesh, N1-N4 uniques index, anchor convection
Topology rework. The mesh's job is width; depth now comes from knowledge.
Slots:
- Local/Wide collapse into ONE mesh pool: 20 desktop / 15 mobile
- Temp referral slots +4..+10 ABOVE the cap (10 desktop / 4 mobile), carry no
knowledge exchange, graduate into a freed mesh slot or expire (5 min TTL),
never evict an established mesh peer
- Fixes the Wide-never-fills bug: outbound paths hardcoded Local while the
growth gate counted Local only, so growth stopped at 71 and 20 slots were
inbound-only. Also closes register_connection paths that enforced no cap.
Knowledge — two-pool uniques announce (replaces N1/N2/N3 share lists):
- Pool 1 (forwardable): our N0-N2 uniques, deduped. Receiver stores shifted one
bounce deeper as N1-N3, reporter-tagged.
- Pool 2 (terminal): our N3, deduped against pool 1. Receiver stores as N4,
USED for search/resolution but NEVER re-announced. No N5 anywhere.
- Uniques merge mesh peers + social directs + CDN file authors + holder peers,
so receivers cannot tell how we know an ID
- Anchor entries carry addresses; every other entry is a bare ID. The pools
double as the anchor directory.
- Dedup at both store and announce; per-bounce caps; 2 MB payload ceiling
- Device-tiered depth: mobile holds 3 bounces and drops the terminal pool
Anchor convection (register loop retired — 0xC0 AnchorRegister deleted):
- Rolling in-memory caller window (no registration DB): a caller gets the 2
most recent viable prior callers, its address goes to the next 2, then
rotates out; still-connected callers preferred; RelayIntroduce coordinated
when both ends are live
- Request classes: entry (<2 connections) always served; top-up refused
cheaply under load, and the refusal feeds the adaptive weights
- Stochastic per-disconnect action {nothing | anchor intro | mesh intro} with
weights from local anchor density + refusal feedback. No thresholds, no
jitter timer. Recovery (mesh < 2) stays immediate and non-stochastic.
- Anchors mined from the uniques pools incl. retained pools of disconnected
peers; known_anchors demoted to a bootstrap cache
Security fixes from review: remote address poisoning (hearsay anchors could be
dialled — known_anchors/is_anchor now written only from completed handshakes),
N4 leaking back into announcements via the anchor mirror (infinite propagation),
missing payload size caps, unbounded candidate scoring, and a bulk SQLite write
held under the ConnectionManager mutex.
deploy.sh: version regex now captures pre-release suffixes (0.8.0-alpha would
have truncated to 0.8.0 and broken every artifact path), announce channel
derives from the suffix, and the anchor runs --publish-registry at genesis.
228 core tests; a3 integration 9/9; new c_topology_test.sh 33/33 (both
independently re-run). EDM corpse, session-relay gating, Iteration A/B intact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LGiPD2cF75mnvneSCjdDC5
This commit is contained in:
Scott Reimers
parent
36e3871c4b
commit
e756fabb11
14 changed files with 5503 additions and 2367 deletions
|
|
@ -230,6 +230,11 @@ async fn main() -> anyhow::Result<()> {
|
|||
println!(" redundancy Show replica counts for your posts");
|
||||
println!(" worm <node_id> Worm lookup (find peer beyond 3-hop map)");
|
||||
println!(" connections Show mesh connections");
|
||||
println!(" slots Show mesh / temp-referral slot usage");
|
||||
println!(" uniques [node_id_hex] Uniques index counts, or where an ID is known");
|
||||
println!(" convection Adaptive convection weights + pool-mined anchors");
|
||||
println!(" convect <anchor_id> Ask an anchor for peers (convection)");
|
||||
println!(" uniques-pull Exchange uniques indexes with mesh peers");
|
||||
println!(" social-routes Show social routing cache");
|
||||
println!(" name <display_name> Set your display name");
|
||||
println!(" register <name> [keywords...] Register default persona in the network registry (30d, re-run to renew)");
|
||||
|
|
@ -248,13 +253,22 @@ async fn main() -> anyhow::Result<()> {
|
|||
|
||||
// Start background tasks (v2: mesh connections)
|
||||
let _accept_handle = node.start_accept_loop();
|
||||
let _pull_handle = node.start_pull_cycle(); // 60s tiered pull cycle
|
||||
let _sync_handle = node.start_sync_cycle(); // 60s: uniques-index exchange + content sync
|
||||
let _diff_handle = node.start_diff_cycle(120); // 2 min routing diff
|
||||
let _rebalance_handle = node.start_rebalance_cycle(600); // 10 min rebalance
|
||||
let _growth_handle = node.start_growth_loop(); // reactive mesh growth
|
||||
// Reactive mesh growth. `ITSGOIN_TEST_NO_GROWTH` suppresses it so an
|
||||
// integration test can hold an exact topology (on loopback the growth loop
|
||||
// otherwise collapses any chain into a full mesh within seconds, and a
|
||||
// full mesh has no N2 at all — every peer is a direct). Test gate only.
|
||||
let _growth_handle = if std::env::var("ITSGOIN_TEST_NO_GROWTH").is_ok() {
|
||||
eprintln!("[test gate] growth loop disabled (ITSGOIN_TEST_NO_GROWTH)");
|
||||
None
|
||||
} else {
|
||||
Some(node.start_growth_loop())
|
||||
};
|
||||
let _recovery_handle = node.start_recovery_loop(); // reactive anchor reconnect on mesh loss
|
||||
let _checkin_handle = node.start_social_checkin_cycle(3600); // 1 hour social checkin
|
||||
let _anchor_handle = node.start_anchor_register_cycle(600); // 10 min anchor register
|
||||
let _convection_handle = node.start_convection_loop(); // stochastic anchor convection + anchor self-probe
|
||||
let _upnp_tcp_handle = node.start_upnp_tcp_renewal_cycle(); // UPnP TCP lease renewal
|
||||
let _http_handle = node.start_http_server(); // HTTP post delivery (if publicly reachable)
|
||||
let _bootstrap_check = node.start_bootstrap_connectivity_check(); // 24h isolation check
|
||||
|
|
@ -858,7 +872,7 @@ async fn main() -> anyhow::Result<()> {
|
|||
println!("(no mesh connections)");
|
||||
} else {
|
||||
println!("Mesh connections ({}):", conns.len());
|
||||
for (nid, slot_kind, connected_at) in conns {
|
||||
for (nid, slot, connected_at) in conns {
|
||||
let name = node.get_display_name(&nid).await.unwrap_or(None);
|
||||
let id_short = &hex::encode(nid)[..12];
|
||||
let label = name.map(|n| format!("{} ({})", n, id_short))
|
||||
|
|
@ -870,7 +884,95 @@ async fn main() -> anyhow::Result<()> {
|
|||
.as_millis() as u64;
|
||||
(now.saturating_sub(connected_at)) / 1000
|
||||
};
|
||||
println!(" {} [{:?}] connected {}s ago", label, slot_kind, duration_secs);
|
||||
println!(" {} [{}] connected {}s ago", label, slot, duration_secs);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// v0.8 Iteration C observability: slot pool + uniques index.
|
||||
"slots" => {
|
||||
let conns = node.list_connections().await;
|
||||
let mesh = conns.iter().filter(|(_, s, _)| s.is_mesh()).count();
|
||||
let temp = conns.len() - mesh;
|
||||
println!("mesh {}/{} temp-referral {}/{}",
|
||||
mesh, node.device_profile().mesh_slots(),
|
||||
temp, node.device_profile().temp_referral_slots());
|
||||
for (nid, slot, _) in conns {
|
||||
println!(" {} {}", &hex::encode(nid)[..12], slot);
|
||||
}
|
||||
}
|
||||
|
||||
"uniques" => {
|
||||
let storage = node.storage.get().await;
|
||||
if parts.len() > 1 {
|
||||
match itsgoin_core::parse_node_id_hex(parts[1]) {
|
||||
Ok(target) => {
|
||||
let mut any = false;
|
||||
for bounce in 2u8..=4 {
|
||||
for reporter in storage.find_reporters_at(&target, bounce).unwrap_or_default() {
|
||||
println!(" N{} via {}", bounce, &hex::encode(reporter)[..12]);
|
||||
any = true;
|
||||
}
|
||||
}
|
||||
if !any {
|
||||
println!("(not in the uniques index)");
|
||||
}
|
||||
}
|
||||
Err(e) => println!("Bad node id: {}", e),
|
||||
}
|
||||
} else {
|
||||
println!("N2 {} N3 {} N4 {} (anchor density {:.2})",
|
||||
storage.count_distinct_n2().unwrap_or(0),
|
||||
storage.count_distinct_n3().unwrap_or(0),
|
||||
storage.count_distinct_n4().unwrap_or(0),
|
||||
storage.anchor_density().unwrap_or(0.0));
|
||||
}
|
||||
}
|
||||
|
||||
// The v0.8 "pull": exchange uniques indexes with every mesh peer.
|
||||
"uniques-pull" => {
|
||||
let n = node.uniques_pull().await;
|
||||
println!("uniques-pull: exchanged with {} mesh peers", n);
|
||||
}
|
||||
|
||||
// On-demand convection against a named anchor.
|
||||
"convect" => {
|
||||
if parts.len() < 2 {
|
||||
println!("Usage: convect <anchor_node_id_hex>");
|
||||
} else {
|
||||
match itsgoin_core::parse_node_id_hex(parts[1]) {
|
||||
Ok(anchor) => match node.convection_request(anchor).await {
|
||||
Ok((_, true, ms)) => println!("convection: refused in {}ms", ms),
|
||||
Ok((n, false, ms)) => println!("convection: {} new peers in {}ms", n, ms),
|
||||
Err(e) => println!("convection failed: {}", e),
|
||||
},
|
||||
Err(e) => println!("Bad node id: {}", e),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Convection diagnostics: the adaptive action weights, the
|
||||
// pool-mined anchor directory, and a manual entry-class request.
|
||||
"convection" => {
|
||||
let (density, bias, mesh) = node.network.conn_handle().convection_state().await;
|
||||
let cap = node.device_profile().mesh_slots();
|
||||
let fill = if cap == 0 { 1.0 } else { mesh as f64 / cap as f64 };
|
||||
let w = itsgoin_core::connection::convection_weights(density, bias, fill);
|
||||
let total = w.nothing + w.anchor + w.mesh;
|
||||
println!("anchor_density {:.3} anchor_bias {:.2} mesh {}/{}", density, bias, mesh, cap);
|
||||
println!("weights: nothing {:.3} anchor {:.3} mesh {:.3} (P: {:.0}% / {:.0}% / {:.0}%)",
|
||||
w.nothing, w.anchor, w.mesh,
|
||||
100.0 * w.nothing / total, 100.0 * w.anchor / total, 100.0 * w.mesh / total);
|
||||
let anchors = {
|
||||
let storage = node.storage.get().await;
|
||||
storage.list_pool_anchors(16).unwrap_or_default()
|
||||
};
|
||||
if anchors.is_empty() {
|
||||
println!("pool anchors: (none)");
|
||||
} else {
|
||||
println!("pool anchors ({}):", anchors.len());
|
||||
for (nid, addrs) in anchors {
|
||||
println!(" {} {}", &hex::encode(nid)[..12], addrs.join(","));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -14,7 +14,7 @@ use crate::content::compute_post_id;
|
|||
use crate::crypto;
|
||||
use crate::storage::Storage;
|
||||
use crate::types::{
|
||||
GroupKeyDistributionContent, GroupKeyRecord, GroupMemberKey, NodeId, Post, PostId,
|
||||
GroupKeyDistributionContent, GroupKeyRecord, NodeId, Post, PostId,
|
||||
PostVisibility, PostingIdentity, VisibilityIntent,
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -10,16 +10,15 @@ use tracing::{debug, error, info, warn};
|
|||
use crate::activity::{ActivityCategory, ActivityLevel, ActivityLog};
|
||||
use crate::blob::BlobStore;
|
||||
use crate::connection::{initial_exchange_accept, initial_exchange_connect, ConnHandle, ConnectionActor, ConnectionManager, ExchangeResult};
|
||||
use crate::content::verify_post_id;
|
||||
use crate::protocol::{
|
||||
read_message_type, read_payload, write_typed_message, BlobRequestPayload, BlobResponsePayload,
|
||||
MessageType, ProfileUpdatePayload,
|
||||
PullSyncRequestPayload, PullSyncResponsePayload, RefuseRedirectPayload,
|
||||
RefuseRedirectPayload,
|
||||
ALPN,
|
||||
};
|
||||
use crate::storage::StoragePool;
|
||||
use crate::types::{
|
||||
DeviceProfile, DeviceRole, NodeId, PeerSlotKind, Post, PostId,
|
||||
DeviceProfile, DeviceRole, NodeId, MeshSlot, Post, PostId,
|
||||
PostVisibility, PublicProfile, SessionReachMethod, WormResult,
|
||||
};
|
||||
|
||||
|
|
@ -734,13 +733,16 @@ impl Network {
|
|||
recv: iroh::endpoint::RecvStream,
|
||||
is_mesh: &AtomicBool,
|
||||
) -> bool {
|
||||
// Try to allocate a slot
|
||||
let accepted = conn_handle.accept_connection(
|
||||
// Try to allocate a slot. `None` = both the mesh pool and the temp
|
||||
// referral band are full.
|
||||
let slot = conn_handle.accept_connection(
|
||||
conn.clone(), remote_node_id, Some(remote_sock),
|
||||
).await;
|
||||
|
||||
info!(peer = hex::encode(remote_node_id), accepted, "try_mesh_upgrade: accept_connection result");
|
||||
if !accepted {
|
||||
info!(peer = hex::encode(remote_node_id), accepted = slot.is_some(), "try_mesh_upgrade: accept_connection result");
|
||||
let slot = match slot {
|
||||
Some(s) => s,
|
||||
None => {
|
||||
let redirect = conn_handle.pick_random_redirect_peer(&remote_node_id).await;
|
||||
let payload = RefuseRedirectPayload {
|
||||
reason: "slots full".to_string(),
|
||||
|
|
@ -752,13 +754,19 @@ impl Network {
|
|||
debug!(peer = hex::encode(remote_node_id), "Refused mesh connection (slots full)");
|
||||
conn_handle.add_session(remote_node_id, conn.clone(), crate::types::SessionReachMethod::Direct, Some(remote_sock)).await;
|
||||
conn_handle.log_activity(ActivityLevel::Info, ActivityCategory::Connection, format!("Refused {} mesh (slots full), added session", &hex::encode(remote_node_id)[..8]), Some(remote_node_id));
|
||||
return false;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
{
|
||||
let s = storage.get().await;
|
||||
// bugs-fixed #2: address always persisted, temp slots included.
|
||||
let _ = s.upsert_peer(&remote_node_id, &[remote_sock], None);
|
||||
let _ = s.add_mesh_peer(&remote_node_id, PeerSlotKind::Local, 0);
|
||||
// But mesh_peers ONLY for a real mesh slot — that omission is what
|
||||
// keeps temp referral peers out of our uniques announce.
|
||||
if slot.is_mesh() {
|
||||
let _ = s.add_mesh_peer(&remote_node_id);
|
||||
}
|
||||
if s.has_social_route(&remote_node_id).unwrap_or(false) {
|
||||
let _ = s.touch_social_route_connect(
|
||||
&remote_node_id,
|
||||
|
|
@ -774,9 +782,10 @@ impl Network {
|
|||
let our_nat_type = conn_handle.nat_type().await;
|
||||
let our_http_capable = conn_handle.is_http_capable();
|
||||
let our_http_addr = conn_handle.http_addr();
|
||||
match initial_exchange_accept(storage, &our_node_id, send, recv, remote_node_id, anchor_addr, Some(remote_sock), our_nat_type, our_http_capable, our_http_addr, conn_handle.device_role(), None, false).await {
|
||||
let our_depth = conn_handle.max_bounce();
|
||||
match initial_exchange_accept(storage, &our_node_id, send, recv, remote_node_id, anchor_addr, Some(remote_sock), our_nat_type, our_http_capable, our_http_addr, conn_handle.device_role(), None, false, our_depth, slot.is_mesh()).await {
|
||||
Ok(()) => {
|
||||
info!(peer = hex::encode(remote_node_id), "Initial exchange complete (upgraded to mesh)");
|
||||
info!(peer = hex::encode(remote_node_id), slot = %slot, "Initial exchange complete (upgraded to mesh)");
|
||||
conn_handle.log_activity(ActivityLevel::Info, ActivityCategory::Connection, format!("Upgraded {} to mesh", &hex::encode(remote_node_id)[..8]), Some(remote_node_id));
|
||||
}
|
||||
Err(e) => {
|
||||
|
|
@ -819,12 +828,15 @@ impl Network {
|
|||
let conn = ConnectionManager::connect_to_unlocked(&self.endpoint, addr).await?;
|
||||
|
||||
// Register the established connection
|
||||
self.conn_handle.register_connection(peer_id, conn.clone(), addrs, PeerSlotKind::Local).await;
|
||||
let slot = match self.conn_handle.register_connection(peer_id, conn.clone(), addrs).await {
|
||||
Some(s) => s,
|
||||
None => anyhow::bail!("no slot available (mesh + temp full)"),
|
||||
};
|
||||
let anchor_addr = self.conn_handle.build_anchor_advertised_addr().await;
|
||||
let our_nat_type = self.conn_handle.nat_type().await;
|
||||
|
||||
// Initial exchange WITHOUT holding conn_mgr lock
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, peer_id, anchor_addr, our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None).await? {
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, peer_id, anchor_addr, our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None, self.conn_handle.max_bounce(), slot.is_mesh()).await? {
|
||||
ExchangeResult::Accepted { duplicate_active } => {
|
||||
if duplicate_active {
|
||||
self.duplicate_detected.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||
|
|
@ -870,27 +882,38 @@ impl Network {
|
|||
}
|
||||
}
|
||||
|
||||
/// Pull from all connected peers.
|
||||
pub async fn pull_from_all(&self) -> anyhow::Result<PullStats> {
|
||||
/// TRANSITIONAL content sync from every connected peer, plus the
|
||||
/// engagement fetch that Iteration A's registry and greetings ride on.
|
||||
///
|
||||
/// Routes through the actor path (`content_sync_unlocked`), NOT a second
|
||||
/// local implementation: the old `Network::pull_from_peer` stored posts
|
||||
/// with `store_post_with_visibility` instead of `control::receive_post`, so
|
||||
/// it silently dropped `intent` (control posts — deletes, revocations, key
|
||||
/// distributions — were never processed) and never called
|
||||
/// `touch_file_holder`, which is why posts received on the startup pull
|
||||
/// registered no CDN holder.
|
||||
pub async fn content_sync_all(&self) -> anyhow::Result<PullStats> {
|
||||
let peers = self.conn_handle.connected_peers().await;
|
||||
|
||||
let mut total_posts = 0;
|
||||
let mut success = 0;
|
||||
|
||||
for peer_id in peers {
|
||||
// Uses Network::pull_from_peer which doesn't hold conn_mgr lock during I/O
|
||||
let result = self.pull_from_peer(&peer_id).await;
|
||||
let result = self.conn_handle.content_sync_from_peer(&peer_id).await;
|
||||
match result {
|
||||
Ok(stats) => {
|
||||
total_posts += stats.posts_received;
|
||||
success += 1;
|
||||
// Also fetch engagement data
|
||||
// Engagement fetch is what carries Iteration A's registry
|
||||
// entries and greeting comments (BlobHeaderRequest/Response
|
||||
// + BlobHeaderDiff). It MUST stay driven on a timer — this
|
||||
// is the pull side of that safety net.
|
||||
let _ = self.conn_handle.fetch_engagement_from_peer(&peer_id).await;
|
||||
if stats.posts_received > 0 {
|
||||
info!(
|
||||
peer = hex::encode(peer_id),
|
||||
posts = stats.posts_received,
|
||||
"Pulled posts"
|
||||
"Content sync: received posts"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -898,7 +921,7 @@ impl Network {
|
|||
debug!(
|
||||
peer = hex::encode(peer_id),
|
||||
error = %e,
|
||||
"Pull failed"
|
||||
"Content sync failed"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -910,82 +933,85 @@ impl Network {
|
|||
})
|
||||
}
|
||||
|
||||
/// Broadcast routing diff to all connected peers.
|
||||
/// Uses ConnHandle to get diff data, then sends outside the lock.
|
||||
pub async fn broadcast_diff(&self) -> anyhow::Result<usize> {
|
||||
use crate::protocol::{NodeListUpdatePayload, write_typed_message, MessageType};
|
||||
|
||||
let snapshot = self.conn_handle.get_diff_data().await;
|
||||
|
||||
if snapshot.n1_added.is_empty() && snapshot.n1_removed.is_empty()
|
||||
&& snapshot.n2_added.is_empty() && snapshot.n2_removed.is_empty()
|
||||
{
|
||||
return Ok(0);
|
||||
/// Run the uniques-index exchange (0x40/0x41) against every MESH peer.
|
||||
///
|
||||
/// This is what a "pull" means in v0.8: an exchange of the IDs each side
|
||||
/// has a live path to. Symmetric, so one round trip refreshes both indexes.
|
||||
/// Temp referral slots are skipped — they carry no knowledge.
|
||||
pub async fn uniques_pull_all(&self) -> usize {
|
||||
let conns = self.conn_handle.get_connection_map().await;
|
||||
let mut exchanged = 0;
|
||||
for (peer_id, conn, slot, _) in conns {
|
||||
if !slot.is_mesh() {
|
||||
continue;
|
||||
}
|
||||
let fut = crate::connection::ConnectionManager::uniques_pull_unlocked(
|
||||
&self.conn_mgr,
|
||||
conn,
|
||||
&peer_id,
|
||||
);
|
||||
match tokio::time::timeout(std::time::Duration::from_secs(20), fut).await {
|
||||
Ok(Ok(n)) => {
|
||||
exchanged += 1;
|
||||
if n > 0 {
|
||||
debug!(peer = hex::encode(peer_id), rows = n, "Uniques pull: index updated");
|
||||
}
|
||||
}
|
||||
Ok(Err(e)) => debug!(peer = hex::encode(peer_id), error = %e, "Uniques pull failed"),
|
||||
Err(_) => debug!(peer = hex::encode(peer_id), "Uniques pull timed out"),
|
||||
}
|
||||
}
|
||||
exchanged
|
||||
}
|
||||
|
||||
let payload = NodeListUpdatePayload {
|
||||
seq: snapshot.diff_seq,
|
||||
n1_added: snapshot.n1_added,
|
||||
n1_removed: snapshot.n1_removed,
|
||||
n2_added: snapshot.n2_added,
|
||||
n2_removed: snapshot.n2_removed,
|
||||
/// Broadcast our uniques announce to every MESH peer.
|
||||
///
|
||||
/// Temporary referral slots are excluded — they carry no knowledge
|
||||
/// exchange. Peers that advertised `depth < 4` get the terminal pool
|
||||
/// stripped before send (the biggest saving on a cellular link).
|
||||
///
|
||||
/// The announce is always a full snapshot; `snapshot.unchanged` is what
|
||||
/// stands in for incremental diffing, so an idle mesh sends nothing.
|
||||
pub async fn broadcast_uniques(&self) -> anyhow::Result<usize> {
|
||||
use crate::protocol::{write_typed_message, MessageType, UniquesSlice};
|
||||
|
||||
let snapshot = self.conn_handle.get_announce_data().await;
|
||||
let payload = match snapshot.payload {
|
||||
Some(p) if !snapshot.unchanged => p,
|
||||
_ => return Ok(0),
|
||||
};
|
||||
|
||||
let mut sent = 0;
|
||||
for (peer_id, conn) in &snapshot.connections {
|
||||
for (peer_id, conn, peer_depth) in &snapshot.connections {
|
||||
let per_peer = crate::protocol::UniquesAnnouncePayload {
|
||||
seq: payload.seq,
|
||||
full: payload.full,
|
||||
fwd: payload.fwd.clone(),
|
||||
term: if *peer_depth >= 4 { payload.term.clone() } else { UniquesSlice::default() },
|
||||
depth: payload.depth,
|
||||
};
|
||||
let result = async {
|
||||
let mut send = conn.open_uni().await?;
|
||||
write_typed_message(&mut send, MessageType::NodeListUpdate, &payload).await?;
|
||||
write_typed_message(&mut send, MessageType::UniquesAnnounce, &per_peer).await?;
|
||||
send.finish()?;
|
||||
anyhow::Ok(())
|
||||
}.await;
|
||||
if result.is_ok() {
|
||||
sent += 1;
|
||||
} else {
|
||||
debug!(peer = hex::encode(peer_id), "Failed to send routing diff");
|
||||
debug!(peer = hex::encode(peer_id), "Failed to send uniques announce");
|
||||
}
|
||||
}
|
||||
Ok(sent)
|
||||
}
|
||||
|
||||
/// Broadcast full N1/N2 state to all mesh peers (periodic catch-up for missed diffs).
|
||||
/// Periodic catch-up: force the next announce even if the pools are
|
||||
/// unchanged, so a peer that missed one resynchronises.
|
||||
/// (The old `broadcast_full_state` is subsumed — every announce is already
|
||||
/// a full snapshot.)
|
||||
pub async fn broadcast_full_state(&self) -> anyhow::Result<usize> {
|
||||
use crate::protocol::{NodeListUpdatePayload, write_typed_message, MessageType};
|
||||
|
||||
let snapshot = self.conn_handle.get_diff_data().await;
|
||||
|
||||
// Build full state: all current N1 and N2 as "added", nothing removed
|
||||
let all_n1 = self.conn_handle.connected_peers().await;
|
||||
let all_n2 = {
|
||||
let storage = self.storage.get().await;
|
||||
storage.build_n2_share().unwrap_or_default()
|
||||
};
|
||||
|
||||
if all_n1.is_empty() && all_n2.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
|
||||
let payload = NodeListUpdatePayload {
|
||||
seq: snapshot.diff_seq,
|
||||
n1_added: all_n1,
|
||||
n1_removed: vec![],
|
||||
n2_added: all_n2,
|
||||
n2_removed: vec![],
|
||||
};
|
||||
|
||||
let mut sent = 0;
|
||||
for (_peer_id, conn) in &snapshot.connections {
|
||||
let result = async {
|
||||
let mut send = conn.open_uni().await?;
|
||||
write_typed_message(&mut send, MessageType::NodeListUpdate, &payload).await?;
|
||||
send.finish()?;
|
||||
anyhow::Ok(())
|
||||
}.await;
|
||||
if result.is_ok() {
|
||||
sent += 1;
|
||||
}
|
||||
}
|
||||
Ok(sent)
|
||||
self.conn_handle.force_announce().await;
|
||||
self.broadcast_uniques().await
|
||||
}
|
||||
|
||||
/// Push a profile update to all audience members (ephemeral-capable).
|
||||
|
|
@ -1191,7 +1217,7 @@ impl Network {
|
|||
}
|
||||
|
||||
/// Get connection info for display.
|
||||
pub async fn connection_info(&self) -> Vec<(NodeId, PeerSlotKind, u64)> {
|
||||
pub async fn connection_info(&self) -> Vec<(NodeId, MeshSlot, u64)> {
|
||||
self.conn_handle.connection_info().await
|
||||
}
|
||||
|
||||
|
|
@ -1226,11 +1252,15 @@ impl Network {
|
|||
}
|
||||
|
||||
match self.connect_to_peer(peer_id, addr.clone()).await {
|
||||
Ok(()) => Ok(()),
|
||||
Ok(()) => {
|
||||
self.promote_proven_anchor(peer_id, &addr).await;
|
||||
Ok(())
|
||||
}
|
||||
Err(e) if e.to_string().contains("mesh refused") => {
|
||||
// Anchor refused mesh — reconnect as session for registration
|
||||
let conn = ConnectionManager::connect_to_unlocked(&self.endpoint, addr).await?;
|
||||
let conn = ConnectionManager::connect_to_unlocked(&self.endpoint, addr.clone()).await?;
|
||||
self.conn_handle.add_session(peer_id, conn, crate::types::SessionReachMethod::Direct, None).await;
|
||||
self.promote_proven_anchor(peer_id, &addr).await;
|
||||
self.conn_handle.log_activity(
|
||||
ActivityLevel::Info,
|
||||
ActivityCategory::Anchor,
|
||||
|
|
@ -1243,6 +1273,28 @@ impl Network {
|
|||
}
|
||||
}
|
||||
|
||||
/// An anchor we have COMPLETED a QUIC handshake to. iroh has proven the
|
||||
/// node ID owns this path, so — and only so — the entry may enter the
|
||||
/// authoritative address state: `peers.is_anchor` and the `known_anchors`
|
||||
/// bootstrap cache.
|
||||
///
|
||||
/// This is the counterpart to removing the announce-path mirror. Hearsay
|
||||
/// anchor claims live in `reachable` only (mineable via `list_pool_anchors`,
|
||||
/// which is what "the pools ARE the anchor directory" means); proven ones
|
||||
/// live here, which is also what makes `known_anchors.last_seen_ms`
|
||||
/// ordering meaningful again — and keeps the DNS-refreshed bootstrap anchor
|
||||
/// from being evicted by a flood of gossip-sourced entries.
|
||||
async fn promote_proven_anchor(&self, peer_id: NodeId, addr: &iroh::EndpointAddr) {
|
||||
let socks: Vec<std::net::SocketAddr> = addr.ip_addrs().copied().collect();
|
||||
if socks.is_empty() {
|
||||
return;
|
||||
}
|
||||
let storage = self.storage.get().await;
|
||||
let _ = storage.upsert_peer(&peer_id, &socks, None);
|
||||
let _ = storage.set_peer_anchor(&peer_id, true);
|
||||
let _ = storage.upsert_known_anchor(&peer_id, &socks);
|
||||
}
|
||||
|
||||
/// Register an already-established QUIC connection as a mesh peer.
|
||||
/// Sends InitialExchange first — if the remote accepts (responds with
|
||||
/// InitialExchange), registers as mesh and spawns the stream loop.
|
||||
|
|
@ -1259,13 +1311,15 @@ impl Network {
|
|||
let anchor_addr = self.conn_handle.build_anchor_advertised_addr().await;
|
||||
let our_nat_type = self.conn_handle.nat_type().await;
|
||||
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, peer_id, anchor_addr, our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None).await {
|
||||
// Decide the slot BEFORE the exchange: the slot class is what says
|
||||
// whether this handshake carries a uniques announce.
|
||||
// bugs-fixed #3: a connection is registered before anything can drop it.
|
||||
let slot = match self.conn_handle.register_connection(peer_id, conn.clone(), vec![]).await {
|
||||
Some(s) => s,
|
||||
None => anyhow::bail!("no slot available (mesh + temp full)"),
|
||||
};
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, peer_id, anchor_addr, our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None, self.conn_handle.max_bounce(), slot.is_mesh()).await {
|
||||
Ok(ExchangeResult::Accepted { .. }) => {
|
||||
self.conn_handle.register_connection(peer_id, conn.clone(), vec![], PeerSlotKind::Local).await;
|
||||
{
|
||||
let s = self.storage.get().await;
|
||||
let _ = s.add_mesh_peer(&peer_id, PeerSlotKind::Local, 0);
|
||||
}
|
||||
|
||||
// Spawn the per-connection stream loop
|
||||
let conn_data = self.conn_handle.get_connection_map().await;
|
||||
|
|
@ -1280,10 +1334,16 @@ impl Network {
|
|||
Ok(ExchangeResult::Refused { redirect }) => {
|
||||
let redir_info = redirect.as_ref().map(|r| r.n.clone());
|
||||
info!(peer = hex::encode(peer_id), redirect = ?redir_info, "Mesh refused after hole punch, keeping as session");
|
||||
self.conn_handle.disconnect_peer(&peer_id).await;
|
||||
self.conn_handle.add_session(peer_id, conn, crate::types::SessionReachMethod::HolePunch, None).await;
|
||||
anyhow::bail!("mesh refused: slots full");
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
Err(e) => {
|
||||
// Roll back the slot we reserved — otherwise a failed handshake
|
||||
// holds a mesh slot until the zombie sweep.
|
||||
self.conn_handle.disconnect_peer(&peer_id).await;
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1344,7 +1404,9 @@ impl Network {
|
|||
None,
|
||||
);
|
||||
|
||||
if remaining == 0 {
|
||||
// Recovery threshold is mesh < 2 (design.html §lifecycle), not zero:
|
||||
// a single remaining peer is a partition waiting to happen.
|
||||
if remaining < 2 {
|
||||
self.conn_handle.notify_recovery();
|
||||
} else {
|
||||
self.conn_handle.notify_growth();
|
||||
|
|
@ -1364,7 +1426,7 @@ impl Network {
|
|||
for peer_id in &newly_connected {
|
||||
let conn = self.conn_handle.get_connection(peer_id).await;
|
||||
if let Some(conn) = conn {
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, *peer_id, anchor_addr.clone(), our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None).await {
|
||||
match initial_exchange_connect(&self.storage, &self.our_node_id, &conn, *peer_id, anchor_addr.clone(), our_nat_type, self.is_http_capable(), self.http_addr(), Some(self.device_role), None, self.conn_handle.max_bounce(), true).await {
|
||||
Ok(ExchangeResult::Accepted { .. }) => {}
|
||||
Ok(ExchangeResult::Refused { redirect }) => {
|
||||
debug!(peer = hex::encode(peer_id), "Auto-connect refused, disconnecting");
|
||||
|
|
@ -1444,10 +1506,10 @@ impl Network {
|
|||
|
||||
loop {
|
||||
// Check slots + pick candidate via ConnHandle (no lock contention)
|
||||
let available = self.conn_handle.available_local_slots().await;
|
||||
let available = self.conn_handle.available_mesh_slots().await;
|
||||
if available == 0 {
|
||||
debug!("Growth loop: local slots full");
|
||||
self.log_activity(ActivityLevel::Info, ActivityCategory::Growth, "Local slots full".into(), None);
|
||||
debug!("Growth loop: mesh slots full");
|
||||
self.log_activity(ActivityLevel::Info, ActivityCategory::Growth, "Mesh slots full".into(), None);
|
||||
break;
|
||||
}
|
||||
let candidate = {
|
||||
|
|
@ -1476,25 +1538,39 @@ impl Network {
|
|||
self.log_activity(ActivityLevel::Info, ActivityCategory::Growth, format!("Trying candidate {} (score {:.1})", &hex::encode(candidate_id)[..8], score), Some(candidate_id));
|
||||
|
||||
// Resolve address via ConnHandle (no lock during I/O)
|
||||
let asked_a_reporter;
|
||||
let addr_str = {
|
||||
let local_addr = self.conn_handle.resolve_peer_addr_local(&candidate_id).await;
|
||||
if let Some(endpoint_addr) = local_addr {
|
||||
asked_a_reporter = true;
|
||||
endpoint_addr.ip_addrs().next().map(|a| a.to_string())
|
||||
} else {
|
||||
// Network resolution: get reporter connections, resolve outside lock
|
||||
// Network resolution: get reporter connections, resolve
|
||||
// outside lock. The horizon here must match the horizon
|
||||
// the CANDIDATE SCORER uses (2..=4, same as
|
||||
// `ConnectionManager::resolve_address`) — when it did
|
||||
// not, the growth loop scored N4 candidates it then had
|
||||
// no reporter to ask about, and burned a
|
||||
// `consecutive_failures` slot plus an UNREACHABLE_EXPIRY
|
||||
// suppression on every one of them.
|
||||
let reporters_and_conns = {
|
||||
let storage = self.storage.get().await;
|
||||
let n2 = storage.find_in_n2(&candidate_id).unwrap_or_default();
|
||||
let n3 = storage.find_in_n3(&candidate_id).unwrap_or_default();
|
||||
let mut reporter_set: std::collections::HashSet<NodeId> =
|
||||
std::collections::HashSet::new();
|
||||
for bounce in 2..=4u8 {
|
||||
for r in storage.find_reporters_at(&candidate_id, bounce).unwrap_or_default() {
|
||||
reporter_set.insert(r);
|
||||
}
|
||||
}
|
||||
drop(storage);
|
||||
|
||||
let conn_map = self.conn_handle.get_connection_map().await;
|
||||
let reporter_set: std::collections::HashSet<NodeId> = n2.into_iter().chain(n3).collect();
|
||||
conn_map.into_iter()
|
||||
.filter(|(nid, _, _, _)| reporter_set.contains(nid))
|
||||
.map(|(_, conn, _, _)| conn)
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
asked_a_reporter = !reporters_and_conns.is_empty();
|
||||
let mut resolved = None;
|
||||
for conn in reporters_and_conns {
|
||||
let result: anyhow::Result<Option<String>> = async {
|
||||
|
|
@ -1521,10 +1597,19 @@ impl Network {
|
|||
None => {
|
||||
debug!(
|
||||
peer = hex::encode(candidate_id),
|
||||
"Growth loop: no address, marking unreachable"
|
||||
asked_a_reporter,
|
||||
"Growth loop: no address"
|
||||
);
|
||||
self.log_activity(ActivityLevel::Warn, ActivityCategory::Growth, format!("No address for {}", &hex::encode(candidate_id)[..8]), Some(candidate_id));
|
||||
self.conn_handle.mark_unreachable(&candidate_id);
|
||||
// Only a candidate we could actually ask about, and
|
||||
// that came back with nothing, has earned an
|
||||
// UNREACHABLE_EXPIRY suppression. Failing for want of a
|
||||
// connected reporter says nothing about the candidate —
|
||||
// suppressing it there would hide it even after a
|
||||
// shallower reporter shows up.
|
||||
if asked_a_reporter {
|
||||
self.conn_handle.mark_unreachable(&candidate_id);
|
||||
}
|
||||
consecutive_failures += 1;
|
||||
if consecutive_failures >= 3 {
|
||||
debug!("Growth loop: 3 consecutive failures, backing off");
|
||||
|
|
@ -1560,7 +1645,7 @@ impl Network {
|
|||
consecutive_failures = 0;
|
||||
|
||||
// Broadcast diff so peers learn about our new connection
|
||||
let _ = self.broadcast_diff().await;
|
||||
let _ = self.broadcast_uniques().await;
|
||||
|
||||
// Brief pause to let InitialExchange update N2/N3 before next pick
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
|
|
@ -1618,7 +1703,7 @@ impl Network {
|
|||
|
||||
if introduced {
|
||||
consecutive_failures = 0;
|
||||
let _ = self.broadcast_diff().await;
|
||||
let _ = self.broadcast_uniques().await;
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
} else {
|
||||
self.conn_handle.mark_unreachable(&candidate_id);
|
||||
|
|
@ -1635,64 +1720,6 @@ impl Network {
|
|||
}
|
||||
}
|
||||
|
||||
/// Pull posts from a peer (persistent if available, ephemeral otherwise).
|
||||
pub async fn pull_from_peer(&self, peer_id: &NodeId) -> anyhow::Result<PullStats> {
|
||||
let conn = self.get_connection(peer_id).await?;
|
||||
let (our_follows, follows_sync, our_personas) = {
|
||||
let storage = self.storage.get().await;
|
||||
(
|
||||
storage.list_follows()?,
|
||||
storage.get_follows_with_last_sync().unwrap_or_default(),
|
||||
storage.list_posting_identities().unwrap_or_default(),
|
||||
)
|
||||
};
|
||||
// Merged pull: include every posting identity we hold so DMs addressed
|
||||
// to any of our personas match on recipient. Our network NodeId is
|
||||
// never an author nor a wrapped_key recipient — including it would
|
||||
// never match and would leak the network↔posting boundary.
|
||||
let mut query_list = our_follows;
|
||||
for pi in &our_personas {
|
||||
if !query_list.contains(&pi.node_id) {
|
||||
query_list.push(pi.node_id);
|
||||
}
|
||||
}
|
||||
let (mut send, mut recv) = conn.open_bi().await?;
|
||||
write_typed_message(
|
||||
&mut send,
|
||||
MessageType::PullSyncRequest,
|
||||
&PullSyncRequestPayload {
|
||||
follows: query_list,
|
||||
since_ms: follows_sync,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
send.finish()?;
|
||||
let msg_type = read_message_type(&mut recv).await?;
|
||||
if msg_type != MessageType::PullSyncResponse {
|
||||
anyhow::bail!("expected PullSyncResponse, got {:?}", msg_type);
|
||||
}
|
||||
let response: PullSyncResponsePayload = read_payload(&mut recv, 64 * 1024 * 1024).await?;
|
||||
let now_ms = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis() as u64;
|
||||
let storage = self.storage.get().await;
|
||||
let mut posts_received = 0;
|
||||
for sp in &response.posts {
|
||||
if !storage.is_deleted(&sp.id)? && verify_post_id(&sp.id, &sp.post) {
|
||||
if storage.store_post_with_visibility(&sp.id, &sp.post, &sp.visibility)? {
|
||||
posts_received += 1;
|
||||
}
|
||||
// Protocol v4: update last_sync_ms for the author
|
||||
let _ = storage.update_follow_last_sync(&sp.post.author, now_ms);
|
||||
}
|
||||
}
|
||||
Ok(PullStats {
|
||||
peers_pulled: 1,
|
||||
posts_received,
|
||||
})
|
||||
}
|
||||
|
||||
/// Send a uni-stream message. Uses persistent connection if available, ephemeral otherwise.
|
||||
pub async fn send_to_peer_uni<T: Serialize>(
|
||||
&self,
|
||||
|
|
@ -1940,41 +1967,146 @@ impl Network {
|
|||
Some(addr)
|
||||
}
|
||||
|
||||
// ---- Anchor referral delegation ----
|
||||
// ---- Anchor convection delegation (design.html §anchors) ----
|
||||
|
||||
/// Request peer referrals from an anchor peer (mesh or session).
|
||||
/// Does NOT hold conn_mgr lock during network I/O.
|
||||
pub async fn request_anchor_referrals(
|
||||
/// Ask an anchor for peers. The request itself enrols us in that anchor's
|
||||
/// rolling window — there is no registration message; 0xC0 is retired.
|
||||
///
|
||||
/// Does NOT hold the conn_mgr lock during network I/O. A refusal comes back
|
||||
/// as a single small message, so the `refused` case costs one round trip,
|
||||
/// not a timeout.
|
||||
pub async fn request_convection(
|
||||
&self,
|
||||
anchor: &NodeId,
|
||||
) -> anyhow::Result<Vec<crate::protocol::AnchorReferral>> {
|
||||
class: crate::protocol::ConvectionClass,
|
||||
) -> anyhow::Result<crate::protocol::ConvectionResponsePayload> {
|
||||
let conn = self.conn_handle.get_any_connection(anchor).await
|
||||
.ok_or_else(|| anyhow::anyhow!("anchor peer not connected (mesh or session)"))?;
|
||||
|
||||
// Self-reported addresses, best first: the stable advertised address
|
||||
// (if we are an anchor ourselves), then the UPnP external mapping, then
|
||||
// whatever the endpoint knows. The anchor prefers what it OBSERVES over
|
||||
// all of these; these only help when it cannot observe us.
|
||||
let endpoint = self.conn_handle.endpoint().await;
|
||||
let our_addrs: Vec<String> = endpoint.addr().ip_addrs()
|
||||
let mut our_addrs: Vec<String> = endpoint.addr().ip_addrs()
|
||||
.filter(|s| is_publicly_routable(s))
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
if let Some(ext) = self.conn_handle.upnp_external_addr().await {
|
||||
let ext_str = ext.to_string();
|
||||
if !our_addrs.contains(&ext_str) {
|
||||
our_addrs.insert(0, ext_str);
|
||||
}
|
||||
}
|
||||
if let Some(anchor_addr) = self.conn_handle.build_anchor_advertised_addr().await {
|
||||
if !our_addrs.contains(&anchor_addr) {
|
||||
our_addrs.insert(0, anchor_addr);
|
||||
}
|
||||
}
|
||||
|
||||
let request = crate::protocol::AnchorReferralRequestPayload {
|
||||
let request = crate::protocol::ConvectionRequestPayload {
|
||||
requester: self.our_node_id,
|
||||
requester_addresses: our_addrs,
|
||||
class,
|
||||
};
|
||||
|
||||
let (mut send, mut recv) = conn.open_bi().await?;
|
||||
crate::protocol::write_typed_message(&mut send, crate::protocol::MessageType::AnchorReferralRequest, &request).await?;
|
||||
crate::protocol::write_typed_message(&mut send, crate::protocol::MessageType::ConvectionRequest, &request).await?;
|
||||
send.finish()?;
|
||||
|
||||
let msg_type = crate::protocol::read_message_type(&mut recv).await?;
|
||||
if msg_type != crate::protocol::MessageType::AnchorReferralResponse {
|
||||
anyhow::bail!("expected AnchorReferralResponse, got {:?}", msg_type);
|
||||
if msg_type != crate::protocol::MessageType::ConvectionResponse {
|
||||
anyhow::bail!("expected ConvectionResponse, got {:?}", msg_type);
|
||||
}
|
||||
let response: crate::protocol::AnchorReferralResponsePayload = crate::protocol::read_payload(&mut recv, 4096).await?;
|
||||
let response: crate::protocol::ConvectionResponsePayload =
|
||||
crate::protocol::read_payload(&mut recv, 4096).await?;
|
||||
|
||||
// Touch session to prevent idle reaping
|
||||
self.conn_handle.touch_session_if_exists(anchor);
|
||||
|
||||
Ok(response.referrals)
|
||||
// Adaptive feedback (round 5): a refusal shifts weight toward mesh
|
||||
// introductions and other anchors; a served request drifts it back.
|
||||
if response.refused {
|
||||
debug!(anchor = hex::encode(anchor), "Convection: refused (top-up, no capacity)");
|
||||
self.conn_handle.record_convection_refusal(anchor);
|
||||
} else {
|
||||
self.conn_handle.record_convection_success(anchor);
|
||||
}
|
||||
|
||||
// The NAT filter probe used to ride the register message. It has to
|
||||
// ride something, and this is now the only message we send an anchor.
|
||||
if self.conn_handle.nat_filtering().await == crate::types::NatFiltering::Unknown {
|
||||
if let Err(e) = self.request_nat_filter_probe(anchor).await {
|
||||
debug!(error = %e, "NAT filter probe request failed");
|
||||
}
|
||||
}
|
||||
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
/// Act on one convection response: connect to each referral, landing it in
|
||||
/// whatever slot class is free (mesh first, then the temp referral band
|
||||
/// above the cap). Returns how many connections were established.
|
||||
///
|
||||
/// `introduced` referrals skip the extra introduction round trip — the
|
||||
/// anchor already pushed a RelayIntroduce toward them naming us, so they
|
||||
/// are punching outward while we dial.
|
||||
pub async fn act_on_convection(
|
||||
&self,
|
||||
anchor: &NodeId,
|
||||
response: &crate::protocol::ConvectionResponsePayload,
|
||||
) -> usize {
|
||||
let mut connected = 0;
|
||||
for referral in &response.referrals {
|
||||
if referral.node_id == self.our_node_id {
|
||||
continue;
|
||||
}
|
||||
if self.conn_handle.is_connected(&referral.node_id).await {
|
||||
continue;
|
||||
}
|
||||
let mut ok = false;
|
||||
// bugs-fixed #8 (receive side): a referral address is a third
|
||||
// party's claim. Dialling `127.0.0.1:22` or `192.168.1.1:443`
|
||||
// because an anchor said so is attacker-directed internal probing
|
||||
// from our own host, and every such connect() enters iroh's
|
||||
// per-endpoint path store permanently (v0.7.3 EDM lesson).
|
||||
let usable: Option<&String> = referral.addresses.iter().find(|a| {
|
||||
a.parse::<std::net::SocketAddr>()
|
||||
.map(|sa| crate::connection::convection_addr_ok(&sa))
|
||||
.unwrap_or(false)
|
||||
});
|
||||
if let Some(addr_str) = usable {
|
||||
let connect_str = format!("{}@{}", hex::encode(referral.node_id), addr_str);
|
||||
if let Ok((rid, raddr)) = crate::parse_connect_string(&connect_str) {
|
||||
let fut = self.connect_to_peer(rid, raddr);
|
||||
match tokio::time::timeout(std::time::Duration::from_secs(15), fut).await {
|
||||
Ok(Ok(())) => {
|
||||
info!(peer = hex::encode(rid), "Convection: connected to referred peer");
|
||||
ok = true;
|
||||
}
|
||||
Ok(Err(e)) => debug!(error = %e, peer = hex::encode(rid), "Convection: direct connect failed"),
|
||||
Err(_) => debug!(peer = hex::encode(rid), "Convection: direct connect timed out"),
|
||||
}
|
||||
}
|
||||
}
|
||||
if !ok && !referral.introduced {
|
||||
// No coordinated introduction was possible — ask for one.
|
||||
match self.connect_via_introduction_as_mesh(referral.node_id, *anchor).await {
|
||||
Ok(()) => {
|
||||
info!(peer = hex::encode(referral.node_id), "Convection: connected via introduction");
|
||||
ok = true;
|
||||
}
|
||||
Err(e) => debug!(error = %e, peer = hex::encode(referral.node_id), "Convection: introduction failed"),
|
||||
}
|
||||
}
|
||||
if ok {
|
||||
connected += 1;
|
||||
}
|
||||
}
|
||||
if connected > 0 {
|
||||
self.notify_growth().await;
|
||||
}
|
||||
connected
|
||||
}
|
||||
|
||||
/// Request NAT filter probe from an anchor (determines address-restricted vs port-restricted).
|
||||
|
|
@ -2028,55 +2160,6 @@ impl Network {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Register our address with an anchor peer (mesh or session).
|
||||
/// Also runs NAT filter probe if filtering is still Unknown.
|
||||
/// Does NOT hold conn_mgr lock during network I/O.
|
||||
pub async fn send_anchor_register(&self, anchor: &NodeId) -> anyhow::Result<()> {
|
||||
let conn = self.conn_handle.get_any_connection(anchor).await
|
||||
.ok_or_else(|| anyhow::anyhow!("anchor peer not connected (mesh or session)"))?;
|
||||
|
||||
let endpoint = self.conn_handle.endpoint().await;
|
||||
let mut our_addrs: Vec<String> = endpoint.addr().ip_addrs()
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
// Prepend UPnP external address (most useful for remote peers)
|
||||
if let Some(ext) = self.conn_handle.upnp_external_addr().await {
|
||||
let ext_str = ext.to_string();
|
||||
if !our_addrs.contains(&ext_str) {
|
||||
our_addrs.insert(0, ext_str);
|
||||
}
|
||||
}
|
||||
// Prepend stable anchor advertised address
|
||||
if let Some(anchor_addr) = self.conn_handle.build_anchor_advertised_addr().await {
|
||||
if !our_addrs.contains(&anchor_addr) {
|
||||
our_addrs.insert(0, anchor_addr);
|
||||
}
|
||||
}
|
||||
|
||||
let payload = crate::protocol::AnchorRegisterPayload {
|
||||
node_id: self.our_node_id,
|
||||
addresses: our_addrs,
|
||||
};
|
||||
|
||||
let mut send = conn.open_uni().await?;
|
||||
crate::protocol::write_typed_message(&mut send, crate::protocol::MessageType::AnchorRegister, &payload).await?;
|
||||
send.finish()?;
|
||||
|
||||
// Touch session to prevent idle reaping
|
||||
self.conn_handle.touch_session_if_exists(anchor);
|
||||
|
||||
debug!(anchor = hex::encode(anchor), "Registered with anchor");
|
||||
|
||||
// Also run NAT filter probe if filtering is still Unknown
|
||||
let needs_probe = self.conn_handle.nat_filtering().await == crate::types::NatFiltering::Unknown;
|
||||
if needs_probe {
|
||||
if let Err(e) = self.request_nat_filter_probe(anchor).await {
|
||||
debug!(error = %e, "NAT filter probe request failed");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Check if a peer is a known anchor.
|
||||
pub async fn is_anchor_peer(&self, node_id: &NodeId) -> bool {
|
||||
let storage = self.storage.get().await;
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -27,13 +27,25 @@ pub struct SyncPost {
|
|||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[repr(u8)]
|
||||
pub enum MessageType {
|
||||
NodeListUpdate = 0x01,
|
||||
UniquesAnnounce = 0x01,
|
||||
InitialExchange = 0x02,
|
||||
AddressRequest = 0x03,
|
||||
AddressResponse = 0x04,
|
||||
RefuseRedirect = 0x05,
|
||||
PullSyncRequest = 0x40,
|
||||
PullSyncResponse = 0x41,
|
||||
/// The uniques-index exchange (design.html §sync). A pull is no longer a
|
||||
/// post transfer: it is "if you want these IDs, talk to me and I'll help
|
||||
/// you find them". Symmetric — request carries our pools, response carries
|
||||
/// theirs, one round trip updates both sides.
|
||||
UniquesPullRequest = 0x40,
|
||||
UniquesPullResponse = 0x41,
|
||||
/// TRANSITIONAL (Iteration D folds this into the update-cadence scheduler +
|
||||
/// CDN replication). Content still has to move somewhere while
|
||||
/// PostFetchRequest (0xD4) serves Public posts only: DMs, group posts,
|
||||
/// Friends and FoFClosed posts have no other carrier. Splitting it off
|
||||
/// 0x40/0x41 keeps §sync honest — a *pull* is not a post transfer, because
|
||||
/// post transfer is not a pull.
|
||||
ContentSyncRequest = 0x46,
|
||||
ContentSyncResponse = 0x47,
|
||||
ProfileUpdate = 0x50,
|
||||
WormQuery = 0x60,
|
||||
WormResponse = 0x61,
|
||||
|
|
@ -49,9 +61,11 @@ pub enum MessageType {
|
|||
RelayIntroduceResult = 0xB1,
|
||||
SessionRelay = 0xB2,
|
||||
CircleProfileUpdate = 0xB4,
|
||||
AnchorRegister = 0xC0,
|
||||
AnchorReferralRequest = 0xC1,
|
||||
AnchorReferralResponse = 0xC2,
|
||||
// 0xC0 retired in v0.8 — anchors keep a rolling window of recent callers
|
||||
// fed by the request itself; there is no registration message and no
|
||||
// registration database.
|
||||
ConvectionRequest = 0xC1,
|
||||
ConvectionResponse = 0xC2,
|
||||
AnchorProbeRequest = 0xC3,
|
||||
AnchorProbeResult = 0xC4,
|
||||
PortScanHeartbeat = 0xC5,
|
||||
|
|
@ -73,13 +87,15 @@ pub enum MessageType {
|
|||
impl MessageType {
|
||||
pub fn from_byte(b: u8) -> Option<Self> {
|
||||
match b {
|
||||
0x01 => Some(Self::NodeListUpdate),
|
||||
0x01 => Some(Self::UniquesAnnounce),
|
||||
0x02 => Some(Self::InitialExchange),
|
||||
0x03 => Some(Self::AddressRequest),
|
||||
0x04 => Some(Self::AddressResponse),
|
||||
0x05 => Some(Self::RefuseRedirect),
|
||||
0x40 => Some(Self::PullSyncRequest),
|
||||
0x41 => Some(Self::PullSyncResponse),
|
||||
0x40 => Some(Self::UniquesPullRequest),
|
||||
0x41 => Some(Self::UniquesPullResponse),
|
||||
0x46 => Some(Self::ContentSyncRequest),
|
||||
0x47 => Some(Self::ContentSyncResponse),
|
||||
0x50 => Some(Self::ProfileUpdate),
|
||||
0x60 => Some(Self::WormQuery),
|
||||
0x61 => Some(Self::WormResponse),
|
||||
|
|
@ -94,9 +110,8 @@ impl MessageType {
|
|||
0xB1 => Some(Self::RelayIntroduceResult),
|
||||
0xB2 => Some(Self::SessionRelay),
|
||||
0xB4 => Some(Self::CircleProfileUpdate),
|
||||
0xC0 => Some(Self::AnchorRegister),
|
||||
0xC1 => Some(Self::AnchorReferralRequest),
|
||||
0xC2 => Some(Self::AnchorReferralResponse),
|
||||
0xC1 => Some(Self::ConvectionRequest),
|
||||
0xC2 => Some(Self::ConvectionResponse),
|
||||
0xC3 => Some(Self::AnchorProbeRequest),
|
||||
0xC4 => Some(Self::AnchorProbeResult),
|
||||
0xC5 => Some(Self::PortScanHeartbeat),
|
||||
|
|
@ -124,13 +139,152 @@ impl MessageType {
|
|||
|
||||
// --- Payload structs ---
|
||||
|
||||
/// Initial exchange: N1/N2 node lists + profile + deletes + post_ids + peer addresses
|
||||
/// A packed set of 32-byte IDs: base64 of the raw concatenated bytes.
|
||||
///
|
||||
/// `NodeId = [u8; 32]` serialised through serde_json becomes a JSON array of 32
|
||||
/// decimal numbers (~100-130 bytes/ID — a 3-4x blowup). The uniques pools can
|
||||
/// carry thousands of IDs, so they ride packed instead.
|
||||
pub type PackedIds = String;
|
||||
|
||||
/// Encode a set of IDs into a packed base64 blob.
|
||||
pub fn pack_ids(ids: &[NodeId]) -> PackedIds {
|
||||
use base64::Engine;
|
||||
let mut raw = Vec::with_capacity(ids.len() * 32);
|
||||
for id in ids {
|
||||
raw.extend_from_slice(id);
|
||||
}
|
||||
base64::engine::general_purpose::STANDARD.encode(raw)
|
||||
}
|
||||
|
||||
/// Decode a packed base64 blob back into IDs. Trailing partial IDs are dropped.
|
||||
pub fn unpack_ids(packed: &str) -> Vec<NodeId> {
|
||||
use base64::Engine;
|
||||
let raw = match base64::engine::general_purpose::STANDARD.decode(packed) {
|
||||
Ok(r) => r,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
raw.chunks_exact(32)
|
||||
.map(|c| {
|
||||
let mut id = [0u8; 32];
|
||||
id.copy_from_slice(c);
|
||||
id
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// How many 32-byte IDs a packed base64 blob holds, WITHOUT decoding it.
|
||||
///
|
||||
/// Standard base64 emits 4 chars per 3 input bytes (padded), so decoded length
|
||||
/// is `chars / 4 * 3` minus padding; integer division by 32 absorbs the padding
|
||||
/// slack for any non-empty blob. Exact for every blob `pack_ids` produces.
|
||||
pub fn packed_id_count(packed: &str) -> usize {
|
||||
(packed.len() / 4) * 3 / 32
|
||||
}
|
||||
|
||||
/// The ONLY entry type in a uniques pool that carries an address.
|
||||
///
|
||||
/// design.html §layers: anchor entries carry addresses so the pools double as
|
||||
/// the anchor directory; every other entry is a bare ID (there is no point
|
||||
/// sharing an address that needs an introduction anyway). Addresses MUST be
|
||||
/// filtered through `is_publicly_routable()` before they go on the wire.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct AnchorEntry {
|
||||
/// hex node id (anchors are always node-class)
|
||||
pub i: String,
|
||||
/// e.g. ["1.2.3.4:4433"]
|
||||
pub a: Vec<String>,
|
||||
}
|
||||
|
||||
/// One slice of a uniques pool. Node-class and author-class IDs live in
|
||||
/// separate arrays so a receiver structurally cannot mistake a persona ID for
|
||||
/// a connectable node ID.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct UniquesSlice {
|
||||
/// Connectable network IDs, bare.
|
||||
#[serde(default)]
|
||||
pub nodes: PackedIds,
|
||||
/// Posting/persona/throwaway IDs, bare. CDN-resolvable only — never a
|
||||
/// connect target.
|
||||
#[serde(default)]
|
||||
pub authors: PackedIds,
|
||||
/// Address-bearing anchor entries (node-class by definition).
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub anchors: Vec<AnchorEntry>,
|
||||
}
|
||||
|
||||
impl UniquesSlice {
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.nodes.is_empty() && self.authors.is_empty() && self.anchors.is_empty()
|
||||
}
|
||||
/// Number of IDs carried (for logging / budget checks).
|
||||
///
|
||||
/// `pack_ids` concatenates ALL IDs and base64-encodes the blob ONCE, so the
|
||||
/// encoded length is `4 * ceil(32n/3)` characters — about 42.67 per ID, not
|
||||
/// 44 (44 is the encoding of exactly ONE 32-byte ID). Counting `len()/44`
|
||||
/// undercounts by ~3% at scale and by 33% for a 3-ID blob. Since this backs
|
||||
/// budget checks, it is computed from the DECODED byte length instead.
|
||||
pub fn len(&self) -> usize {
|
||||
packed_id_count(&self.nodes) + packed_id_count(&self.authors) + self.anchors.len()
|
||||
}
|
||||
}
|
||||
|
||||
/// 0x01 — the two-pool uniques announce (design.html §layers).
|
||||
///
|
||||
/// POOL 1 (`fwd`) is FORWARDABLE and indexed by OUR bounce:
|
||||
/// fwd[0] = N0 — ourselves (and our own anchor address, if we are an anchor)
|
||||
/// fwd[1] = N1 — our own uniques: mesh peers, social directs, CDN file
|
||||
/// authors, CDN file-holder peers, merged so the receiver
|
||||
/// cannot tell which source an entry came from
|
||||
/// fwd[2] = N2 — what our N1 reported to us
|
||||
/// The receiver stores `fwd[i]` at bounce `i + 1` (their N1..N3), tagged to us
|
||||
/// as reporter, and MAY re-announce it shifted one deeper.
|
||||
///
|
||||
/// POOL 2 (`term`) is TERMINAL: our N3, deduplicated against every fwd slice.
|
||||
/// The receiver stores it as their N4 and USES it for search / address
|
||||
/// resolution, but NEVER re-announces it. The structural separation — a
|
||||
/// distinct field, not a flag in a merged list — is what makes "never
|
||||
/// re-announce" un-forgettable at the announce builder: the builder simply
|
||||
/// never reads bounce-4 rows. There is no N5 anywhere.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct UniquesAnnouncePayload {
|
||||
pub seq: u64,
|
||||
/// Always true in v0.8 — the announce is a full snapshot that replaces this
|
||||
/// reporter's rows. (Incremental diffing over four layers with anchor
|
||||
/// metadata was not worth the divergence risk; the sender skips the send
|
||||
/// entirely when nothing changed, which recovers most of the saving.)
|
||||
pub full: bool,
|
||||
/// Pool 1, forwardable. Always length 3 (bounces 0, 1, 2).
|
||||
pub fwd: Vec<UniquesSlice>,
|
||||
/// Pool 2, terminal. Empty when the recipient advertised `depth < 4`.
|
||||
#[serde(default)]
|
||||
pub term: UniquesSlice,
|
||||
/// Deepest bounce the SENDER retains (4 desktop, 3 mobile). Lets the peer
|
||||
/// skip building/sending the terminal pool for us — the single biggest
|
||||
/// bandwidth saving on a cellular link.
|
||||
pub depth: u8,
|
||||
}
|
||||
|
||||
impl UniquesAnnouncePayload {
|
||||
pub fn empty(seq: u64, depth: u8) -> Self {
|
||||
Self {
|
||||
seq,
|
||||
full: true,
|
||||
fwd: vec![UniquesSlice::default(), UniquesSlice::default(), UniquesSlice::default()],
|
||||
term: UniquesSlice::default(),
|
||||
depth,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Initial exchange: uniques announce + profile + deletes + post_ids + peer addresses
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct InitialExchangePayload {
|
||||
/// Our connections + social contacts NodeIds (no addresses)
|
||||
pub n1_node_ids: Vec<NodeId>,
|
||||
/// Our deduplicated N2 NodeIds (no addresses)
|
||||
pub n2_node_ids: Vec<NodeId>,
|
||||
/// Our uniques pools (full snapshot). `None` means "this connection carries
|
||||
/// no knowledge" — i.e. the sender placed us in a temporary referral slot.
|
||||
/// Explicit on the wire rather than a pair of empty vectors so the receiver
|
||||
/// can tell "nothing to share" from "not sharing".
|
||||
#[serde(default)]
|
||||
pub uniques: Option<UniquesAnnouncePayload>,
|
||||
/// Our profile
|
||||
pub profile: Option<PublicProfile>,
|
||||
/// Our delete records
|
||||
|
|
@ -162,28 +316,32 @@ pub struct InitialExchangePayload {
|
|||
pub duplicate_active: Option<bool>,
|
||||
}
|
||||
|
||||
/// Incremental N1/N2 changes
|
||||
/// 0x40/0x41 — the uniques-index exchange, both directions.
|
||||
///
|
||||
/// The requester sends its pools; the responder answers with its own. One
|
||||
/// round trip refreshes both indexes, which is what makes an index *exchange*
|
||||
/// rather than a fetch. Content does not ride this path.
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct NodeListUpdatePayload {
|
||||
pub seq: u64,
|
||||
pub n1_added: Vec<NodeId>,
|
||||
pub n1_removed: Vec<NodeId>,
|
||||
pub n2_added: Vec<NodeId>,
|
||||
pub n2_removed: Vec<NodeId>,
|
||||
pub struct UniquesPullPayload {
|
||||
/// The sender's two pools. `None` = "I carry no knowledge on this link"
|
||||
/// (temp referral slot), mirroring `InitialExchangePayload.uniques`.
|
||||
#[serde(default)]
|
||||
pub uniques: Option<UniquesAnnouncePayload>,
|
||||
}
|
||||
|
||||
/// Pull-based post sync request
|
||||
/// 0x46 — TRANSITIONAL content sync request. Identical in shape to the v0.7
|
||||
/// pull request; only the opcode moved. See `MessageType::ContentSyncRequest`.
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct PullSyncRequestPayload {
|
||||
pub struct ContentSyncRequestPayload {
|
||||
/// Our follows (for the responder to filter)
|
||||
pub follows: Vec<NodeId>,
|
||||
/// Per-author last-sync timestamps (Vec of tuples for serde compat)
|
||||
pub since_ms: Vec<(NodeId, u64)>,
|
||||
}
|
||||
|
||||
/// Pull-based post sync response
|
||||
/// 0x47 — TRANSITIONAL content sync response.
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct PullSyncResponsePayload {
|
||||
pub struct ContentSyncResponsePayload {
|
||||
pub posts: Vec<SyncPost>,
|
||||
}
|
||||
|
||||
|
|
@ -387,33 +545,63 @@ pub struct CircleProfileUpdatePayload {
|
|||
pub updated_at: u64,
|
||||
}
|
||||
|
||||
// --- Anchor referral payloads ---
|
||||
// --- Anchor convection payloads (design.html §anchors) ---
|
||||
|
||||
/// Node registers its address with an anchor (uni-stream)
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct AnchorRegisterPayload {
|
||||
pub node_id: NodeId,
|
||||
pub addresses: Vec<String>,
|
||||
/// The one request-class bit (round-5 ruling).
|
||||
///
|
||||
/// ENTRY = bootstrap or recovery, i.e. the caller has fewer than 2 mesh
|
||||
/// connections. Always served — a node that cannot get in cannot come back.
|
||||
/// TOP_UP = growth with a working mesh. Served capacity-permitting and refused
|
||||
/// CHEAPLY (a single response message, no timeout burned). Top-ups are the
|
||||
/// convection medium — they are what keeps referral windows fresh — so they
|
||||
/// are not throttled aggressively; the refusal doubles as the adaptive load
|
||||
/// signal the caller feeds back into its action weights.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum ConvectionClass {
|
||||
Entry,
|
||||
TopUp,
|
||||
}
|
||||
|
||||
/// Node requests peer referrals from an anchor (bi-stream request)
|
||||
impl ConvectionClass {
|
||||
/// Entry class is decided purely by "can this node function right now".
|
||||
pub fn for_mesh_count(mesh_count: usize) -> Self {
|
||||
if mesh_count < 2 { Self::Entry } else { Self::TopUp }
|
||||
}
|
||||
pub fn is_entry(self) -> bool {
|
||||
matches!(self, Self::Entry)
|
||||
}
|
||||
}
|
||||
|
||||
/// 0xC1 — "I'm joining / I need peers". The request itself is what enrols the
|
||||
/// caller in the anchor's rolling window; there is no separate registration.
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct AnchorReferralRequestPayload {
|
||||
pub struct ConvectionRequestPayload {
|
||||
pub requester: NodeId,
|
||||
pub requester_addresses: Vec<String>,
|
||||
pub class: ConvectionClass,
|
||||
}
|
||||
|
||||
/// Anchor responds with peer referrals (bi-stream response)
|
||||
/// 0xC2 — up to 2 recent prior callers, or a cheap refusal.
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct AnchorReferralResponsePayload {
|
||||
pub referrals: Vec<AnchorReferral>,
|
||||
pub struct ConvectionResponsePayload {
|
||||
pub referrals: Vec<ConvectionReferral>,
|
||||
/// Cheap refusal of a TOP_UP request: one message, immediately, so the
|
||||
/// caller never burns a timeout. Never set for ENTRY-class requests.
|
||||
#[serde(default)]
|
||||
pub refused: bool,
|
||||
}
|
||||
|
||||
/// A single peer referral from an anchor
|
||||
/// A single referral out of the anchor's rolling caller window.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct AnchorReferral {
|
||||
pub struct ConvectionReferral {
|
||||
pub node_id: NodeId,
|
||||
pub addresses: Vec<String>,
|
||||
/// The anchor had BOTH ends live and has pushed a RelayIntroduce toward
|
||||
/// this peer naming the caller as requester — so the peer is already
|
||||
/// punching outward. Dial straight through instead of paying for a second
|
||||
/// introduction round trip.
|
||||
#[serde(default)]
|
||||
pub introduced: bool,
|
||||
}
|
||||
|
||||
// --- Anchor probe payloads ---
|
||||
|
|
@ -634,13 +822,15 @@ mod tests {
|
|||
#[test]
|
||||
fn message_type_roundtrip() {
|
||||
let types = [
|
||||
MessageType::NodeListUpdate,
|
||||
MessageType::UniquesAnnounce,
|
||||
MessageType::InitialExchange,
|
||||
MessageType::AddressRequest,
|
||||
MessageType::AddressResponse,
|
||||
MessageType::RefuseRedirect,
|
||||
MessageType::PullSyncRequest,
|
||||
MessageType::PullSyncResponse,
|
||||
MessageType::UniquesPullRequest,
|
||||
MessageType::UniquesPullResponse,
|
||||
MessageType::ContentSyncRequest,
|
||||
MessageType::ContentSyncResponse,
|
||||
MessageType::ProfileUpdate,
|
||||
MessageType::WormQuery,
|
||||
MessageType::WormResponse,
|
||||
|
|
@ -655,9 +845,8 @@ mod tests {
|
|||
MessageType::RelayIntroduceResult,
|
||||
MessageType::SessionRelay,
|
||||
MessageType::CircleProfileUpdate,
|
||||
MessageType::AnchorRegister,
|
||||
MessageType::AnchorReferralRequest,
|
||||
MessageType::AnchorReferralResponse,
|
||||
MessageType::ConvectionRequest,
|
||||
MessageType::ConvectionResponse,
|
||||
MessageType::AnchorProbeRequest,
|
||||
MessageType::AnchorProbeResult,
|
||||
MessageType::PortScanHeartbeat,
|
||||
|
|
@ -779,56 +968,105 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn anchor_register_payload_roundtrip() {
|
||||
let payload = AnchorRegisterPayload {
|
||||
node_id: [1u8; 32],
|
||||
addresses: vec!["192.168.1.5:4433".to_string(), "10.0.0.1:4433".to_string()],
|
||||
};
|
||||
let json = serde_json::to_string(&payload).unwrap();
|
||||
let decoded: AnchorRegisterPayload = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(decoded.node_id, [1u8; 32]);
|
||||
assert_eq!(decoded.addresses.len(), 2);
|
||||
assert_eq!(decoded.addresses[0], "192.168.1.5:4433");
|
||||
fn convection_request_carries_the_class_bit() {
|
||||
for class in [ConvectionClass::Entry, ConvectionClass::TopUp] {
|
||||
let payload = ConvectionRequestPayload {
|
||||
requester: [2u8; 32],
|
||||
requester_addresses: vec!["10.0.0.2:4433".to_string()],
|
||||
class,
|
||||
};
|
||||
let json = serde_json::to_string(&payload).unwrap();
|
||||
let decoded: ConvectionRequestPayload = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(decoded.requester, [2u8; 32]);
|
||||
assert_eq!(decoded.requester_addresses, vec!["10.0.0.2:4433"]);
|
||||
assert_eq!(decoded.class, class);
|
||||
}
|
||||
// The class is derived purely from "can this node function right now".
|
||||
assert!(ConvectionClass::for_mesh_count(0).is_entry());
|
||||
assert!(ConvectionClass::for_mesh_count(1).is_entry());
|
||||
assert!(!ConvectionClass::for_mesh_count(2).is_entry());
|
||||
assert!(!ConvectionClass::for_mesh_count(20).is_entry());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn anchor_referral_request_payload_roundtrip() {
|
||||
let payload = AnchorReferralRequestPayload {
|
||||
requester: [2u8; 32],
|
||||
requester_addresses: vec!["10.0.0.2:4433".to_string()],
|
||||
};
|
||||
let json = serde_json::to_string(&payload).unwrap();
|
||||
let decoded: AnchorReferralRequestPayload = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(decoded.requester, [2u8; 32]);
|
||||
assert_eq!(decoded.requester_addresses, vec!["10.0.0.2:4433"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn anchor_referral_response_payload_roundtrip() {
|
||||
let payload = AnchorReferralResponsePayload {
|
||||
fn convection_response_roundtrip_including_cheap_refusal() {
|
||||
let payload = ConvectionResponsePayload {
|
||||
referrals: vec![
|
||||
AnchorReferral {
|
||||
ConvectionReferral {
|
||||
node_id: [3u8; 32],
|
||||
addresses: vec!["10.0.0.3:4433".to_string()],
|
||||
introduced: false,
|
||||
},
|
||||
AnchorReferral {
|
||||
ConvectionReferral {
|
||||
node_id: [4u8; 32],
|
||||
addresses: vec!["10.0.0.4:4433".to_string(), "192.168.1.4:4433".to_string()],
|
||||
addresses: vec!["10.0.0.4:4433".to_string()],
|
||||
introduced: true,
|
||||
},
|
||||
],
|
||||
refused: false,
|
||||
};
|
||||
let json = serde_json::to_string(&payload).unwrap();
|
||||
let decoded: AnchorReferralResponsePayload = serde_json::from_str(&json).unwrap();
|
||||
let decoded: ConvectionResponsePayload = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(decoded.referrals.len(), 2);
|
||||
assert_eq!(decoded.referrals[0].node_id, [3u8; 32]);
|
||||
assert_eq!(decoded.referrals[0].addresses, vec!["10.0.0.3:4433"]);
|
||||
assert_eq!(decoded.referrals[1].node_id, [4u8; 32]);
|
||||
assert_eq!(decoded.referrals[1].addresses.len(), 2);
|
||||
assert!(!decoded.referrals[0].introduced);
|
||||
assert!(decoded.referrals[1].introduced);
|
||||
assert!(!decoded.refused);
|
||||
|
||||
// Empty referrals
|
||||
let empty = AnchorReferralResponsePayload { referrals: vec![] };
|
||||
let json2 = serde_json::to_string(&empty).unwrap();
|
||||
let decoded2: AnchorReferralResponsePayload = serde_json::from_str(&json2).unwrap();
|
||||
// A refusal is one small message with no referrals — the caller reads
|
||||
// it immediately instead of burning a timeout.
|
||||
let refusal = ConvectionResponsePayload { referrals: vec![], refused: true };
|
||||
let json2 = serde_json::to_string(&refusal).unwrap();
|
||||
assert!(json2.len() < 64, "refusal must stay tiny: {}", json2);
|
||||
let decoded2: ConvectionResponsePayload = serde_json::from_str(&json2).unwrap();
|
||||
assert!(decoded2.refused);
|
||||
assert!(decoded2.referrals.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uniques_pull_is_an_index_exchange_not_a_post_transfer() {
|
||||
let payload = UniquesPullPayload {
|
||||
uniques: Some(UniquesAnnouncePayload::empty(7, 4)),
|
||||
};
|
||||
let json = serde_json::to_string(&payload).unwrap();
|
||||
let decoded: UniquesPullPayload = serde_json::from_str(&json).unwrap();
|
||||
let u = decoded.uniques.unwrap();
|
||||
assert_eq!(u.seq, 7);
|
||||
assert_eq!(u.depth, 4);
|
||||
assert_eq!(u.fwd.len(), 3);
|
||||
// No post field exists on this payload at all — content rides 0x46/0x47.
|
||||
assert!(!json.contains("posts"));
|
||||
|
||||
// A temp referral slot carries no knowledge in either direction.
|
||||
let none = UniquesPullPayload { uniques: None };
|
||||
let json2 = serde_json::to_string(&none).unwrap();
|
||||
let decoded2: UniquesPullPayload = serde_json::from_str(&json2).unwrap();
|
||||
assert!(decoded2.uniques.is_none());
|
||||
}
|
||||
|
||||
/// `len()` backs budget checks, so it must be EXACT — not the old
|
||||
/// `bytes / 44` approximation, which assumed one base64 encoding per ID
|
||||
/// when `pack_ids` encodes the whole concatenated blob once.
|
||||
#[test]
|
||||
fn uniques_slice_len_counts_packed_ids_exactly() {
|
||||
for n in [0usize, 1, 2, 3, 7, 100, 1000] {
|
||||
let ids: Vec<crate::types::NodeId> = (0..n)
|
||||
.map(|i| {
|
||||
let mut id = [0u8; 32];
|
||||
id[0] = (i % 251) as u8;
|
||||
id[1] = (i / 251) as u8;
|
||||
id
|
||||
})
|
||||
.collect();
|
||||
let packed = pack_ids(&ids);
|
||||
assert_eq!(packed_id_count(&packed), n, "packed_id_count for n={}", n);
|
||||
assert_eq!(unpack_ids(&packed).len(), n);
|
||||
|
||||
let slice = UniquesSlice {
|
||||
nodes: packed.clone(),
|
||||
authors: packed.clone(),
|
||||
anchors: vec![AnchorEntry { i: "aa".into(), a: vec!["1.2.3.4:1".into()] }],
|
||||
};
|
||||
assert_eq!(slice.len(), n * 2 + 1, "UniquesSlice::len for n={}", n);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -649,26 +649,67 @@ impl NatProfile {
|
|||
}
|
||||
|
||||
/// Device profile — determines connection slot budget
|
||||
///
|
||||
/// v0.8 (Iteration C): the Local/Wide split is gone. There is ONE mesh pool
|
||||
/// (`mesh_slots`) plus a strictly-additional band of temporary referral slots
|
||||
/// (`temp_referral_slots`) that live ABOVE the mesh cap and carry no knowledge
|
||||
/// exchange. See design.html §connections.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum DeviceProfile {
|
||||
/// Desktop: 71 local + 20 wide = 91 mesh connections
|
||||
/// Desktop: 20 mesh + up to 10 temp referral
|
||||
Desktop,
|
||||
/// Mobile: 7 local + 5 wide = 12 mesh connections
|
||||
/// Mobile: 15 mesh + up to 4 temp referral
|
||||
Mobile,
|
||||
}
|
||||
|
||||
impl DeviceProfile {
|
||||
pub fn local_slots(&self) -> usize {
|
||||
/// The single mesh pool size (v0.8 narrow mesh, deep knowledge).
|
||||
///
|
||||
/// `ITSGOIN_TEST_MESH_SLOTS` overrides it so integration tests can reach
|
||||
/// the temp-referral boundary with a handful of nodes instead of 21. Test
|
||||
/// gate only — same pattern as `ITSGOIN_TEST_ALLOW_REGISTRY_GENESIS`.
|
||||
///
|
||||
/// Read ONCE into a `OnceLock`: this is called from `ConnectionManager`
|
||||
/// construction and from every path that wants a fresh cap, and an env
|
||||
/// lookup per call is a syscall-shaped cost on a hot path (and lets the
|
||||
/// mesh cap change under a running node, which nothing expects).
|
||||
pub fn mesh_slots(&self) -> usize {
|
||||
static OVERRIDE: std::sync::OnceLock<Option<usize>> = std::sync::OnceLock::new();
|
||||
let over = *OVERRIDE.get_or_init(|| {
|
||||
std::env::var("ITSGOIN_TEST_MESH_SLOTS")
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<usize>().ok())
|
||||
.filter(|n| *n > 0)
|
||||
});
|
||||
if let Some(n) = over {
|
||||
return n;
|
||||
}
|
||||
match self {
|
||||
DeviceProfile::Desktop => 71,
|
||||
DeviceProfile::Mobile => 7,
|
||||
DeviceProfile::Desktop => 20,
|
||||
DeviceProfile::Mobile => 15,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn wide_slots(&self) -> usize {
|
||||
/// Temporary referral slots, allocated ABOVE `mesh_slots`. These exist only
|
||||
/// to facilitate introductions: they carry no uniques exchange, are never
|
||||
/// persisted to `mesh_peers`, and either graduate into a freed mesh slot or
|
||||
/// expire. Ruling: +4..+10 above the cap.
|
||||
pub fn temp_referral_slots(&self) -> usize {
|
||||
match self {
|
||||
DeviceProfile::Desktop => 20,
|
||||
DeviceProfile::Mobile => 5,
|
||||
DeviceProfile::Desktop => 10,
|
||||
DeviceProfile::Mobile => 4,
|
||||
}
|
||||
}
|
||||
|
||||
/// Deepest bounce this device stores in the `reachable` index.
|
||||
/// Desktop keeps the full 4-bounce horizon; mobile caps at 3 (it drops the
|
||||
/// terminal pool on receipt and advertises `depth = 3` so senders skip
|
||||
/// building it). Alternative considered and NOT implemented: Bloom-compress
|
||||
/// N4 on mobile (see `apply_uniques_announce` doc comment).
|
||||
pub fn max_bounce(&self) -> u8 {
|
||||
match self {
|
||||
DeviceProfile::Desktop => 4,
|
||||
DeviceProfile::Mobile => 3,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -708,24 +749,103 @@ impl std::fmt::Display for SessionReachMethod {
|
|||
}
|
||||
}
|
||||
|
||||
/// Slot kind for mesh connections
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PeerSlotKind {
|
||||
/// Diverse local connections (Desktop: 71, Mobile: 7)
|
||||
Local,
|
||||
/// Bloom-sourced random distant connections (Desktop: 20, Mobile: 5)
|
||||
Wide,
|
||||
/// Which class of slot a live connection occupies.
|
||||
///
|
||||
/// v0.8: `Mesh` is the real pool (capped at `DeviceProfile::mesh_slots`) and is
|
||||
/// the ONLY class that participates in the uniques announce. `TempReferral`
|
||||
/// slots sit above the cap, exist purely to broker introductions, and are never
|
||||
/// written to `mesh_peers` — which is what structurally keeps them out of our
|
||||
/// own announcements. They graduate into a freed mesh slot or expire.
|
||||
///
|
||||
/// NOT on the wire — purely local accounting.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum MeshSlot {
|
||||
Mesh,
|
||||
TempReferral { expires_at_ms: u64 },
|
||||
}
|
||||
|
||||
impl std::fmt::Display for PeerSlotKind {
|
||||
impl MeshSlot {
|
||||
pub fn is_mesh(&self) -> bool {
|
||||
matches!(self, MeshSlot::Mesh)
|
||||
}
|
||||
pub fn is_temp(&self) -> bool {
|
||||
!self.is_mesh()
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for MeshSlot {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
PeerSlotKind::Local => write!(f, "local"),
|
||||
PeerSlotKind::Wide => write!(f, "wide"),
|
||||
MeshSlot::Mesh => write!(f, "mesh"),
|
||||
MeshSlot::TempReferral { .. } => write!(f, "temp"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Class of an ID carried in a uniques pool.
|
||||
///
|
||||
/// A `Node` ID is a network identity we may connect to / hole-punch / relay to.
|
||||
/// An `Author` ID is a posting identity (persona, throwaway, greeting ID). It
|
||||
/// has NO address and NO device linkage (Iteration A stripped
|
||||
/// `AuthorManifest.author_addresses`), so it must NEVER enter the address
|
||||
/// resolution cascade or the growth-candidate scorer — it resolves only through
|
||||
/// CDN holder search. The pools keep the two classes in separate arrays so a
|
||||
/// receiver structurally cannot confuse them.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum IdClass {
|
||||
Node = 0,
|
||||
Author = 1,
|
||||
}
|
||||
|
||||
impl IdClass {
|
||||
pub fn as_i64(self) -> i64 {
|
||||
self as i64
|
||||
}
|
||||
pub fn from_i64(v: i64) -> Self {
|
||||
if v == 1 { IdClass::Author } else { IdClass::Node }
|
||||
}
|
||||
}
|
||||
|
||||
/// One entry in the uniques index / announce pools.
|
||||
///
|
||||
/// `addresses` is non-empty ONLY when `is_anchor` — that is the §layers privacy
|
||||
/// invariant, enforced at the store (`add_reach` blanks the column otherwise)
|
||||
/// and at the wire builder (only `AnchorEntry` has an address field at all).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ReachEntry {
|
||||
pub id: NodeId,
|
||||
pub id_class: IdClass,
|
||||
pub is_anchor: bool,
|
||||
pub addresses: Vec<String>,
|
||||
}
|
||||
|
||||
impl ReachEntry {
|
||||
/// A bare node-class entry (no address).
|
||||
pub fn node(id: NodeId) -> Self {
|
||||
Self { id, id_class: IdClass::Node, is_anchor: false, addresses: Vec::new() }
|
||||
}
|
||||
/// A bare author-class entry (no address, never a connect target).
|
||||
pub fn author(id: NodeId) -> Self {
|
||||
Self { id, id_class: IdClass::Author, is_anchor: false, addresses: Vec::new() }
|
||||
}
|
||||
/// An anchor entry — the only kind that carries an address.
|
||||
pub fn anchor(id: NodeId, addresses: Vec<String>) -> Self {
|
||||
Self { id, id_class: IdClass::Node, is_anchor: true, addresses }
|
||||
}
|
||||
}
|
||||
|
||||
/// The two announce pools, pre-wire.
|
||||
///
|
||||
/// `fwd` is always length 3 (our N0, N1, N2) and is FORWARDABLE — the receiver
|
||||
/// stores `fwd[i]` at bounce `i + 1` and may re-announce it shifted.
|
||||
/// `term` is our N3 and is TERMINAL — the receiver stores it at bounce 4, uses
|
||||
/// it, and never re-announces it.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct UniquesPools {
|
||||
pub fwd: Vec<Vec<ReachEntry>>,
|
||||
pub term: Vec<ReachEntry>,
|
||||
}
|
||||
|
||||
// --- Social Routing Cache ---
|
||||
|
||||
/// How we last reached a social contact
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@
|
|||
use std::net::SocketAddr;
|
||||
use std::num::NonZeroU16;
|
||||
use std::time::Duration;
|
||||
use tracing::{debug, info, warn};
|
||||
use tracing::{debug, info};
|
||||
|
||||
/// An active port mapping. While this value is held, the underlying
|
||||
/// `portmapper::Client` keeps the lease alive in a background task.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ use tracing::info;
|
|||
|
||||
use itsgoin_core::identity::IdentityManager;
|
||||
use itsgoin_core::node::Node;
|
||||
use itsgoin_core::types::{NodeId, PeerSlotKind, Post, PostId, PostVisibility, VisibilityIntent};
|
||||
use itsgoin_core::types::{NodeId, Post, PostId, PostVisibility, VisibilityIntent};
|
||||
|
||||
/// The active Node, swappable on identity switch.
|
||||
type AppNode = Arc<tokio::sync::RwLock<Arc<Node>>>;
|
||||
|
|
@ -1363,7 +1363,7 @@ async fn list_peers(state: State<'_, AppNode>) -> Result<Vec<PeerDto>, String> {
|
|||
.into_iter()
|
||||
.map(|(nid, _, _)| nid)
|
||||
.collect();
|
||||
let (social_ids, n2_ids, n3_ids) = {
|
||||
let (social_ids, n2_ids, n3_ids, n4_ids) = {
|
||||
let storage = node.storage.get().await;
|
||||
let social: std::collections::HashSet<_> = storage
|
||||
.list_social_routes()
|
||||
|
|
@ -1372,7 +1372,7 @@ async fn list_peers(state: State<'_, AppNode>) -> Result<Vec<PeerDto>, String> {
|
|||
.map(|r| r.node_id)
|
||||
.collect();
|
||||
let n2: std::collections::HashSet<_> = storage
|
||||
.build_n2_share()
|
||||
.list_reach_ids_at(2)
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
|
@ -1381,7 +1381,12 @@ async fn list_peers(state: State<'_, AppNode>) -> Result<Vec<PeerDto>, String> {
|
|||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.collect();
|
||||
(social, n2, n3)
|
||||
let n4: std::collections::HashSet<_> = storage
|
||||
.list_distinct_n4()
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.collect();
|
||||
(social, n2, n3, n4)
|
||||
};
|
||||
|
||||
let mut dtos = Vec::with_capacity(records.len());
|
||||
|
|
@ -1406,6 +1411,8 @@ async fn list_peers(state: State<'_, AppNode>) -> Result<Vec<PeerDto>, String> {
|
|||
"n2"
|
||||
} else if n3_ids.contains(&rec.node_id) {
|
||||
"n3"
|
||||
} else if n4_ids.contains(&rec.node_id) {
|
||||
"n4"
|
||||
} else {
|
||||
"known"
|
||||
};
|
||||
|
|
@ -2066,12 +2073,12 @@ async fn list_connections(state: State<'_, AppNode>) -> Result<Vec<ConnectionDto
|
|||
let node = get_node(&state).await;
|
||||
let conns = node.list_connections().await;
|
||||
let mut dtos = Vec::with_capacity(conns.len());
|
||||
for (nid, slot_kind, connected_at) in conns {
|
||||
for (nid, slot, connected_at) in conns {
|
||||
let display_name = node.get_display_name(&nid).await.unwrap_or(None);
|
||||
dtos.push(ConnectionDto {
|
||||
node_id: hex::encode(nid),
|
||||
display_name,
|
||||
slot_kind: format!("{:?}", slot_kind),
|
||||
slot_kind: slot.to_string(),
|
||||
connected_at,
|
||||
});
|
||||
}
|
||||
|
|
@ -2493,11 +2500,13 @@ async fn sync_from_peer(state: State<'_, AppNode>, node_id_hex: String) -> Resul
|
|||
#[derive(Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
struct NetworkSummaryDto {
|
||||
local_count: usize,
|
||||
wide_count: usize,
|
||||
mesh_count: usize,
|
||||
temp_count: usize,
|
||||
mesh_cap: usize,
|
||||
total_connections: usize,
|
||||
n2_distinct: usize,
|
||||
n3_distinct: usize,
|
||||
n4_distinct: usize,
|
||||
has_public_v6: bool,
|
||||
has_public_v4: bool,
|
||||
has_upnp: bool,
|
||||
|
|
@ -2507,26 +2516,24 @@ struct NetworkSummaryDto {
|
|||
async fn get_network_summary(state: State<'_, AppNode>) -> Result<NetworkSummaryDto, String> {
|
||||
let node = get_node(&state).await;
|
||||
let conns = node.list_connections().await;
|
||||
let mut local = 0usize;
|
||||
let mut wide = 0usize;
|
||||
for (_nid, slot_kind, _at) in &conns {
|
||||
match slot_kind {
|
||||
PeerSlotKind::Local => local += 1,
|
||||
PeerSlotKind::Wide => wide += 1,
|
||||
}
|
||||
}
|
||||
let (n2, n3) = {
|
||||
let mesh = conns.iter().filter(|(_, slot, _)| slot.is_mesh()).count();
|
||||
let temp = conns.len() - mesh;
|
||||
let (n2, n3, n4) = {
|
||||
let storage = node.storage.get().await;
|
||||
let n2 = storage.count_distinct_n2().unwrap_or(0);
|
||||
let n3 = storage.count_distinct_n3().unwrap_or(0);
|
||||
(n2, n3)
|
||||
(
|
||||
storage.count_distinct_n2().unwrap_or(0),
|
||||
storage.count_distinct_n3().unwrap_or(0),
|
||||
storage.count_distinct_n4().unwrap_or(0),
|
||||
)
|
||||
};
|
||||
Ok(NetworkSummaryDto {
|
||||
local_count: local,
|
||||
wide_count: wide,
|
||||
mesh_count: mesh,
|
||||
temp_count: temp,
|
||||
mesh_cap: node.device_profile().mesh_slots(),
|
||||
total_connections: conns.len(),
|
||||
n2_distinct: n2,
|
||||
n3_distinct: n3,
|
||||
n4_distinct: n4,
|
||||
has_public_v6: node.network.has_public_v6(),
|
||||
has_public_v4: node.network.is_anchor(),
|
||||
has_upnp: node.network.has_upnp(),
|
||||
|
|
@ -2748,15 +2755,14 @@ struct ActivityLogDto {
|
|||
events: Vec<ActivityEventDto>,
|
||||
rebalance_last_ms: u64,
|
||||
rebalance_interval_secs: u64,
|
||||
anchor_register_last_ms: u64,
|
||||
anchor_register_interval_secs: u64,
|
||||
convection_last_ms: u64,
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
async fn get_activity_log(state: State<'_, AppNode>) -> Result<ActivityLogDto, String> {
|
||||
let node = get_node(&state).await;
|
||||
let events = node.get_activity_log(200);
|
||||
let (rebalance_last, anchor_last) = node.timer_state();
|
||||
let (rebalance_last, convection_last) = node.timer_state();
|
||||
let dto_events: Vec<ActivityEventDto> = events.into_iter().map(|e| {
|
||||
ActivityEventDto {
|
||||
timestamp_ms: e.timestamp_ms,
|
||||
|
|
@ -2770,8 +2776,7 @@ async fn get_activity_log(state: State<'_, AppNode>) -> Result<ActivityLogDto, S
|
|||
events: dto_events,
|
||||
rebalance_last_ms: rebalance_last,
|
||||
rebalance_interval_secs: 600,
|
||||
anchor_register_last_ms: anchor_last,
|
||||
anchor_register_interval_secs: 600,
|
||||
convection_last_ms: convection_last,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -2787,31 +2792,47 @@ async fn trigger_rebalance(state: State<'_, AppNode>) -> Result<String, String>
|
|||
async fn request_referrals(state: State<'_, AppNode>) -> Result<String, String> {
|
||||
let node = get_node(&state).await;
|
||||
let node_id = node.node_id;
|
||||
// Try known_anchors table first (populated by anchor register cycle),
|
||||
// fall back to anchor peers from the peers table (is_anchor = true)
|
||||
let mesh = node.network.conn_handle().mesh_count().await;
|
||||
// Manual trigger uses the same class rule as the automatic path: below 2
|
||||
// mesh peers this is an ENTRY request and is always served.
|
||||
let class = itsgoin_core::protocol::ConvectionClass::for_mesh_count(mesh);
|
||||
|
||||
// Pool-mined anchors first (the uniques pools ARE the anchor directory),
|
||||
// then the known_anchors bootstrap cache, then anchor-flagged peers.
|
||||
let anchors: Vec<(NodeId, Vec<std::net::SocketAddr>)> = {
|
||||
let storage = node.storage.get().await;
|
||||
let known = storage.list_known_anchors().unwrap_or_default();
|
||||
if !known.is_empty() {
|
||||
known
|
||||
} else {
|
||||
storage.list_anchor_peers().unwrap_or_default()
|
||||
.into_iter()
|
||||
.map(|r| (r.node_id, r.addresses))
|
||||
.collect()
|
||||
let mut out: Vec<(NodeId, Vec<std::net::SocketAddr>)> = Vec::new();
|
||||
let mut seen: std::collections::HashSet<NodeId> = std::collections::HashSet::new();
|
||||
for (nid, addrs) in storage.list_pool_anchors(16).unwrap_or_default() {
|
||||
let socks: Vec<std::net::SocketAddr> = addrs.iter().filter_map(|a| a.parse().ok()).collect();
|
||||
if !socks.is_empty() && seen.insert(nid) {
|
||||
out.push((nid, socks));
|
||||
}
|
||||
}
|
||||
for (nid, addrs) in storage.list_known_anchors().unwrap_or_default() {
|
||||
if seen.insert(nid) {
|
||||
out.push((nid, addrs));
|
||||
}
|
||||
}
|
||||
for r in storage.list_anchor_peers().unwrap_or_default() {
|
||||
if seen.insert(r.node_id) {
|
||||
out.push((r.node_id, r.addresses));
|
||||
}
|
||||
}
|
||||
out
|
||||
};
|
||||
if anchors.is_empty() {
|
||||
return Ok("No known anchors".to_string());
|
||||
}
|
||||
let mut total = 0usize;
|
||||
let mut reachable = 0usize;
|
||||
|
||||
let mut connected = 0usize;
|
||||
let mut asked = 0usize;
|
||||
let mut refused = 0usize;
|
||||
for (anchor_nid, anchor_addrs) in &anchors {
|
||||
if *anchor_nid == node_id {
|
||||
continue;
|
||||
}
|
||||
// Connect to anchor if not already connected
|
||||
if !node.network.is_peer_connected(anchor_nid).await {
|
||||
if !node.network.is_peer_connected_or_session(anchor_nid).await {
|
||||
let endpoint_id = match itsgoin_core::EndpointId::from_bytes(anchor_nid) {
|
||||
Ok(eid) => eid,
|
||||
Err(_) => continue,
|
||||
|
|
@ -2820,40 +2841,26 @@ async fn request_referrals(state: State<'_, AppNode>) -> Result<String, String>
|
|||
for sa in anchor_addrs {
|
||||
addr = addr.with_ip_addr(*sa);
|
||||
}
|
||||
if let Err(_) = node.network.connect_to_peer(*anchor_nid, addr).await {
|
||||
if node.network.connect_to_anchor(*anchor_nid, addr).await.is_err() {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
match node.network.request_anchor_referrals(anchor_nid).await {
|
||||
Ok(referrals) => {
|
||||
reachable += 1;
|
||||
for referral in &referrals {
|
||||
if referral.node_id == node_id {
|
||||
continue;
|
||||
}
|
||||
if let Some(addr_str) = referral.addresses.first() {
|
||||
let connect_str = format!(
|
||||
"{}@{}",
|
||||
hex::encode(referral.node_id),
|
||||
addr_str,
|
||||
);
|
||||
if let Ok((rid, raddr)) = itsgoin_core::parse_connect_string(&connect_str) {
|
||||
match node.network.connect_to_peer(rid, raddr).await {
|
||||
Ok(()) => {}
|
||||
Err(_) => {
|
||||
// Direct connect failed (NAT) — try hole punch via anchor
|
||||
let _ = node.network.connect_via_introduction(rid, *anchor_nid).await;
|
||||
}
|
||||
}
|
||||
total += 1;
|
||||
}
|
||||
}
|
||||
match node.network.request_convection(anchor_nid, class).await {
|
||||
Ok(response) => {
|
||||
asked += 1;
|
||||
if response.refused {
|
||||
refused += 1;
|
||||
} else {
|
||||
connected += node.network.act_on_convection(anchor_nid, &response).await;
|
||||
}
|
||||
}
|
||||
Err(_) => {}
|
||||
}
|
||||
}
|
||||
Ok(format!("Got {} referrals from {} anchors", total, reachable))
|
||||
Ok(format!(
|
||||
"Convection: {} new peers from {} anchors ({} refused)",
|
||||
connected, asked, refused
|
||||
))
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
|
|
@ -3197,13 +3204,13 @@ async fn switch_identity(
|
|||
|
||||
// Start background tasks on the new node
|
||||
new_node.start_accept_loop();
|
||||
new_node.start_pull_cycle();
|
||||
new_node.start_sync_cycle();
|
||||
new_node.start_diff_cycle(120);
|
||||
new_node.start_rebalance_cycle(600);
|
||||
new_node.start_growth_loop();
|
||||
new_node.start_recovery_loop();
|
||||
new_node.start_social_checkin_cycle(3600);
|
||||
new_node.start_anchor_register_cycle(600);
|
||||
new_node.start_convection_loop();
|
||||
new_node.start_upnp_tcp_renewal_cycle();
|
||||
new_node.start_http_server();
|
||||
new_node.start_bootstrap_connectivity_check();
|
||||
|
|
@ -3321,13 +3328,13 @@ async fn clear_duplicate_flag(state: State<'_, AppNode>) -> Result<(), String> {
|
|||
node.network.duplicate_detected.store(false, std::sync::atomic::Ordering::Relaxed);
|
||||
// Start the sync tasks that were skipped during bootstrap
|
||||
node.start_accept_loop();
|
||||
node.start_pull_cycle();
|
||||
node.start_sync_cycle();
|
||||
node.start_diff_cycle(120);
|
||||
node.start_rebalance_cycle(600);
|
||||
node.start_growth_loop();
|
||||
node.start_recovery_loop();
|
||||
node.start_social_checkin_cycle(3600);
|
||||
node.start_anchor_register_cycle(600);
|
||||
node.start_convection_loop();
|
||||
node.start_upnp_tcp_renewal_cycle();
|
||||
node.start_http_server();
|
||||
node.start_bootstrap_connectivity_check();
|
||||
|
|
@ -3569,13 +3576,13 @@ pub fn run() {
|
|||
|
||||
// Start all background networking tasks
|
||||
boot_node.start_accept_loop();
|
||||
boot_node.start_pull_cycle();
|
||||
boot_node.start_sync_cycle();
|
||||
boot_node.start_diff_cycle(120);
|
||||
boot_node.start_rebalance_cycle(600);
|
||||
boot_node.start_growth_loop();
|
||||
boot_node.start_recovery_loop();
|
||||
boot_node.start_social_checkin_cycle(3600);
|
||||
boot_node.start_anchor_register_cycle(600);
|
||||
boot_node.start_convection_loop();
|
||||
boot_node.start_upnp_tcp_renewal_cycle();
|
||||
boot_node.start_http_server();
|
||||
boot_node.start_bootstrap_connectivity_check();
|
||||
|
|
|
|||
18
deploy.sh
18
deploy.sh
|
|
@ -18,8 +18,19 @@ SSH_OPTS="-o StrictHostKeyChecking=no"
|
|||
KEYSTORE="itsgoin.keystore"
|
||||
KS_ALIAS="itsgoin"
|
||||
|
||||
VERSION=$(grep '"version"' crates/tauri-app/tauri.conf.json | head -1 | sed 's/.*"\([0-9.]*\)".*/\1/')
|
||||
echo "=== Deploying v${VERSION} ==="
|
||||
# Captures the FULL version string including any pre-release suffix
|
||||
# (e.g. 0.8.0-alpha). The old digits-and-dots-only pattern silently
|
||||
# truncated at the dash, so every ${VERSION} filename below missed the
|
||||
# artifacts Tauri had actually produced.
|
||||
VERSION=$(grep '"version"' crates/tauri-app/tauri.conf.json | head -1 | sed 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/')
|
||||
# Pre-release versions announce on their own channel so stable clients
|
||||
# are not told an alpha is "the" current release.
|
||||
case "$VERSION" in
|
||||
*-*) ANN_CHANNEL="${VERSION#*-}" ;;
|
||||
*) ANN_CHANNEL="stable" ;;
|
||||
esac
|
||||
|
||||
echo "=== Deploying v${VERSION} (announce channel: ${ANN_CHANNEL}) ==="
|
||||
|
||||
# Builds run SERIALLY — parallel cargo invocations write to the same
|
||||
# target/ directory, which causes intermittent failures (linuxdeploy
|
||||
|
|
@ -76,10 +87,11 @@ sshpass -p "$SSH_PASS" ssh $SSH_OPTS "$SSH_HOST" "
|
|||
kill \$(cat ~/itsgoin-anchor.pid 2>/dev/null) 2>/dev/null
|
||||
sleep 1
|
||||
mv ~/bin/itsgoin.new ~/bin/itsgoin && chmod +x ~/bin/itsgoin
|
||||
~/bin/itsgoin ~/itsgoin-anchor-data --publish-registry 2>&1 | tail -3
|
||||
~/bin/itsgoin ~/itsgoin-anchor-data \
|
||||
--announce \
|
||||
--ann-category release \
|
||||
--ann-channel stable \
|
||||
--ann-channel ${ANN_CHANNEL} \
|
||||
--ann-version ${VERSION} \
|
||||
--ann-url https://itsgoin.com/download.html \
|
||||
--ann-title 'ItsGoin v${VERSION} available' \
|
||||
|
|
|
|||
|
|
@ -27,6 +27,12 @@ check() { # check <desc> <cmd...>
|
|||
else FAIL=$((FAIL+1)); echo "FAIL: $desc"; fi
|
||||
}
|
||||
strip_ansi() { sed 's/\x1b\[[0-9;]*m//g' "$1" | grep -v "WARN.*netlink\|WARN.*buffer_tool"; }
|
||||
# Same noise filter for the tail-window checks below. The netlink/mDNS watcher
|
||||
# logs in unpredictable bursts, and a burst landing between a REPL command and
|
||||
# its assertion pushes the answer out of a raw `tail -N` window — a false
|
||||
# failure with nothing wrong in the node.
|
||||
quiet_tail() { grep -av "netlink\|buffer_tool\|iroh_quinn\|swarm_discovery" "$1" | tail -"$2"; }
|
||||
export -f quiet_tail
|
||||
sq() { sqlite3 "/tmp/itsgoin-test$1/itsgoin.db" "$2"; }
|
||||
|
||||
cleanup() {
|
||||
|
|
@ -91,7 +97,7 @@ sleep 3
|
|||
echo "search rust" >&15
|
||||
sleep 8
|
||||
check "node3 re-search finds nothing after signed delete" \
|
||||
bash -c 'tail -20 /tmp/itsgoin-cli3.log | grep -q "no registry matches"'
|
||||
bash -c 'quiet_tail /tmp/itsgoin-cli3.log 20 | grep -q "no registry matches"'
|
||||
|
||||
echo "== step 3: greeting roundtrip =="
|
||||
# node2's bio post id (latest Profile post) from its DB.
|
||||
|
|
@ -128,7 +134,8 @@ sleep 4
|
|||
echo "search rust" >&15
|
||||
sleep 8
|
||||
check "node3 sees exactly one (newest) entry for node2's persona" \
|
||||
bash -c 'tail -6 /tmp/itsgoin-cli3.log | grep -c "rust" | grep -q "^1$" && tail -6 /tmp/itsgoin-cli3.log | grep -q AliceV2'
|
||||
bash -c 'quiet_tail /tmp/itsgoin-cli3.log 6 | grep -c "rust" | grep -q "^1$" &&
|
||||
quiet_tail /tmp/itsgoin-cli3.log 6 | grep -q AliceV2'
|
||||
|
||||
echo
|
||||
echo "== results: $PASS passed, $FAIL failed =="
|
||||
|
|
|
|||
278
scripts/c_topology_test.sh
Executable file
278
scripts/c_topology_test.sh
Executable file
|
|
@ -0,0 +1,278 @@
|
|||
#!/usr/bin/env bash
|
||||
# v0.8 Iteration C integration test — TOPOLOGY.
|
||||
#
|
||||
# Run from the repo root after `cargo build -p itsgoin-cli`, AFTER
|
||||
# scripts/a3_integration_test.sh (C does not replace A). Drives interactive
|
||||
# REPLs over FIFOs, asserts by grepping logs and by sqlite3 against node DBs.
|
||||
#
|
||||
# SCENARIO 1 Two-pool uniques exchange over a chain: pool 1 shifts one bounce
|
||||
# deeper per hop, pool 2 lands at N4, and N4 is USED but NEVER
|
||||
# re-announced (no N5 anywhere). Plus the §layers privacy
|
||||
# invariant: only anchor entries carry an address.
|
||||
# SCENARIO 2 Anchor convection: the rolling window hands each caller the 2
|
||||
# most recent prior callers, produces a REAL connection, rotates
|
||||
# entries out after 2 hand-outs, coordinates a RelayIntroduce when
|
||||
# both ends are live, and refuses a top-up CHEAPLY.
|
||||
# SCENARIO 3 Temp referral slots: allocated ABOVE the mesh cap, carrying no
|
||||
# knowledge, never evicting an established mesh peer, graduating
|
||||
# into a freed mesh slot.
|
||||
# STANDING 0xC0 AnchorRegister is gone; no automatic session relay anywhere.
|
||||
#
|
||||
# Exit code 0 = all checks passed. Logs: /tmp/itsgoin-ctop{1..7}.log
|
||||
set -u
|
||||
cd "$(dirname "$0")/.."
|
||||
BIN=target/debug/itsgoin
|
||||
[ -x "$BIN" ] || { echo "build first: cargo build -p itsgoin-cli"; exit 2; }
|
||||
|
||||
PASS=0; FAIL=0
|
||||
check() { # check <desc> <cmd...>
|
||||
local desc="$1"; shift
|
||||
if "$@" >/dev/null 2>&1; then PASS=$((PASS+1)); echo "PASS: $desc";
|
||||
else FAIL=$((FAIL+1)); echo "FAIL: $desc"; fi
|
||||
}
|
||||
strip_ansi() { sed 's/\x1b\[[0-9;]*m//g' "$1" | grep -v "WARN.*netlink\|WARN.*buffer_tool"; }
|
||||
sq() { sqlite3 "/tmp/itsgoin-ctop$1/itsgoin.db" "$2"; }
|
||||
|
||||
# Is `id` (hex) present in node N's uniques index at `bounce`?
|
||||
at_bounce() { # at_bounce <node> <id_hex> <bounce>
|
||||
[ "$(sq "$1" "SELECT count(*) FROM reachable WHERE lower(hex(reachable_id))='$2' AND bounce=$3")" != "0" ]
|
||||
}
|
||||
anywhere() { # anywhere <node> <id_hex>
|
||||
[ "$(sq "$1" "SELECT count(*) FROM reachable WHERE lower(hex(reachable_id))='$2'")" != "0" ]
|
||||
}
|
||||
# `check` runs its command via `bash -c`, which does not inherit shell functions.
|
||||
export -f strip_ansi sq at_bounce anywhere
|
||||
|
||||
NODES="1 2 3 4 5 6 7"
|
||||
FDS=""
|
||||
cleanup() {
|
||||
for fd in 21 22 23 24 25 26 27; do eval "exec $fd>&-" 2>/dev/null || true; done
|
||||
kill $(pgrep -f 'itsgoin.*itsgoin-ctop') 2>/dev/null
|
||||
rm -f /tmp/itsgoin-ctopcmd*
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Test gates:
|
||||
# - loopback referrals: the convection address filter is publicly-routable-only
|
||||
# (bugs-fixed #8), which would make every 127.0.0.1 referral unusable.
|
||||
# - mesh cap 2: reaches the temp-referral boundary with 4 nodes, not 21.
|
||||
# - no growth loop: on loopback the growth loop collapses any chain into a
|
||||
# full mesh in seconds, and a full mesh has no N2 at all (every peer is a
|
||||
# direct). Topology here is script-controlled on purpose.
|
||||
export ITSGOIN_TEST_ALLOW_LOOPBACK_REFERRALS=1
|
||||
export ITSGOIN_TEST_ALLOW_REGISTRY_GENESIS=1
|
||||
export ITSGOIN_TEST_NO_GROWTH=1
|
||||
export RUST_LOG="${RUST_LOG:-itsgoin_core=debug,info,iroh=warn,swarm_discovery=warn}"
|
||||
|
||||
echo "== setup =="
|
||||
kill $(pgrep -f 'itsgoin.*itsgoin-ctop') 2>/dev/null; sleep 1
|
||||
for i in $NODES; do
|
||||
rm -rf /tmp/itsgoin-ctop$i /tmp/itsgoin-ctopcmd$i /tmp/itsgoin-ctop$i.log
|
||||
mkdir -p /tmp/itsgoin-ctop$i
|
||||
mkfifo /tmp/itsgoin-ctopcmd$i
|
||||
done
|
||||
|
||||
start_node() { # start_node <n> <port> [extra env assignments...]
|
||||
local i="$1" port="$2"; shift 2
|
||||
( exec env "$@" "$BIN" /tmp/itsgoin-ctop$i --bind 127.0.0.1:$port \
|
||||
< /tmp/itsgoin-ctopcmd$i > /tmp/itsgoin-ctop$i.log 2>&1 ) &
|
||||
}
|
||||
|
||||
# Nodes 1-6: chain + convection. Node 7 joins late (fresh index).
|
||||
for i in 1 2 3 4 5 6 7; do start_node $i $((18430 + i)) ITSGOIN_TEST_NO_GROWTH=1; done
|
||||
exec 21>/tmp/itsgoin-ctopcmd1 22>/tmp/itsgoin-ctopcmd2 23>/tmp/itsgoin-ctopcmd3 \
|
||||
24>/tmp/itsgoin-ctopcmd4 25>/tmp/itsgoin-ctopcmd5 26>/tmp/itsgoin-ctopcmd6 \
|
||||
27>/tmp/itsgoin-ctopcmd7
|
||||
sleep 7
|
||||
|
||||
for i in $NODES; do
|
||||
eval "n${i}=\$(strip_ansi /tmp/itsgoin-ctop$i.log | grep -m1 'Node ID:' | awk '{print \$3}')"
|
||||
done
|
||||
echo "n1=$n1 n2=$n2 n3=$n3 n4=$n4 n5=$n5"
|
||||
[ -n "$n5" ] || { echo "FAIL: nodes did not start"; exit 1; }
|
||||
|
||||
##############################################################################
|
||||
echo
|
||||
echo "== scenario 1: two-pool uniques exchange, N4 terminal =="
|
||||
##############################################################################
|
||||
# Chain 1 - 2 - 3 - 4 - 5. Built inward so each hop's initial exchange already
|
||||
# carries the deeper knowledge where possible.
|
||||
echo "connect $n1@127.0.0.1:18431" >&22; sleep 3
|
||||
echo "connect $n2@127.0.0.1:18432" >&23; sleep 3
|
||||
echo "connect $n3@127.0.0.1:18433" >&24; sleep 3
|
||||
echo "connect $n4@127.0.0.1:18434" >&25; sleep 4
|
||||
|
||||
# Drive propagation explicitly: each uniques-pull round moves knowledge one
|
||||
# bounce along the chain. This IS the v0.8 pull — an index exchange, no posts.
|
||||
for round in 1 2 3 4 5; do
|
||||
for fd in 21 22 23 24 25; do echo "uniques-pull" >&$fd; done
|
||||
sleep 3
|
||||
done
|
||||
|
||||
check "uniques pull is an index exchange (peers exchanged, no post transfer)" \
|
||||
bash -c 'grep -q "uniques-pull: exchanged with [1-9]" /tmp/itsgoin-ctop3.log'
|
||||
|
||||
# Pool 1 shifts one bounce deeper per hop, tagged to the reporter.
|
||||
check "node1 holds node3 at N2 (pool 1 shifted one bounce)" at_bounce 1 "$n3" 2
|
||||
check "node1 holds node4 at N3" at_bounce 1 "$n4" 3
|
||||
check "node1 holds node5 at N4 (terminal pool)" at_bounce 1 "$n5" 4
|
||||
check "node5 holds node1 at N4 from the other end of the chain" at_bounce 5 "$n1" 4
|
||||
|
||||
# N4 is USED — address resolution and search read it.
|
||||
check "node1 can look up its N4 entry (used, not merely stored)" \
|
||||
bash -c 'grep -q "N4 via" <(echo "uniques '"$n5"'" >&21; sleep 3; tail -40 /tmp/itsgoin-ctop1.log)'
|
||||
|
||||
# §layers privacy invariant: ONLY anchor entries carry an address.
|
||||
check "no non-anchor index row carries an address (on any node)" \
|
||||
bash -c 'for i in 1 2 3 4 5; do
|
||||
c=$(sqlite3 /tmp/itsgoin-ctop$i/itsgoin.db \
|
||||
"SELECT count(*) FROM reachable WHERE is_anchor=0 AND addresses NOT IN ('"''"',\"[]\")");
|
||||
[ "$c" = "0" ] || exit 1; done'
|
||||
|
||||
# Per-reporter dedup: one row per (reporter, id) — an ID can never sit at two
|
||||
# depths from the same reporter.
|
||||
check "one index row per (reporter, id) — no duplicate depths" \
|
||||
bash -c 'for i in 1 2 3 4 5; do
|
||||
d=$(sqlite3 /tmp/itsgoin-ctop$i/itsgoin.db \
|
||||
"SELECT count(*) FROM (SELECT reporter_node_id, reachable_id FROM reachable
|
||||
GROUP BY reporter_node_id, reachable_id HAVING count(*) > 1)");
|
||||
[ "$d" = "0" ] || exit 1; done'
|
||||
|
||||
# Nothing is stored past bounce 4. There is no N5 table and no N5 row.
|
||||
check "stored horizon never exceeds 4 bounces" \
|
||||
bash -c 'for i in 1 2 3 4 5; do
|
||||
m=$(sqlite3 /tmp/itsgoin-ctop$i/itsgoin.db "SELECT ifnull(max(bounce),0) FROM reachable");
|
||||
[ "$m" -le 4 ] || exit 1; done'
|
||||
|
||||
# THE KEY ASSERTION: a fresh node joining node1 must NOT learn node5. node1
|
||||
# knows node5 only at N4, and the terminal pool is never re-announced.
|
||||
echo "connect $n1@127.0.0.1:18431" >&27; sleep 4
|
||||
for round in 1 2 3; do echo "uniques-pull" >&27; echo "uniques-pull" >&21; sleep 3; done
|
||||
check "N4 is never re-announced: node7 never learns node5" \
|
||||
bash -c '! /usr/bin/env bash -c "[ \"\$(sqlite3 /tmp/itsgoin-ctop7/itsgoin.db \
|
||||
\"SELECT count(*) FROM reachable WHERE lower(hex(reachable_id))=\\\"'"$n5"'\\\"\")\" != \"0\" ]"'
|
||||
check "node7 DID learn node2 (the exchange itself works)" anywhere 7 "$n2"
|
||||
|
||||
##############################################################################
|
||||
echo
|
||||
echo "== scenario 2: anchor convection =="
|
||||
##############################################################################
|
||||
# Node 6 plays the anchor: directly dialable, and it is the one every caller
|
||||
# asks. Callers are node1 and node5 — the two ENDS of the chain, so they have
|
||||
# no path to each other and a referral that lands is unambiguous.
|
||||
echo "connect $n6@127.0.0.1:18436" >&21; sleep 3
|
||||
echo "convect $n6" >&21; sleep 5 # node1 admitted; window was empty
|
||||
echo "connect $n6@127.0.0.1:18436" >&25; sleep 3
|
||||
echo "convect $n6" >&25; sleep 10 # node5 is referred to node1
|
||||
|
||||
check "anchor serves referrals out of the rolling window" \
|
||||
bash -c 'grep -q "Convection: serving referrals" /tmp/itsgoin-ctop6.log'
|
||||
check "the anchor admitted callers to the window without any registration msg" \
|
||||
bash -c 'grep -q "Convection: admitted caller to rolling window" /tmp/itsgoin-ctop6.log'
|
||||
check "the anchor referred the PRIOR caller to the next one" \
|
||||
bash -c 'grep "Convection: referring peer" /tmp/itsgoin-ctop6.log | grep -q "'"$n1"'"'
|
||||
check "a convection referral produced a real peer connection" \
|
||||
bash -c 'grep -q "Convection: connected to referred peer\|Convection: connected via introduction" \
|
||||
/tmp/itsgoin-ctop5.log'
|
||||
check "node5 is now actually mesh-connected to node1 (chain ends joined)" \
|
||||
bash -c 'echo "connections" >&25; sleep 3; strip_ansi /tmp/itsgoin-ctop5.log | tail -20 | grep -q "'"${n1:0:12}"'"'
|
||||
|
||||
# Both ends live on the anchor => coordinate an introduction, not a cold address.
|
||||
check "anchor coordinates RelayIntroduce when both ends are live" \
|
||||
bash -c 'grep -q "Convection: coordinated introduction" /tmp/itsgoin-ctop6.log'
|
||||
|
||||
# Rotation: an entry is handed to exactly 2 callers, then leaves the window.
|
||||
echo "convect $n6" >&24; sleep 6
|
||||
echo "convect $n6" >&23; sleep 6
|
||||
echo "convect $n6" >&27; sleep 6
|
||||
check "each window entry is handed out at most twice, then rotates out" \
|
||||
bash -c 'n=$(grep "Convection: referring peer" /tmp/itsgoin-ctop6.log | grep -c "referred=\"\?'"$n1"'"); \
|
||||
[ "$n" -le 2 ]'
|
||||
check "later callers are served the more RECENT callers" \
|
||||
bash -c 'grep "Convection: referring peer" /tmp/itsgoin-ctop6.log | tail -3 | grep -q "'"$n5"'\|'"$n4"'"'
|
||||
|
||||
# CHEAP REFUSAL: node3 is an interior chain node (2 mesh peers) so its request
|
||||
# is TOP-UP class. It asks node5, whose own convection window is empty — there
|
||||
# is nothing fresh to circulate, so the refusal must be one message, not a
|
||||
# burned timeout.
|
||||
echo "connect $n5@127.0.0.1:18435" >&23; sleep 4
|
||||
echo "convect $n5" >&23; sleep 6
|
||||
check "top-up against an empty window is refused" \
|
||||
bash -c 'grep -q "convection: refused" /tmp/itsgoin-ctop3.log'
|
||||
check "the refusal is CHEAP (single message, well under a timeout)" \
|
||||
bash -c 'ms=$(grep -o "convection: refused in [0-9]*ms" /tmp/itsgoin-ctop3.log | tail -1 |
|
||||
grep -o "[0-9]*" | head -1); [ -n "$ms" ] && [ "$ms" -lt 3000 ]'
|
||||
check "entry class is served even where top-up was refused" \
|
||||
bash -c 'grep "Convection: serving referrals" /tmp/itsgoin-ctop6.log | grep -q entry'
|
||||
check "the refusing anchor lands in the caller's penalty box (refusal feedback)" \
|
||||
bash -c 'echo "convection" >&23; sleep 3; strip_ansi /tmp/itsgoin-ctop3.log | tail -20 |
|
||||
grep -oE "anchor_bias [0-9.]+" | tail -1 | awk "{ exit !(\$2 < 1.0) }"'
|
||||
|
||||
##############################################################################
|
||||
echo
|
||||
echo "== scenario 3: temp referral slots (mesh cap 2) =="
|
||||
##############################################################################
|
||||
# Restart nodes 1-4 with a mesh cap of 2 so the temp-referral band is reachable.
|
||||
for fd in 21 22 23 24 25 26 27; do echo "quit" >&$fd 2>/dev/null; done
|
||||
sleep 3
|
||||
kill $(pgrep -f 'itsgoin.*itsgoin-ctop') 2>/dev/null; sleep 2
|
||||
for fd in 21 22 23 24 25 26 27; do eval "exec $fd>&-" 2>/dev/null || true; done
|
||||
|
||||
for i in 1 2 3 4; do
|
||||
rm -rf /tmp/itsgoin-ctop$i /tmp/itsgoin-ctopcmd$i /tmp/itsgoin-ctop$i.log
|
||||
mkdir -p /tmp/itsgoin-ctop$i; mkfifo /tmp/itsgoin-ctopcmd$i
|
||||
start_node $i $((18440 + i)) ITSGOIN_TEST_MESH_SLOTS=2 \
|
||||
ITSGOIN_TEST_ALLOW_LOOPBACK_REFERRALS=1 ITSGOIN_TEST_NO_GROWTH=1
|
||||
done
|
||||
exec 21>/tmp/itsgoin-ctopcmd1 22>/tmp/itsgoin-ctopcmd2 23>/tmp/itsgoin-ctopcmd3 \
|
||||
24>/tmp/itsgoin-ctopcmd4
|
||||
sleep 7
|
||||
for i in 1 2 3 4; do
|
||||
eval "m${i}=\$(strip_ansi /tmp/itsgoin-ctop$i.log | grep -m1 'Node ID:' | awk '{print \$3}')"
|
||||
done
|
||||
|
||||
# Fill node1's 2 mesh slots, then send a third peer at it.
|
||||
echo "connect $m1@127.0.0.1:18441" >&22; sleep 4
|
||||
echo "connect $m1@127.0.0.1:18441" >&23; sleep 4
|
||||
echo "slots" >&21; sleep 3
|
||||
check "mesh pool fills to the cap" \
|
||||
bash -c 'strip_ansi /tmp/itsgoin-ctop1.log | grep -q "mesh 2/2"'
|
||||
|
||||
echo "connect $m1@127.0.0.1:18441" >&24; sleep 5
|
||||
echo "slots" >&21; sleep 3
|
||||
check "the 4th peer lands in a temp referral slot ABOVE the cap" \
|
||||
bash -c 'strip_ansi /tmp/itsgoin-ctop1.log | tail -30 | grep -q "mesh 2/2 temp-referral 1/"'
|
||||
check "an established mesh peer was never evicted to make room" \
|
||||
bash -c 'strip_ansi /tmp/itsgoin-ctop1.log | tail -30 | grep -q "mesh 2/2"'
|
||||
check "the temp slot carries NO knowledge (never a reporter in our index)" \
|
||||
bash -c '[ "$(sqlite3 /tmp/itsgoin-ctop1/itsgoin.db \
|
||||
"SELECT count(*) FROM reachable WHERE lower(hex(reporter_node_id))=\"'"$m4"'\"")" = "0" ]'
|
||||
check "and a temp peer is never written to mesh_peers" \
|
||||
bash -c '[ "$(sqlite3 /tmp/itsgoin-ctop1/itsgoin.db \
|
||||
"SELECT count(*) FROM mesh_peers WHERE lower(hex(node_id))=\"'"$m4"'\"")" = "0" ]'
|
||||
|
||||
# Free a mesh slot: the temp peer graduates into it. The wait covers the mesh
|
||||
# keepalive interval (30s) — an abruptly-gone QUIC peer is noticed when the
|
||||
# next keepalive write fails, not instantly.
|
||||
echo "quit" >&22; sleep 45
|
||||
echo "slots" >&21; sleep 5
|
||||
check "the temp referral slot GRADUATES into the freed mesh slot" \
|
||||
bash -c 'strip_ansi /tmp/itsgoin-ctop1.log | tail -20 | grep -q "temp-referral 0/"'
|
||||
check "the graduated peer now exchanges knowledge (written to mesh_peers)" \
|
||||
bash -c '[ "$(sqlite3 /tmp/itsgoin-ctop1/itsgoin.db \
|
||||
"SELECT count(*) FROM mesh_peers WHERE lower(hex(node_id))=\"'"$m4"'\"")" != "0" ]'
|
||||
|
||||
##############################################################################
|
||||
echo
|
||||
echo "== standing assertions =="
|
||||
##############################################################################
|
||||
check "0xC0 AnchorRegister is gone from the tree" \
|
||||
bash -c '! grep -rn "AnchorRegister" crates/ --include=*.rs'
|
||||
check "no automatic session relay anywhere in the convection path" \
|
||||
bash -c '! grep -riE "session relay" /tmp/itsgoin-ctop*.log'
|
||||
check "no registration cycle writes known_anchors (bootstrap cache only)" \
|
||||
bash -c '! grep -rn "start_anchor_register_cycle" crates/ --include=*.rs'
|
||||
|
||||
echo
|
||||
echo "== results: $PASS passed, $FAIL failed =="
|
||||
[ "$FAIL" -eq 0 ]
|
||||
|
|
@ -129,11 +129,11 @@
|
|||
<table>
|
||||
<tr><th>Cycle</th><th>Interval</th><th>Purpose</th><th>Status</th></tr>
|
||||
<tr><td>Update cadence scheduler</td><td>Descending scale (minutes → days) per author, keyed on freshness × relationship tier, jittered</td><td>CDN update checks + uniques-list refresh; timing shaped to look like uniform network overhead. See <a href="#keep-alive">update cadence</a>.</td><td><span class="badge badge-rework">Rework</span> — today: 60s pull tick with a 4h stale-author threshold</td></tr>
|
||||
<tr><td>Routing diff</td><td>120s (2 min)</td><td>Announce network-knowledge changes to mesh peers (target: uniques-list diffs; today: N1/N2 share-list diffs)</td><td><span class="badge badge-rework">Rework</span></td></tr>
|
||||
<tr><td>Routing diff</td><td>120s (2 min)</td><td>Announce network knowledge to mesh peers as a full two-pool uniques snapshot; skipped when the content digest is unchanged</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Rebalance</td><td>600s (10 min)</td><td>Clean dead connections, signal growth</td><td><span class="badge badge-rework">Rework</span> — preferred-peer Priority 0 deleted (Iteration B); remaining priorities still target the wide model</td></tr>
|
||||
<tr><td>Growth loop</td><td>Reactive (signal-driven on knowledge receipt)</td><td>Fill mesh slots toward 20 with the most diverse candidates; secondary peer-finding path alongside convection</td><td><span class="badge badge-rework">Rework</span> — mechanism is live; targets 101 today and only counts local slots</td></tr>
|
||||
<tr><td>Convection trigger</td><td>Stochastic, per-disconnect</td><td>On each mesh disconnect, random choice: do nothing / ask a random known anchor for an introduction / ask a mesh peer. See <a href="#anchors">anchors</a>.</td><td><span class="badge badge-planned">Planned</span></td></tr>
|
||||
<tr><td>Recovery loop</td><td>Reactive (mesh drops below 2)</td><td>Emergency reconnect: mine retained uniques pools for anchor addresses, then <code>known_anchors</code> cache, then any connected anchor peers</td><td><span class="badge badge-rework">Rework</span> — today's loop still sends <code>AnchorRegister</code></td></tr>
|
||||
<tr><td>Growth loop</td><td>Reactive (signal-driven on knowledge receipt)</td><td>Fill the single mesh pool toward 20 with the most diverse candidates (fewest reporters wins, shallower bounce breaks ties); secondary peer-finding path alongside convection</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Convection trigger</td><td>Stochastic, per-disconnect</td><td>On each <em>mesh</em> disconnect (temp referral expiry does not count), random choice: do nothing / ask a random known anchor for an introduction / ask a mesh peer. See <a href="#anchors">anchors</a>.</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Recovery loop</td><td>Reactive (mesh drops below 2)</td><td>Emergency reconnect: mine retained uniques pools for anchor addresses, then <code>known_anchors</code> cache, then any connected anchor peers. Non-stochastic — below 2 it always acts.</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Anchor reachability watcher</td><td>Event-driven (portmapper watch channel)</td><td>Clears anchor candidacy after >5min of no port mapping; restores it when the mapping comes back. Network roams self-heal without restart. Mobile never auto-anchors.</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
</table>
|
||||
<div class="note">
|
||||
|
|
@ -163,30 +163,32 @@
|
|||
<p>A port mapping or public IPv6 address is a claim, not proof. The node selects a witness from its network knowledge that is NOT a current peer — a genuine stranger with no prior connection, no cached address, no warm path — reached via the peer that reported it (<code>AnchorProbeRequest</code>, <code>0xC3</code>). The witness performs a <strong>raw cold QUIC connect</strong> to the candidate's external address (15s timeout), deliberately skipping the entire resolution cascade — hole punch, introduction, or relay success would be a false positive.</p>
|
||||
<p><strong>Asymmetric return path</strong>: on success the witness reports back over its fresh direct connection (<code>AnchorProbeResult</code>, <code>0xC4</code>); on failure there is by definition no direct path, so the result routes back through the reporting peer.</p>
|
||||
<div class="note">
|
||||
<strong>v0.8 change</strong>: Probe scheduling currently rides the anchor register cycle, which is removed. The target triggers re-probes from the reachability watcher (on mapping acquisition/restoration) and on failed inbound connections.
|
||||
<strong>v0.8 (Iteration C)</strong>: the anchor register cycle the probe used to ride is gone. The probe now rides the <strong>convection loop's 10-minute maintenance tick</strong>. Note the consequence: <code>is_anchor_candidate</code> no longer <em>requires</em> a successful probe — candidacy is reachability + opt-in, and the probe is confirmation that runs alongside, not a gate in front. Triggering re-probes from the reachability watcher (on mapping acquisition/restoration) and on failed inbound connections remains <span class="badge badge-planned">Planned</span>.
|
||||
</div>
|
||||
|
||||
<h3>Connection convection <span class="badge badge-rework">Rework</span></h3>
|
||||
<h3>Connection convection <span class="badge badge-complete">Implemented</span></h3>
|
||||
<p>The anchor's referral service is a <strong>rotating referral chain</strong> — "connection convection." The anchor keeps only a small rolling window of recent callers; there is no peer database and no persistent registration.</p>
|
||||
<ol style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li>A caller connects and announces "I'm joining / I need peers."</li>
|
||||
<li>The anchor returns the addresses of the <strong>2 most recent viable prior callers</strong>.</li>
|
||||
<li>The caller's own address is handed out to the <strong>next 2 callers</strong>, then rotates out of the window.</li>
|
||||
</ol>
|
||||
<p><strong>Window parameters</strong> (previously open, now decided in code): the window holds <strong>16 entries</strong>; each entry is handed to <strong>2</strong> callers before rotating out; an entry older than <strong>15 minutes</strong> is not handed out at all — a stale address is worse than none, because it costs the next caller a hole-punch timeout. A caller with no routable address is admitted to nothing. Self-reported addresses are a caller's unverified claim: at most 2 are kept, and when the anchor has an observed address for the caller (essentially always over QUIC) self-reported entries are kept only if they share its IP — a different port is the legitimate endpoint-dependent-mapping case, a different host is not. Only the <em>observed</em> address is ever named in an anchor-initiated introduction; self-reported ones ride the referral payload as a hint the referred peer may try itself.</p>
|
||||
<p><strong>Sparse-window supplement</strong>: a young anchor with an empty window may fill referrals from its own live mesh peers — but those peers never called and never offered themselves for redistribution, so the supplement is consent-shaped. It discloses <strong>no address</strong> (the referral carries an empty address list and relies on the coordinated introduction, so the target discloses its own addresses on its own terms), it is used only for peers the anchor can actually introduce, and each supplemented peer carries its own hand-out budget (2 per 15 minutes). Without that budget, unconditional entry-class service plus a reshuffle would be an enumeration oracle over the anchor's entire mesh membership.</p>
|
||||
<p><strong>Request classes</strong>: convection requests carry a one-bit class — <em>entry</em> (bootstrap or recovery: fewer than 2 connections) versus <em>top-up</em> (growth with a working mesh). Anchors always serve entry; top-ups are served capacity-permitting and refused <em>cheaply</em> (a single refusal message, no timeout burned), so deprioritizing the non-needy is harmless. The refusal doubles as the load signal the adaptive weights consume. Top-ups are not freeloading: every caller freshens the anchor's referral window, so top-up traffic is the convection medium — a network where nobody tops up hands its next genuine joiner stale referrals.</p>
|
||||
<p><strong>Referral quality</strong>: the anchor prefers callers it is still connected to — and when both the referred peer and the caller are live on the anchor, it coordinates a <code>RelayIntroduce</code> hole-punch introduction between them instead of handing out a cold address. Referred peers land in the caller's <a href="#connections">temporary referral slots</a> (above the 20-peer mesh cap, no knowledge exchange) and either graduate into mesh slots or expire.</p>
|
||||
<p><strong>Trigger — stochastic, per-disconnect</strong>: each time a mesh peer disconnects, the node randomly chooses one of three actions: <em>do nothing</em>, <em>request an introduction from a random known anchor</em> (mined from the uniques pools — see bookkeeping below), or <em>request an introduction via a connected mesh peer</em> (the growth loop's diversity-scored path — see <a href="#lifecycle">connection lifecycle</a>). The randomness itself de-synchronizes the network's reaction to shared disconnection events — no jitter timer, no threshold. Because every caller becomes a referral for the next callers, ordinary churn keeps the chain circulating: convection. With IPv6 making anchors abundant (an estimated 20–30% of nodes), anchors rebuild the mesh among themselves and with their callers, absorbing even large disconnection events. The action-choice weights are <strong>adaptive, not fixed</strong>. The anchor ratio is locally observable — anchor entries are the ones carrying addresses in the pools, so anchor density among known uniques is a free statistic needing no census — and it sets the prior: anchor-rich networks lean anchor-heavy (which also keeps convection windows fresh), anchor-poor lean mesh-ward. Refusals then steer: an anchor refusing a growth request shifts the caller's weights toward mesh introductions and other anchors, drifting back as requests succeed. How many operators will announce addresses versus hide them (or lack public ones) is unknowable before deployment — the adaptive weights are designed to find that ratio rather than assume it. (The node-ID–to–persona ratio stays deliberately unknowable; this statistic never needs it.)</p>
|
||||
<p><strong>Referral quality</strong>: the anchor prefers callers it is still connected to — and when both the referred peer and the caller are live on the anchor, it coordinates a <code>RelayIntroduce</code> hole-punch introduction between them instead of handing out a cold address. A referred peer lands in whatever slot class is free: a <strong>mesh slot first</strong>, and only the <a href="#connections">temporary referral band</a> above the cap when the mesh is full — from there it graduates into a freed mesh slot or expires.</p>
|
||||
<p><strong>Trigger — stochastic, per-disconnect</strong> <span class="badge badge-complete">Implemented</span>: each time a <strong>mesh</strong> peer disconnects — a temp referral slot expiring is not mesh churn and does not roll the dice — the node randomly chooses one of three actions: <em>do nothing</em>, <em>request an introduction from a random known anchor</em> (mined from the uniques pools — see bookkeeping below), or <em>request an introduction via a connected mesh peer</em> (the growth loop's diversity-scored path — see <a href="#lifecycle">connection lifecycle</a>). The randomness itself de-synchronizes the network's reaction to shared disconnection events — no jitter timer, no threshold. Because every caller becomes a referral for the next callers, ordinary churn keeps the chain circulating: convection. With IPv6 making anchors abundant (an estimated 20–30% of nodes), anchors rebuild the mesh among themselves and with their callers, absorbing even large disconnection events. The action-choice weights are <strong>adaptive, not fixed</strong>. The anchor ratio is locally observable — anchor entries are the ones carrying addresses in the pools, so anchor density among known uniques is a free statistic needing no census — and it sets the prior: anchor-rich networks lean anchor-heavy (which also keeps convection windows fresh), anchor-poor lean mesh-ward. Refusals then steer: an anchor refusing a growth request shifts the caller's weights toward mesh introductions and other anchors, drifting back as requests succeed. How many operators will announce addresses versus hide them (or lack public ones) is unknowable before deployment — the adaptive weights are designed to find that ratio rather than assume it. (The node-ID–to–persona ratio stays deliberately unknowable; this statistic never needs it.)</p>
|
||||
<div class="note">
|
||||
Current code implements the older registration model this replaces: an in-memory referral list with tiered use caps (3 uses under 50 entries, fewer as the list grows), a 2-minute disconnect grace, and least-used-first selection. The convection window replaces all of it.
|
||||
<strong>Done (Iteration C)</strong>: the older registration model is gone — <code>0xC0 AnchorRegister</code> and the periodic register cycle are removed along with the in-memory referral list, its tiered use caps and least-used-first selection. The rolling window replaced all of it, and recovery (below 2 mesh peers) stays non-stochastic: it always acts immediately.
|
||||
</div>
|
||||
|
||||
<h3>Anchor bookkeeping <span class="badge badge-rework">Rework</span></h3>
|
||||
<h3>Anchor bookkeeping <span class="badge badge-complete">Implemented</span></h3>
|
||||
<p><strong>Target: the uniques pools ARE the anchor directory.</strong> Anchor entries carry addresses in the N0–N3 pools (see <a href="#layers">Network Knowledge</a>), so a node needing a connection mines its retained pools for a random anchor-flagged entry. Pool knowledge is <em>overwritten memory</em>: a slot's contributed entries are wiped only when a new handshake replaces them — so a node knocked down from 20 peers to 4 still holds 16 dead peers' pools full of anchor addresses to reconnect through. No separately curated anchor list is needed.</p>
|
||||
<p>The shipped <code>known_anchors</code> table (<code>node_id</code>, <code>addresses</code>, <code>last_seen</code>, <code>success_count</code> — a count of successful connects accreted during v0.7.x bootstrap hardening, ranked descending, capped at 5) survives as the <strong>bootstrap cache</strong>. Its careful ranking solved an anchor-scarcity problem that pool-mined abundance retires.</p>
|
||||
<p>The shipped <code>known_anchors</code> table survives as the <strong>bootstrap cache</strong>, capped at 32 entries and ordered by <code>last_seen_ms</code> (freshest first). Its old <code>success_count</code> ranking solved an anchor-scarcity problem that pool-mined abundance retires. It is written <strong>only for anchors we have actually connected to</strong> — never from a peer's announce — which is both what keeps gossip from re-inflating a table the ruling deliberately demoted, and what makes its recency ordering mean something again (a cache refreshed by hearsay would evict the DNS-resolved bootstrap anchor first).</p>
|
||||
<p>Selection order:</p>
|
||||
<ol style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>Uniques pools</strong> <span class="badge badge-planned">Planned</span> — random anchor-flagged entry, including entries from disconnected peers' retained pools.</li>
|
||||
<li><strong><code>known_anchors</code> bootstrap cache</strong> <span class="badge badge-complete">Implemented</span> — ordered by <code>success_count</code> descending. Failed probes to entries not seen for 3 days are deleted on the spot (self-healing against stale data dirs).</li>
|
||||
<li><strong>Uniques pools</strong> <span class="badge badge-complete">Implemented</span> — anchor-flagged entries, shallowest and freshest first, including entries from disconnected peers' retained pools.</li>
|
||||
<li><strong><code>known_anchors</code> bootstrap cache</strong> <span class="badge badge-complete">Implemented</span> — ordered by <code>last_seen_ms</code> descending. Failed probes to entries not seen for 3 days are deleted on the spot (self-healing against stale data dirs).</li>
|
||||
<li><strong>Hardcoded default anchor(s)</strong> — only if everything above is empty or exhausted. A brand-new node hits the hardcoded anchor once on first bootstrap, populates its caches from that session, and the hardcoded list recedes to pure fallback.</li>
|
||||
</ol>
|
||||
|
||||
|
|
@ -194,7 +196,7 @@
|
|||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>Bootstrap</strong>: First startup with an empty peers table — batched probing as described in <a href="#identity">Identity & Bootstrap</a>, then a convection referral request.</li>
|
||||
<li><strong>Recovery</strong>: When the mesh drops below 2 connections. Reconnect via anchor addresses mined from the retained uniques pools, then the <code>known_anchors</code> cache, then any still-connected anchor peers.</li>
|
||||
<li><strong>Convection top-up</strong> <span class="badge badge-planned">Planned</span>: the per-disconnect stochastic action described above. This is the ONLY ongoing anchor contact — there is no periodic registration.</li>
|
||||
<li><strong>Convection top-up</strong> <span class="badge badge-complete">Implemented</span>: the per-disconnect stochastic action described above. This is the ONLY ongoing anchor contact — there is no periodic registration.</li>
|
||||
<li><strong>itsgoin.net node</strong>: A permanent, well-connected ItsGoin node runs on itsgoin.net as part of the share-link redirect infrastructure (see <a href="#share-links">share links</a>). It participates as a standard anchor — it bootstraps new nodes and serves convection referrals — and is not special-cased in the protocol. Its value comes from permanent uptime, not from any privileged role.</li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -203,12 +205,12 @@
|
|||
</section>
|
||||
<!-- Connections & Slot Architecture -->
|
||||
<section id="connections">
|
||||
<h2>4. Connections & Slot Architecture <span class="badge badge-rework">Rework</span></h2>
|
||||
<h2>4. Connections & Slot Architecture <span class="badge badge-complete">Implemented</span></h2>
|
||||
|
||||
<h3>Connection types</h3>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>Mesh connection</strong> — long-lived routing slot in a single ~20-slot pool. Structural backbone for search and propagation; participates in the N1–N4 knowledge system's uniques announce (two pools: forwardable N0–N2 + terminal N3 — see <a href="#layers">Network Knowledge</a>). DB table: <code>mesh_peers</code>.</li>
|
||||
<li><strong>Temp referral connection</strong> <span class="badge badge-planned">Planned</span> — short-lived slot above the mesh cap, created by anchor convection or peer introductions. Carries NO knowledge exchange; exists purely to facilitate introductions. Graduates into a mesh slot or expires.</li>
|
||||
<li><strong>Temp referral connection</strong> <span class="badge badge-complete">Implemented</span> — short-lived slot above the mesh cap, created by anchor convection or peer introductions. Carries NO knowledge exchange; exists purely to facilitate introductions. Graduates into a mesh slot or expires.</li>
|
||||
<li><strong>Cadence-driven session</strong> — connection to a social or file-layer peer that isn't in the mesh pool, opened on demand by the update-cadence cycle and closed when idle (see <a href="#keep-alive">Update Cadence & Keep-Alive</a>). There is no standing keep-alive pool — only the shaped check cycle recurs.</li>
|
||||
<li><strong>Session connection</strong> — short-lived, held open for active interaction (DM conversations, group activity, anchor matchmaking). Tracks <code>remote_addr</code> so a relay can inject observed addresses during introductions.</li>
|
||||
<li><strong>Ephemeral connection</strong> — single request/response, no slot allocation.</li>
|
||||
|
|
@ -217,8 +219,8 @@
|
|||
<h3>Slot architecture</h3>
|
||||
<table>
|
||||
<tr><th>Slot kind</th><th>Desktop</th><th>Mobile</th><th>Status</th><th>Purpose</th></tr>
|
||||
<tr><td><strong>Mesh</strong> (single pool)</td><td><strong>20</strong></td><td><strong>15</strong></td><td><span class="badge badge-rework">Rework</span></td><td>Long-lived routing backbone; every slot equal, filled by growth loop + inbound</td></tr>
|
||||
<tr><td>Temp referral</td><td>+4–+10</td><td>+4–+10</td><td><span class="badge badge-planned">Planned</span></td><td>Introduction facilitation above the cap; no knowledge exchange; graduate or expire</td></tr>
|
||||
<tr><td><strong>Mesh</strong> (single pool)</td><td><strong>20</strong></td><td><strong>15</strong></td><td><span class="badge badge-complete">Implemented</span></td><td>Long-lived routing backbone; every slot equal, filled by growth loop + inbound</td></tr>
|
||||
<tr><td>Temp referral</td><td>+10</td><td>+4</td><td><span class="badge badge-complete">Implemented</span></td><td>Introduction facilitation above the cap; no knowledge exchange; graduate or expire. The +4..+10 ruling is a range; these are the shipped points in it.</td></tr>
|
||||
<tr><td>Sessions (interactive)</td><td>20</td><td>5</td><td><span class="badge badge-complete">Implemented</span></td><td>Active DM, group interaction, anchor matchmaking. At capacity, the oldest idle session is evicted for a new one.</td></tr>
|
||||
<tr><td>Relay pipes</td><td>10</td><td>2</td><td><span class="badge badge-complete">Implemented</span></td><td>Session-relay byte pipes. Serving others is opt-in and default OFF (see <a href="#relay">Relay & NAT Traversal</a>).</td></tr>
|
||||
</table>
|
||||
|
|
@ -230,12 +232,12 @@
|
|||
</div>
|
||||
|
||||
<h3>MeshConnection struct</h3>
|
||||
<p>Each mesh connection tracks: <code>node_id</code>, <code>connection</code> (QUIC), <code>remote_addr</code> (captured from Incoming before accept), <code>last_activity</code> (AtomicU64, updated on every stream accept), <code>connected_at</code>. The per-slot <code>slot_kind</code> discriminator (Preferred/Local/Wide) goes away with the pool merge; a boolean or small enum distinguishes mesh slots from temp referral slots.</p>
|
||||
<p>Each mesh connection tracks: <code>node_id</code>, <code>connection</code> (QUIC), <code>remote_addr</code> (captured from Incoming before accept), <code>last_activity</code> (AtomicU64, updated on every stream accept), <code>connected_at</code>. The per-slot <code>slot_kind</code> discriminator (Preferred/Local/Wide) is gone with the pool merge; a small enum distinguishes <code>Mesh</code> from <code>TempReferral { expires_at_ms }</code>. Allocation is mesh-first, then the temp band, then refuse — nothing in the rule can evict anything, so an established mesh peer is never displaced by an arriving stranger.</p>
|
||||
|
||||
<h3>Temp referral slots <span class="badge badge-planned">Planned</span></h3>
|
||||
<p>Nodes hold <strong>+4 to +10 temporary connections above the 20-slot mesh cap</strong>. These exist so that introductions (anchor convection referrals, growth-loop introductions) never fail for want of a free slot, without paying the knowledge-state cost of a full mesh peer:</p>
|
||||
<h3>Temp referral slots <span class="badge badge-complete">Implemented</span></h3>
|
||||
<p>Nodes hold temporary connections <strong>above</strong> the mesh cap — <strong>+10 on desktop, +4 on mobile</strong>, within the +4..+10 ruling. These exist so that introductions (anchor convection referrals, growth-loop introductions) never fail for want of a free slot, without paying the knowledge-state cost of a full mesh peer:</p>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>No knowledge exchange</strong> — temp referral peers do not participate in the uniques announce exchange. Lightweight by construction.</li>
|
||||
<li><strong>No knowledge exchange</strong> — temp referral peers do not participate in the uniques announce exchange, in either direction, and the boundary is structural: a temp peer is never written to <code>mesh_peers</code>, which is the table the announce builder reads. The knowledge gate covers the <em>whole</em> initial exchange, not just the uniques pools: a temp referral peer is also sent an empty peer-address list, since up to 10 of our mesh peers with their addresses is knowledge by any reading. On the wire "no knowledge" is an explicit <code>null</code>, not empty vectors, so the peer can tell "nothing to share" from "not sharing".</li>
|
||||
<li><strong>Graduate or expire</strong> — when a mesh slot is free, a temp connection can be promoted into it (and only then joins knowledge exchange). Otherwise it expires after its introduction purpose is served.</li>
|
||||
<li>Graduation requires a free mesh slot — a temp referral never evicts an established mesh peer.</li>
|
||||
</ul>
|
||||
|
|
@ -249,8 +251,8 @@
|
|||
<li>Full session/ephemeral interaction still works — messages, probes, routing participation; blacklisted nodes never consume each other's mesh slots</li>
|
||||
</ul>
|
||||
|
||||
<h3><code>--max-mesh <n></code> CLI flag <span class="badge badge-planned">Planned</span></h3>
|
||||
<p>Topology control for testing. Forces a node to cap its mesh connections below the 20 default, keeping it permanently in N2 of other nodes. <code>--max-mesh 0</code> = pure N2 participant (free rider — consumes routing knowledge without carrying mesh load); <code>--max-mesh 20</code> = default full behavior. Reuses <code>RefuseRedirect</code> (<code>0x05</code>) — no new protocol machinery. <strong>Testing affordance only.</strong></p>
|
||||
<h3>Mesh-cap override <span class="badge badge-complete">Implemented</span></h3>
|
||||
<p>Topology control for testing, shipped as the <code>ITSGOIN_TEST_MESH_SLOTS</code> environment override rather than the originally-planned <code>--max-mesh <n></code> CLI flag. It caps a node's mesh pool below the 20 default so the integration suite can reach the temp-referral boundary with four nodes instead of twenty-one, and so a node can be held permanently in other nodes' N2. Read once into a <code>OnceLock</code> at first use — the cap cannot change under a running node. Refusals reuse <code>RefuseRedirect</code> (<code>0x05</code>) — no new protocol machinery. <strong>Testing affordance only.</strong></p>
|
||||
|
||||
<h3>Keepalive <span class="badge badge-complete">Implemented</span></h3>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
|
|
@ -277,7 +279,7 @@
|
|||
<strong>v0.8 change</strong>: growth targets the whole 20-slot pool. (Today the loop stops when the <em>Local</em> sub-pool is full and Wide slots only fill from inbound — a casualty of the old taxonomy that dies with it.) <span class="badge badge-rework">Rework</span>
|
||||
</div>
|
||||
<div class="note">
|
||||
<strong>v0.8 change — stochastic growth</strong> <span class="badge badge-planned">Planned</span>: threshold triggers are replaced by a per-disconnect random choice among {do nothing | convection request to a random known anchor | introduction via a connected mesh peer}. The mesh path uses the diversity-scored candidate machinery below; the anchor path defers candidate choice to the anchor's convection window. Randomness replaces jitter for de-synchronization (see <a href="#anchors">Anchors & Connection Convection</a>).
|
||||
<strong>v0.8 — stochastic growth</strong> <span class="badge badge-complete">Implemented</span>: threshold triggers are replaced by a per-disconnect random choice among {do nothing | convection request to a random known anchor | introduction via a connected mesh peer}. The mesh path uses the diversity-scored candidate machinery below; the anchor path defers candidate choice to the anchor's convection window. Randomness replaces jitter for de-synchronization (see <a href="#anchors">Anchors & Connection Convection</a>).
|
||||
</div>
|
||||
|
||||
<p><strong>Candidate selection</strong> (N2 diversity scoring) <span class="badge badge-complete">Implemented</span>:</p>
|
||||
|
|
@ -313,7 +315,7 @@
|
|||
<p><strong>Trigger</strong>: fires when the mesh drops below 2 connections — the one non-stochastic case: at the edge of isolation the node always acts, immediately.</p>
|
||||
<ol style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li>Debounce 2 seconds (let cascading disconnects settle), coalesce queued signals</li>
|
||||
<li>Gather anchors: mine the retained uniques pools for anchor-flagged addresses (<span class="badge badge-planned">Planned</span> — disconnected peers' pools survive as overwritten memory until new handshakes replace them), then the <code>known_anchors</code> bootstrap cache, then anchor-flagged entries in the peers table</li>
|
||||
<li>Gather anchors: mine the retained uniques pools for anchor-flagged addresses (<span class="badge badge-complete">Implemented</span> — disconnected peers' pools survive as overwritten memory until new handshakes replace them), then the <code>known_anchors</code> bootstrap cache, then anchor-flagged entries in the peers table. Anchors that recently refused us are pushed to the back rather than dropped — a refusal is a load signal, not a blacklist.</li>
|
||||
<li>Make a convection call to each anchor in turn: receive referrals, try direct connect to each, fall back to anchor-coordinated hole punch for unreachable referrals (15s per attempt)</li>
|
||||
<li>Signal the growth loop as soon as any connection lands; stale anchors are pruned per the 3-day rule (see <a href="#anchors">Anchors & Connection Convection</a>)</li>
|
||||
</ol>
|
||||
|
|
@ -337,18 +339,18 @@
|
|||
</ul>
|
||||
<p><strong>Processing</strong>: their N1 → our N2 table (tagged to reporter), their N2 → our N3 table (tagged to reporter). Store profile, apply deletes, record replica overlaps. New N1 knowledge signals the growth loop immediately.</p>
|
||||
<div class="note">
|
||||
<strong>v0.8 change</strong>: the knowledge-share component becomes the <em>uniques announce</em> (two pools: forwardable N0–N2 + terminal N3; stored N4 is never transmitted), with the receiver storing each entry shifted one bounce deeper, our terminal pool becoming their N4 (see <a href="#layers">Network Knowledge</a>). The handshake's other fields carry forward unchanged. <span class="badge badge-rework">Rework</span>
|
||||
<strong>v0.8 (Iteration C, done)</strong>: the knowledge-share component is now the <em>uniques announce</em> (two pools: forwardable N0–N2 + terminal N3; stored N4 is never transmitted), with the receiver storing each entry shifted one bounce deeper, our terminal pool becoming their N4 (see <a href="#layers">Network Knowledge</a>). The handshake's other fields carry forward unchanged, except that a temporary referral slot now sends <code>null</code> uniques and an empty peer-address list. <span class="badge badge-complete">Implemented</span>
|
||||
</div>
|
||||
|
||||
<h3>Incremental Routing Diffs (every 120s + 4h full state) <span class="badge badge-rework">Rework</span></h3>
|
||||
<p><code>NodeListUpdate</code> (<code>0x01</code>) carries N1 added/removed and N2 added/removed, sent to connected peers on a 120s cycle; every 4 hours a full-state re-broadcast catches missed diffs. Diffs are also pushed eagerly after each growth-loop connection. Receiver processing mirrors the initial exchange: their N1 adds → our N2 adds, their N2 adds → our N3 adds; diffs with N1 additions signal the growth loop.</p>
|
||||
<div class="note">
|
||||
<strong>v0.8 change</strong>: diffs become uniques-announce diffs under the N1–N4 model, and their cadence folds into the update-cadence system's traffic shaping (see <a href="#keep-alive">Update Cadence & Keep-Alive</a>). <span class="badge badge-rework">Rework</span>
|
||||
<strong>v0.8 (Iteration C, done)</strong>: the 2-minute broadcast is now a full uniques announce under the N1–N4 model, skipped entirely when the content digest is unchanged. Folding its cadence into the update-cadence system's traffic shaping (see <a href="#keep-alive">Update Cadence & Keep-Alive</a>) is <span class="badge badge-planned">Planned</span> for Iteration D.
|
||||
</div>
|
||||
</section>
|
||||
<!-- Network Knowledge: N1–N4 Uniques -->
|
||||
<section id="layers">
|
||||
<h2>6. Network Knowledge: N1–N4 Uniques <span class="badge badge-rework">Rework</span></h2>
|
||||
<h2>6. Network Knowledge: N1–N4 Uniques <span class="badge badge-complete">Implemented</span></h2>
|
||||
|
||||
<p>Every node maintains a layered index of <strong>uniques</strong> — IDs it can help other nodes reach. The index spans four bounces (N1–N4) and is exchanged only between directly-connected mesh peers. Collectively, these per-node indexes form the network's <strong>distributed search index</strong>: a pull (see <a href="#sync">Sync Protocol</a>) is an exchange of uniques lists — "if you want these IDs, talk to me and I'll help you find them" — and answering "who can get me to ID X?" is a walk down reporter chains. Content never travels with the index; it lives in the CDN (see <a href="#files">Files & Storage</a>).</p>
|
||||
|
||||
|
|
@ -365,25 +367,30 @@
|
|||
<li><strong>CDN file peers</strong> — holders we exchanged files with</li>
|
||||
</ul>
|
||||
<p>Sources are <strong>merged before announcing</strong> — a receiver cannot tell whether we know an ID from a mesh slot, a friendship, or a file transfer. Anonymous and throwaway IDs (temporary poster IDs in comment chains, greeting senders) are <strong>deliberately kept in the lists</strong>: they are network noise that protects anonymity and future hooks for anonymous encrypted messaging over temp IDs. Comment TTL expiry (see <a href="#deletes">Delete Propagation</a>) retires them naturally — when the content dies, the throwaway ID falls out of everyone's uniques.</p>
|
||||
<p><strong>Our own posting identities are never in our own uniques</strong> <span class="badge badge-complete">Implemented</span>. Our node ID — and, when we are an anchor, our address — ride slice 0 of the same announce, so listing one of our personas at bounce 1 would be a signed-by-transport persona→device mapping handed to all ~20 mesh peers. It is also the easiest possible mapping to extract: our own personas are the invariant across every announce we send while everything else churns. Every persona is excluded (not just the default posting ID), as is the author of any comment we wrote locally — which covers the per-greeting throwaway IDs we mint, where we would otherwise be the first node in the network to announce the ID, at bounce 1, the instant the greeting was created. Our personas stay discoverable through everyone <em>else</em> who holds our content; we simply must not be the reporter. See <a href="#privacy">Graph Privacy</a>.</p>
|
||||
<p>Temp referral slots (see <a href="#connections">Connections</a>) are the one class of direct connection that does <strong>not</strong> enter the exchange: they carry no knowledge state in either direction until they graduate into mesh slots.</p>
|
||||
|
||||
<h3>The four layers</h3>
|
||||
<table>
|
||||
<tr><th>Layer</th><th>Bounce</th><th>Source</th><th>Announced?</th><th>Stored in</th></tr>
|
||||
<tr><td>N1</td><td>1</td><td>Our own uniques (merged from all sources above)</td><td>Yes</td><td><code>mesh_peers</code> + <code>social_routes</code> + <code>file_holders</code></td></tr>
|
||||
<tr><td>N2</td><td>2</td><td>Peers' announced N1, tagged to reporter</td><td>Yes</td><td><code>reachable_n2</code></td></tr>
|
||||
<tr><td>N3</td><td>3</td><td>Peers' announced N2, tagged to reporter</td><td>Yes</td><td><code>reachable_n3</code></td></tr>
|
||||
<tr><td>N4</td><td>4</td><td>Peers' announced N3, tagged to reporter</td><td><strong>Never</strong> — terminates</td><td><code>reachable_n4</code> <span class="badge badge-planned">Planned</span></td></tr>
|
||||
<tr><td>N1</td><td>1</td><td>Our own uniques (merged from all sources above)</td><td>Yes</td><td>Derived at build time from <code>mesh_peers</code> + <code>social_routes</code> + <code>file_holders</code> + <code>post_replicas</code> + <code>cdn_manifests</code> + <code>comments</code></td></tr>
|
||||
<tr><td>N2</td><td>2</td><td>Peers' announced N1, tagged to reporter</td><td>Yes</td><td><code>reachable</code>, <code>bounce = 2</code></td></tr>
|
||||
<tr><td>N3</td><td>3</td><td>Peers' announced N2, tagged to reporter</td><td>Yes</td><td><code>reachable</code>, <code>bounce = 3</code></td></tr>
|
||||
<tr><td>N4</td><td>4</td><td>Peers' announced N3, tagged to reporter</td><td><strong>Never</strong> — terminates</td><td><code>reachable</code>, <code>bounce = 4</code></td></tr>
|
||||
</table>
|
||||
<p>There is <strong>one</strong> <code>reachable</code> table, keyed <code>(reporter_node_id, reachable_id)</code>, with <code>bounce</code>, <code>id_class</code>, <code>is_anchor</code>, <code>addresses</code> and <code>updated_at</code> columns — not a table per layer. The composite key gives per-reporter set-merge for free, and the upsert keeps the <em>shallowest</em> bounce ever heard from that reporter, so one ID can never occupy two depths from one source (the ambiguity the old <code>reachable_n2</code>/<code>reachable_n3</code> pair papered over by sorting).</p>
|
||||
|
||||
<h3>Announce protocol <span class="badge badge-rework">Rework</span></h3>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>To/from N1 only.</strong> Uniques are exchanged exclusively with direct mesh peers — never forwarded on a node's behalf.</li>
|
||||
<li><strong>Full exchange on connect</strong>, incremental seq-numbered add/remove diffs afterward (today's <code>NodeListUpdate</code> <code>0x01</code> is the seed; it grows an N3 layer).</li>
|
||||
<li><strong>Every announce is a full snapshot <span class="badge badge-complete">Implemented</span></strong> (<code>full: true</code>) that replaces the sender's rows wholesale. Incremental seq-numbered diffing across four layers with anchor metadata was rejected as not worth the divergence risk; instead the sender hashes the pools (ignoring <code>seq</code>) and <strong>skips the send entirely when nothing changed</strong>, which recovers most of the saving. Slice trimming is deterministic (anchors first, then by ID) so a stable index yields a stable digest rather than thrashing it.</li>
|
||||
<li><strong>An announcement carries two explicitly separated pools.</strong> Pool 1 — the <em>forwardable pool</em>: our N0–N2 uniques (self + own uniques + two bounces), deduplicated internally. Pool 2 — the <em>terminal pool</em>: our N3 uniques, deduplicated against pool 1. The receiver stores each entry shifted one bounce deeper (pool 1 → their N1–N3, pool 2 → their N4), all tagged to us as reporter.</li>
|
||||
<li><strong>The terminal pool is used, never forwarded.</strong> Entries received via pool 2 (stored as N4) may be searched and resolved through, but never appear in the receiver's own announcements. The explicit terminal tag is what stops eternal re-propagation — some data must be differentiated as use-only, and there is no N5 anywhere in the network. (Carrying stored N4 in the payload was considered and rejected: dedup math put the resulting stored horizon near ~760k entries per node.)</li>
|
||||
<li><strong>Anchor entries carry addresses; everything else is a bare ID.</strong> There is no point sharing an address that cannot be connected without an introduction, so pool entries carry addresses only when flagged as anchors (directly reachable). The uniques index thereby doubles as the network's <strong>anchor directory</strong> — and with IPv6 pushing an estimated 20–30% of nodes into anchor candidacy, every node's pools hold a deep reserve of re-entry addresses.</li>
|
||||
<li><strong>Disconnect cleanup:</strong> when a peer drops, every entry it reported is removed. Stale entries are pruned after 5 hours regardless.</li>
|
||||
<li><strong>Node-class and author-class IDs travel in separate packed arrays <span class="badge badge-complete">Implemented</span>.</strong> A slice is <code>{ nodes, authors, anchors }</code>: two base64 blobs of concatenated 32-byte IDs plus the address-bearing anchor array. The separation is load-bearing, not cosmetic — a receiver structurally cannot mistake a persona ID for a connectable node ID (so persona IDs never enter connect/hole-punch paths and never burn punch timeouts), and the anchor array is the only place an address can appear. An author-class entry is emitted as a bare ID even if its stored row carries an anchor flag.</li>
|
||||
<li><strong>Anchor entries carry addresses; everything else is a bare ID.</strong> There is no point sharing an address that cannot be connected without an introduction, so pool entries carry addresses only when flagged as anchors (directly reachable). The uniques index thereby doubles as the network's <strong>anchor directory</strong> — and with IPv6 pushing an estimated 20–30% of nodes into anchor candidacy, every node's pools hold a deep reserve of re-entry addresses. A received anchor entry is an <strong>index row, not an address record</strong>: nothing has proven the address belongs to that node ID, so it stays in the uniques index (where the anchor mine reads it) and never mutates authoritative address state. Only an anchor we have completed a QUIC handshake to — iroh having proven the node ID owns that path — is promoted into <code>peers.is_anchor</code> and the <code>known_anchors</code> bootstrap cache. Without that rule a single mesh peer could overwrite our stored address for any node (including the DNS-resolved bootstrap anchor), aim thousands of entries at one victim IP, and have every node one bounce out re-announce them.</li>
|
||||
<li><strong>Sizes are enforced, not merely budgeted <span class="badge badge-complete">Implemented</span>.</strong> The uniques opcodes read against a dedicated 2 MB ceiling rather than the 64 MB file-transfer cap; built slices are capped (4,000 shallow / 12,000 terminal); received slices are truncated to a per-bounce accept cap (8,000 at N2 and N3, 24,000 at N4) with a log rather than rejected, so a large-but-honest peer degrades gracefully and a hostile one cannot make us write six figures of rows per announce. A payload whose forwardable pool is not exactly three slices is rejected outright — nothing may ever be stored past bounce 4.</li>
|
||||
<li><strong>Addresses are filtered on receipt as well as on send <span class="badge badge-complete">Implemented</span>.</strong> Inbound anchor addresses are normalized (IPv4-mapped IPv6 → v4) and dropped unless publicly routable; an anchor entry left with no usable address degrades to a bare node entry. Storing a peer-supplied <code>127.0.0.1</code> or RFC1918 address would turn the index into attacker-directed internal probing from our own host — and every such dial enters iroh's per-endpoint path store permanently.</li>
|
||||
<li><strong>Disconnect retention <span class="badge badge-complete">Implemented</span>:</strong> when a peer drops, its reported entries are <strong>kept</strong>. Pool knowledge is <em>overwritten memory</em> — a reporter's rows are replaced (per reporter, per bounce) only when a new handshake from that reporter lands, which is exactly what lets a node knocked down from 20 peers to 4 mine 16 dead peers' pools for anchor addresses to reconnect through (see <a href="#anchors">Anchors</a>). Staleness is bounded by the 5-hour prune, not by disconnection. Wiping on disconnect would empty the index — and collapse the anchor-density prior that steers the adaptive convection weights — at precisely the moment the node needs both most.</li>
|
||||
<li>Refreshes ride the regular update cadence (see <a href="#keep-alive">Update Cadence & Keep-Alive</a>) so index traffic blends into uniform network overhead.</li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -402,7 +409,8 @@
|
|||
<p>Because N1 includes directs and CDN contacts (not just the 20 mesh slots), per-node branching runs above 20 and real N4 lists can exceed the pure-mesh bound — the "160k+" is a planning floor, not a ceiling. With reporter tags and SQLite index overhead, expect low tens of MB worst-case on disk. That is acceptable on desktop; if list sizes blow past the budget in practice, the depth default shrinks (<20 peers @ 3 bounces was the explicit fallback in the design ruling).</p>
|
||||
|
||||
<h3>Device-tiered depth</h3>
|
||||
<p>Depth is a per-device choice, not a protocol constant. Desktop and anchor nodes hold the full N1–N4. Mobile may hold only 3 bounces (~8k entries, ~256 KB) or keep N4 as a <strong>Bloom filter</strong> (~10 bits/entry at 1% false-positive → ~200 KB for 160k entries). A Bloom N4 answers the only question N4 exists for — "can this peer help me reach ID X?" — without supporting enumeration, which N4 never needs since it is never announced.</p>
|
||||
<p>Depth is a per-device choice, not a protocol constant. Desktop and anchor nodes hold the full N1–N4. <strong>Mobile holds 3 bounces</strong> <span class="badge badge-complete">Implemented</span>: it drops the terminal pool on receipt and advertises <code>depth = 3</code> in its announce, so senders skip <em>building</em> the terminal slice for it — the single biggest bandwidth saving on a cellular link. Mobile still announces its own N3, so it remains a full participant in the network's depth; it only declines to store the terminal layer.</p>
|
||||
<p><strong>Bloom-compressed N4 <span class="badge badge-planned">Planned</span></strong> was the alternative (~10 bits/entry at 1% false-positive → ~200 KB for 160k entries) and is <em>not</em> implemented. A Bloom N4 would answer the only question N4 exists for — "can this peer help me reach ID X?" — without supporting enumeration, which N4 never needs since it is never announced. Dropping is the simpler of the two and is what ships.</p>
|
||||
|
||||
<h3>Indexed access</h3>
|
||||
<p>A node has <strong>indexed access</strong> to a target if the target appears anywhere in its N1–N4: the target is then resolvable by reporter-chain queries alone, with no worm search or relay introduction. The update-cadence system (see <a href="#keep-alive">Update Cadence & Keep-Alive</a>) uses indexed access to decide whether extra session connections are needed for social and file operations. (This replaces the old "<N4 access" term, which meant "within 3 hops" back when N4 didn't exist as a layer.)</p>
|
||||
|
|
@ -425,7 +433,7 @@
|
|||
<tr><td>5</td><td>Relay introduction</td><td>15s</td><td>Hole punch coordinated via an intermediary (see <a href="#relay">Relay & NAT Traversal</a>)</td></tr>
|
||||
<tr><td>6</td><td>Session relay</td><td>—</td><td><strong>Opt-in only, default OFF</strong> on both the serving and using side (see <a href="#relay">Relay & NAT Traversal</a>)</td></tr>
|
||||
</table>
|
||||
<p>Peers that fail all direct steps are marked likely-unreachable, so later attempts skip straight to introduction rather than burning timeouts on dead addresses. The cascade grows one lookup deeper when N4 lands (a 3-link reporter chain); everything else carries over unchanged.</p>
|
||||
<p>Peers that fail all direct steps are marked likely-unreachable, so later attempts skip straight to introduction rather than burning timeouts on dead addresses — but only when a reporter was actually available to ask; failing for want of a connected reporter says nothing about the candidate and must not suppress it. Resolution and growth-candidate scoring now share one horizon (bounces 2–4), and depth is a <em>graded</em> penalty in the scorer so a bounce-4 sighting can never outrank a bounce-2 one. <strong>The third reporter-chain link — resolving an N4 hit through the reporter's reporter — has not landed</strong> <span class="badge badge-planned">Planned</span>: today an N4 entry is resolved by asking its direct reporter, exactly like N2 and N3.</p>
|
||||
</section>
|
||||
|
||||
<!-- Three-Layer Architecture -->
|
||||
|
|
@ -1059,10 +1067,10 @@
|
|||
</table>
|
||||
<p style="color: var(--text-muted);">Retired type bytes, never reused: <code>0x42</code>–<code>0x45</code> (PostNotification/PostPush/AudienceRequest/AudienceResponse, v0.6.2), <code>0x95</code> (BlobDeleteNotice — holders evict via LRU, no notice needed), <code>0xA0</code> (GroupKeyDistribute — group seeds travel as encrypted posts).</p>
|
||||
|
||||
<h3>Pull = uniques-index exchange <span class="badge badge-rework">Rework</span></h3>
|
||||
<h3>Pull = uniques-index exchange <span class="badge badge-complete">Implemented</span></h3>
|
||||
<p>In v0.8 a pull is <strong>not</strong> a post transfer. A pull exchanges <strong>uniques lists</strong>: the set of unique IDs each side can help the other reach — mesh peers, direct contacts, CDN file authors, and CDN file peers, including throwaway/anonymous IDs (deliberate noise that protects real identities). The semantics are “if you want one of these IDs, talk to me and I'll help you find their CDN holders to connect to and search.” The union of these lists across the mesh forms a <strong>distributed search index</strong> — the same index that powers person-discovery for the <a href="#discovery">registry</a>. What a node knows and announces at each depth (N1–N4, termination rules, dedup, size budgets) is specified in <a href="#layers">Network Knowledge</a>; this section covers only the wire exchange.</p>
|
||||
<p>Content itself never rides the pull path: posts and blobs travel through the CDN (<code>PostFetchRequest</code>/<code>BlobRequest</code> against holders resolved through the index — see <a href="#content">Content Propagation</a>). The old “never pull from mesh peers” principle dissolves cleanly: the mesh exchanges the <em>index</em>, the CDN carries the <em>content</em>.</p>
|
||||
<p>The exact request/response payload shape (announce diffs, Bloom compression for mobile N4) is still open — see Network Knowledge.</p>
|
||||
<p>The pull opcodes (<code>0x40</code>/<code>0x41</code>) now carry <strong>only</strong> the index — that is what makes "a pull is not a post transfer" literally true of them. Post bodies still move between mesh peers over a separate, explicitly <strong>transitional</strong> content-sync pair (<code>0x46 ContentSyncRequest</code> / <code>0x47 ContentSyncResponse</code>) <span class="badge badge-rework">Rework</span>, which exists until the CDN cadence work in Iteration D can carry everything. Read caps differ accordingly: the uniques opcodes read against a 2 MB ceiling, not the 64 MB file-transfer cap.</p>
|
||||
<div class="note">
|
||||
<strong>What it replaces</strong>: today <code>PullSyncRequest</code> (0x40) carries a merged query list plus per-author <code>since_ms</code> timestamps (Self Last Encounter), and the responder returns posts newer than each timestamp after <code>should_send_post()</code> visibility filtering, plus visibility updates for own posts the requester already holds. The cycle ticks every 60s: a full pull from all connected peers on the first tick, then pulls only for authors stale beyond 4 hours. This per-author post-pull dies with v0.8; its staleness-tiering idea survives inside the <a href="#keep-alive">update cadence system</a>.
|
||||
</div>
|
||||
|
|
@ -1402,9 +1410,9 @@
|
|||
<h3>What is never shared</h3>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>Follows</strong> are never shared in gossip or profiles. <span class="badge badge-complete">Implemented</span></li>
|
||||
<li><strong>Uniques announcements carry bare IDs — except anchors.</strong> The uniques announce (see <a href="#layers">Network Knowledge</a>) ships NodeIds without addresses; the sole exception is anchor-flagged entries, whose addresses ride the pools by design (self-declared reachable infrastructure — publishing them links nothing to any person). <span class="badge badge-rework">Rework</span> (today's N1/N2 share lists are fully address-free; the uniques format adds the anchor exception)</li>
|
||||
<li><strong>N4 is never re-announced</strong> — knowledge terminates at the fourth bounce, so an observer M hops away cannot enumerate your neighborhood through transitive gossip. <span class="badge badge-planned">Planned</span></li>
|
||||
<li><strong>Posting identities never map to device addresses on the wire.</strong> Addresses attach to <em>network</em> identities only; the persona split (see <a href="#identity-architecture">Identity Architecture</a>) keeps the two namespaces unlinkable. <span class="badge badge-rework">Rework</span> — today's <code>AuthorManifest.author_addresses</code> ships the device's real addresses under a posting-identity author. This is a privacy bug; the v0.8 manifest carries no device addresses, and the fix must land before or with the discovery registry (location-anonymity dependency).</li>
|
||||
<li><strong>Uniques announcements carry bare IDs — except anchors.</strong> The uniques announce (see <a href="#layers">Network Knowledge</a>) ships NodeIds without addresses; the sole exception is anchor-flagged entries, whose addresses ride the pools by design (self-declared reachable infrastructure — publishing them links nothing to any person). Node-class and author-class IDs travel in separate packed arrays, and the anchor array is the only address-bearing field, so the exception is structural rather than conventional. <span class="badge badge-complete">Implemented</span></li>
|
||||
<li><strong>N4 is never re-announced</strong> — knowledge terminates at the fourth bounce, so an observer M hops away cannot enumerate your neighborhood through transitive gossip. The announce builder simply never reads bounce-4 rows, and third-party anchor reports are never promoted into our own bounce-1 slice — that promotion would have reset the horizon of every address-bearing entry on every hop, forever. <span class="badge badge-complete">Implemented</span></li>
|
||||
<li><strong>Posting identities never map to device addresses on the wire.</strong> Addresses attach to <em>network</em> identities only; the persona split (see <a href="#identity-architecture">Identity Architecture</a>) keeps the two namespaces unlinkable. <span class="badge badge-complete">Implemented</span> — <code>AuthorManifest.author_addresses</code> is gone (Iteration A). The same invariant is enforced a second time in the uniques announce (Iteration C): the wire separates node-class from author-class IDs into distinct arrays so an address can never attach to a persona, and our own personas — every one of them, plus the author of any comment we wrote locally, which covers per-greeting throwaway IDs — are excluded from the uniques we announce, because our node ID and anchor address ride the same payload. Our personas stay discoverable through other holders of our content; we must simply never be the reporter. See <a href="#layers">Network Knowledge</a>.</li>
|
||||
</ul>
|
||||
|
||||
<h3>Where addresses do travel, and why</h3>
|
||||
|
|
@ -1416,7 +1424,7 @@
|
|||
</ul>
|
||||
|
||||
<h3>Uniques lists as cover traffic</h3>
|
||||
<p>The N1 uniques list deliberately merges mesh peers, social contacts, CDN file authors, CDN file peers, and throwaway/anonymous IDs into one flat set. An observer diffing your announcements over time is looking at a list dominated by high-churn CDN-derived entries and deliberate throwaway-ID noise; the stable social core does not stand out the way it did when the share list was just mesh-plus-contacts. Throwaway IDs additionally age out of the network via comment TTL (<a href="#deletes">above</a>), so the noise floor is self-renewing rather than monotonically accumulating. <span class="badge badge-planned">Planned</span> (today's N1 share merges only mesh peers + social contacts)</p>
|
||||
<p>The N1 uniques list deliberately merges mesh peers, social contacts, CDN file authors, CDN file peers, and throwaway/anonymous IDs into one flat set. An observer diffing your announcements over time is looking at a list dominated by high-churn CDN-derived entries and deliberate throwaway-ID noise; the stable social core does not stand out the way it did when the share list was just mesh-plus-contacts. Throwaway IDs additionally age out of the network via comment TTL (<a href="#deletes">above</a>), so the noise floor is self-renewing rather than monotonically accumulating. Our own personas and our own greeting throwaways are the deliberate exception — they are excluded, because we would be the reporter. <span class="badge badge-complete">Implemented</span></p>
|
||||
|
||||
<h3>The timing adversary <span class="badge badge-planned">Planned</span></h3>
|
||||
<p>With content encrypted and recipient lists off the wire, the remaining deanonymizer is <strong>request-timing correlation</strong>: an observer who can watch traffic sees <em>who fetches what, when</em>. If your device checks an author's CDN seconds after every new post, you are that author's reader; if fresh content always radiates outward from one node first, that node is the author. The v0.8 defenses are behavioral, not cryptographic:</p>
|
||||
|
|
@ -1892,8 +1900,14 @@
|
|||
<tr><td>WORM_DEDUP</td><td>10s</td><td>In-flight worm dedup</td></tr>
|
||||
<tr><td>WORM_COOLDOWN</td><td>300s (5 min)</td><td>Miss cooldown before retry</td></tr>
|
||||
<!-- connection.rs:61 REFERRAL_DISCONNECT_GRACE_MS = 120_000; connection.rs:63 REFERRAL_LIST_CAP = 50 -->
|
||||
<tr><td>REFERRAL_DISCONNECT_GRACE</td><td>120s (2 min)</td><td><span class="badge badge-rework">Rework</span> Current-code registration model: anchor keeps a disconnected caller referable during this grace; retired by the convection window (see <a href="#anchors">Anchors</a>)</td></tr>
|
||||
<tr><td>REFERRAL_LIST_CAP</td><td>50</td><td><span class="badge badge-rework">Rework</span> Current-code registration model: soft cap on the anchor's registration list, driving per-entry use tiering; retired by the convection window (replacement window size still open)</td></tr>
|
||||
<tr><td>CONVECTION_WINDOW_SIZE</td><td>16</td><td><span class="badge badge-complete">Implemented</span> Rolling caller window on the anchor. Small on purpose: the point is recency, not coverage. (Retired <code>REFERRAL_DISCONNECT_GRACE</code> 120s and <code>REFERRAL_LIST_CAP</code> 50 from the registration model.)</td></tr>
|
||||
<tr><td>CONVECTION_HANDOUT_CAP</td><td>2</td><td><span class="badge badge-complete">Implemented</span> Hand-outs before a window entry rotates out ("2 before, 2 after")</td></tr>
|
||||
<tr><td>CONVECTION_REFERRALS</td><td>2</td><td><span class="badge badge-complete">Implemented</span> Referrals returned per served request</td></tr>
|
||||
<tr><td>CONVECTION_ENTRY_MAX_AGE</td><td>15 min</td><td><span class="badge badge-complete">Implemented</span> Older window entries are not handed out — a stale address costs the next caller a punch timeout</td></tr>
|
||||
<tr><td>CONVECTION_SUPPLEMENT_CAP</td><td>2 per 15 min</td><td><span class="badge badge-complete">Implemented</span> Hand-out budget for mesh peers used to supplement a sparse window (address-free, introduction-only)</td></tr>
|
||||
<tr><td>ANCHOR_REFUSAL_PENALTY</td><td>5 min</td><td><span class="badge badge-complete">Implemented</span> How long a refusing anchor is de-prioritised; the penalty map is swept on every insert</td></tr>
|
||||
<tr><td>UNIQUES read cap</td><td>2 MB</td><td><span class="badge badge-complete">Implemented</span> Dedicated ceiling for 0x01/0x40/0x41, distinct from the 64 MB file-transfer cap</td></tr>
|
||||
<tr><td>UNIQUES build / accept caps</td><td>4,000 & 12,000 / 8,000 & 24,000</td><td><span class="badge badge-complete">Implemented</span> Per-slice build caps (shallow, terminal) and per-bounce accept caps (N2–N3, N4); received slices are truncated, not rejected</td></tr>
|
||||
<!-- connection.rs:3243 prune_n2_n3(5*60*60*1000); disconnect clear connection.rs:3127-3129; boot sweep node.rs:305-311 -->
|
||||
<tr><td>KNOWLEDGE_STALE_PRUNE</td><td>Immediate on disconnect + 5h fallback</td><td>Remove reach entries tagged to disconnected reporters; age fallback for stragglers. Boot sweep clears all non-mesh entries.</td></tr>
|
||||
<!-- connection.rs:55 WATCHER_EXPIRY_MS = 30 days; prune at connection.rs:3247 -->
|
||||
|
|
@ -1933,8 +1947,8 @@
|
|||
<tr><td>HTTP_HEADER_TIMEOUT</td><td>5s</td><td>HTTP request header read timeout</td></tr>
|
||||
<tr><td>HTTP_REDIRECT_PROBE</td><td>200ms</td><td>TCP liveness probe of a holder before issuing a 302</td></tr>
|
||||
<!-- Ruling: project_v08_mesh_redesign.md round 4 — per-disconnect stochastic action replaces threshold+jitter -->
|
||||
<tr><td>CONVECTION_ACTION</td><td>stochastic, per-disconnect</td><td><span class="badge badge-planned">Planned</span> On each mesh disconnect: random {nothing | anchor introduction | mesh-peer introduction}; weights adaptive = local anchor-density prior + refusal feedback</td></tr>
|
||||
<tr><td>CONVECTION_CLASS</td><td>entry / top-up (1 bit)</td><td><span class="badge badge-planned">Planned</span> Entry (<2 connections) always served; top-up refused cheaply under load — refusal feeds the adaptive weights</td></tr>
|
||||
<tr><td>CONVECTION_ACTION</td><td>stochastic, per-disconnect</td><td><span class="badge badge-complete">Implemented</span> On each mesh disconnect: random {nothing | anchor introduction | mesh-peer introduction}; weights adaptive = local anchor-density prior + refusal feedback</td></tr>
|
||||
<tr><td>CONVECTION_CLASS</td><td>entry / top-up (1 bit)</td><td><span class="badge badge-complete">Implemented</span> Entry (<2 connections) always served; top-up refused cheaply under load — refusal feeds the adaptive weights</td></tr>
|
||||
<!-- Ruling: project_v08_mesh_redesign.md — comment TTL rand(30–365d) fixed at creation -->
|
||||
<tr><td>COMMENT_TTL</td><td>rand(30–365 days)</td><td><span class="badge badge-planned">Planned</span> Comment expiry, fixed at creation and carried with the comment</td></tr>
|
||||
<!-- Ruling: project_v08_mesh_redesign.md — descending update cadence minutes→days -->
|
||||
|
|
@ -1950,7 +1964,7 @@
|
|||
<!-- Target per ruling (~20 mesh); current code Preferred/Local/Wide 10/71/20 desktop, 3/7/5 mobile (types.rs:657-676) -->
|
||||
<tr><td>Mesh slots</td><td>~20 (Desktop) / 15 (Mobile)</td><td>Single pool, no tiers. <span class="badge badge-rework">Rework</span> — current code runs 101/15 across a Preferred/Local/Wide split</td></tr>
|
||||
<!-- Ruling round 3: +4..+10 temp referral slots, no knowledge exchange -->
|
||||
<tr><td>Temp referral slots</td><td>+4 to +10 above the cap</td><td><span class="badge badge-planned">Planned</span> Introduction facilitation only; no uniques exchange; graduate or expire</td></tr>
|
||||
<tr><td>Temp referral slots</td><td>+10 desktop / +4 mobile, above the cap</td><td><span class="badge badge-complete">Implemented</span> Introduction facilitation only; no uniques or peer-address exchange; graduate or expire (5 min TTL)</td></tr>
|
||||
<!-- Ruling: 20^4, N4 terminates; mobile 3 bounces / Bloom N4 -->
|
||||
<tr><td>Knowledge depth</td><td>4 bounces (N4 terminates)</td><td><span class="badge badge-planned">Planned</span> Mobile may hold 3 bounces or Bloom-compress N4</td></tr>
|
||||
<!-- types.rs:678-683 session_slots 20/5 -->
|
||||
|
|
@ -1986,12 +2000,12 @@
|
|||
<tr><th>Area</th><th>Status</th></tr>
|
||||
|
||||
<tr><td colspan="2" style="background: var(--bg-muted); font-weight: 600;">Mesh & knowledge</td></tr>
|
||||
<tr><td>Single ~20-slot mesh pool (tiers eliminated)</td><td><span class="badge badge-rework">Rework</span> — code runs Local/Wide, 91 desktop / 12 mobile (preferred tier deleted, Iteration B)</td></tr>
|
||||
<tr><td>Single ~20-slot mesh pool (tiers eliminated)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C): one pool, 20 desktop / 15 mobile</td></tr>
|
||||
<tr><td>Preferred-peer elimination (slots, MeshPrefer, prunes, watchers)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration B, Roadmap item 3)</td></tr>
|
||||
<tr><td>Temp referral slots (+4..+10, no knowledge exchange)</td><td><span class="badge badge-planned">Planned</span></td></tr>
|
||||
<tr><td>Uniques announce, payload N1–N3 (N4 stored but terminates, dedup at store + announce)</td><td><span class="badge badge-rework">Rework</span> — code exchanges N1–N3 share lists</td></tr>
|
||||
<tr><td>Pull as uniques-index exchange (distributed search index)</td><td><span class="badge badge-rework">Rework</span> — code pulls posts via <code>since_ms</code></td></tr>
|
||||
<tr><td>Growth loop (signal-driven, diversity scoring, stochastic anchor path)</td><td><span class="badge badge-rework">Rework</span> — reactive core + scoring shipped; whole-pool counting and the per-disconnect stochastic action are target</td></tr>
|
||||
<tr><td>Temp referral slots (+4..+10, no knowledge exchange)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C): +10 desktop / +4 mobile, above the cap, never evicting a mesh peer</td></tr>
|
||||
<tr><td>Uniques announce, payload N1–N3 (N4 stored but terminates, dedup at store + announce)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C): two-pool announce, terminal N3, single <code>reachable</code> table, size caps enforced</td></tr>
|
||||
<tr><td>Pull as uniques-index exchange (distributed search index)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C); post bodies moved to the transitional 0x46/0x47 content sync</td></tr>
|
||||
<tr><td>Growth loop (signal-driven, diversity scoring, stochastic anchor path)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C): whole-pool counting, bounded SQL shortlist, graded depth penalty, per-disconnect stochastic action</td></tr>
|
||||
<tr><td>Worm search (point queries: posts, blobs, nodes; no keyword flooding)</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Social routing cache + checkins + reconnect watchers</td><td><span class="badge badge-complete">Implemented</span> — preferred_tree field retires with preferred peers</td></tr>
|
||||
<tr><td>Update-cadence scheduler (freshness × relationship, jitter, dedup)</td><td><span class="badge badge-rework">Rework</span> — freshness-only engagement tiers exist</td></tr>
|
||||
|
|
@ -1999,7 +2013,7 @@
|
|||
<tr><td colspan="2" style="background: var(--bg-muted); font-weight: 600;">Anchors & connectivity</td></tr>
|
||||
<tr><td>Reachability-based anchor candidacy (portmapper watcher, bidirectional)</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Anchor opt-in gate</td><td><span class="badge badge-planned">Planned</span> — reachable desktops auto-anchor today</td></tr>
|
||||
<tr><td>Connection convection (2-before/2-after rolling referrals, per-disconnect stochastic trigger)</td><td><span class="badge badge-rework">Rework</span> — register loop + tiered referral list exist; convection semantics are target</td></tr>
|
||||
<tr><td>Connection convection (2-before/2-after rolling referrals, per-disconnect stochastic trigger)</td><td><span class="badge badge-complete">Implemented</span> — Done (Iteration C): rolling window, entry/top-up classes, adaptive weights, 0xC0 register loop retired</td></tr>
|
||||
<tr><td>Batched bootstrap probes + 3-day stale-anchor self-prune</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>Anchor self-verification probe (3rd-party witness)</td><td><span class="badge badge-complete">Implemented</span> — scheduling re-keys to the watcher in v0.8</td></tr>
|
||||
<tr><td>Port mapping (UPnP-IGD + NAT-PMP + PCP via <code>portmapper</code>, all platforms)</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
|
|
@ -2027,7 +2041,7 @@
|
|||
<tr><td>FoF visibility Layers 1–5 (vouch, gated comments, FoFClosed, rotation/burn, unlock cache)</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
|
||||
<tr><td colspan="2" style="background: var(--bg-muted); font-weight: 600;">Sync & content</td></tr>
|
||||
<tr><td>Wire protocol (length-prefixed JSON, 64 MB payload)</td><td><span class="badge badge-rework">Rework</span> — 40 types today; six (0x51/0x52/0x71/0xA1/0xA2/0xB3) deleted in the clean-break purge (Roadmap item 3); 0xC0 AnchorRegister stays live until anchor convection (Roadmap item 5)</td></tr>
|
||||
<tr><td>Wire protocol (length-prefixed JSON, 64 MB payload)</td><td><span class="badge badge-complete">Implemented</span> — six types (0x51/0x52/0x71/0xA1/0xA2/0xB3) deleted in the clean-break purge (Iteration B); 0xC0 retired with anchor convection (Iteration C); the uniques opcodes read against a 2 MB cap</td></tr>
|
||||
<tr><td>Merged pull with recipient matching (all posting identities)</td><td><span class="badge badge-complete">Implemented</span> — payload semantics change with the uniques redefinition</td></tr>
|
||||
<tr><td>Manifest machinery (push, refresh, header diffs)</td><td><span class="badge badge-complete">Implemented</span></td></tr>
|
||||
<tr><td>AuthorManifest privacy (no device addresses, 2–5 neighborhood)</td><td><span class="badge badge-rework">Rework</span> — code ships addresses + 10+10 neighborhood</td></tr>
|
||||
|
|
@ -2076,11 +2090,11 @@
|
|||
</div>
|
||||
<div class="card">
|
||||
<h3>4. N1–N4 uniques knowledge layer</h3>
|
||||
<p>Narrow the mesh to ~20 slots (single pool), add temp referral slots (+4..+10, no knowledge exchange), and replace N1–N3 share lists with the two-pool uniques announce: forwardable N0–N2 + terminal N3, received entries stored shifted one bounce deeper, N4 held but never re-announced. Anchor entries carry addresses (the pools double as the anchor directory); everything else is bare IDs. Redefine pulls as uniques-index exchange. Device-tiered depth and Bloom-compressed N4 for mobile as needed.</p>
|
||||
<p><strong>Done (Iteration C)</strong> — with size caps enforced, receive-side address filtering, and our own personas excluded from our own announce. Bloom-compressed N4 remains unbuilt; mobile drops the terminal pool instead. Narrow the mesh to ~20 slots (single pool), add temp referral slots (+4..+10, no knowledge exchange), and replace N1–N3 share lists with the two-pool uniques announce: forwardable N0–N2 + terminal N3, received entries stored shifted one bounce deeper, N4 held but never re-announced. Anchor entries carry addresses (the pools double as the anchor directory); everything else is bare IDs. Redefine pulls as uniques-index exchange. Device-tiered depth and Bloom-compressed N4 for mobile as needed.</p>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h3>5. Anchor convection</h3>
|
||||
<p>Retire the register loop. Anchor candidacy = reachability watcher + opt-in (opt-in gate is low priority — IPv6-abundant anchors are unlikely to be overwhelmed); the anchor service becomes a rolling referral window (caller gets 2 recent prior callers, is referred to the next 2), with RelayIntroduce coordination when both ends are live, and entry/top-up request classes (entry always served; top-up refused cheaply under load). Clients act stochastically on each mesh disconnect — random {do nothing | anchor introduction | mesh-peer introduction} with adaptive weights (anchor-density prior + refusal feedback). Anchors are mined from the uniques pools; <code>known_anchors</code> recedes to a bootstrap cache.</p>
|
||||
<p><strong>Done (Iteration C)</strong> — except the opt-in gate, which stays low priority per the ruling. Third-party anchor claims are index rows only; <code>known_anchors</code> and <code>peers.is_anchor</code> are written from proven handshakes. Retire the register loop. Anchor candidacy = reachability watcher + opt-in (opt-in gate is low priority — IPv6-abundant anchors are unlikely to be overwhelmed); the anchor service becomes a rolling referral window (caller gets 2 recent prior callers, is referred to the next 2), with RelayIntroduce coordination when both ends are live, and entry/top-up request classes (entry always served; top-up refused cheaply under load). Clients act stochastically on each mesh disconnect — random {do nothing | anchor introduction | mesh-peer introduction} with adaptive weights (anchor-density prior + refusal feedback). Anchors are mined from the uniques pools; <code>known_anchors</code> recedes to a bootstrap cache.</p>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h3>6. Post concealment (2–5 budget)</h3>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue