TECHNICAL_ARCHITECTURE

The seven layers of storage

From TCP to HTTP API: content addressing, erasure coding, active replication and self-healing.

Seven-layer architecture

L4Agent layerSoul · claw-mem · DIDL3Storage layerErasure coding · pushToK · repairTick · PoSe S challengesL2Network layerKademlia DHT · Wire · gossipL1Settlement layer (Palium)CidRegistry · StorageRewardManagerContentProof
Content is written top-down into storage and the network; proofs and payments flow bottom-up to settle on Palium.
L7 · HTTP APIIPFS-compatible interface: add / cat / get / pin / ls / stat, plus the /ipfs/<cid> gateway
L6 · Erasure codingOptional Reed-Solomon via ?erasure=N+M — RS(4+2) / (6+3) / (8+4)
L5 · Replication and repairpushToK active push, fetchRemote passive retrieval, repairTick periodic healing
L4 · Content addressingUnixFS 256 KiB chunks, CIDv1, directory DAGs with HAMT
L3 · DiscoveryKademlia DHT, keccak256(cid) projection, provider records with 24 h TTL
L2 · TransportTCP Wire framing + HTTP gossip, dual channel
L1 · SettlementPalium: CidRegistry anchoring, PoSe storage challenges, StorageRewardManager

Four core algorithms

pushToK

On write, every block is actively pushed to K=3 nodes chosen by DHT distance; the writer is never the only holder.

fetchRemote

When a block is missing locally, it is fetched in parallel from provider records, cached, and re-announced so the fetcher becomes a provider too.

pushStripe

Erasure-coded stripes are spread shard by shard across different nodes; any M shards may be lost and the stripe still reconstructs.

repairTick

Every 10 minutes, up to 50 CIDs are scanned and topped back up to K replicas — repair is the network's normal behaviour.

Erasure-coding economics

The same fault tolerance at half the cost.

SchemeStorage costTolerated loss
Triple replication3.0×2 of 3 copies
RS(4+2)1.5×2 of 6 shards
RS(6+3)1.5×3 of 9 shards
RS(8+4)1.5×4 of 12 shards

PoSe storage challenges (type S)

Storage nodes do not self-report capacity; they are continuously sampled. A challenger names a chunk, the node returns a Merkle path proof within 6 seconds, witnesses sign, and the epoch settles.

  • A pass rate of at least 70% is required to qualify for the storage bucket that epoch
  • Storage score = pass rate × √(min(storedGB, 500) / 500): diminishing returns on capacity, no whale monopoly
  • Results are committed on-chain as a Merkle root; nodes claim $MESH with a proof
  • Losing data is slashed from the chain node's existing $PALI bond — no separate $MESH stake

Identity and memory layer

  • DIDRegistry: did:coc registration, key rotation, delegation chains with cascading revocation
  • SoulRegistry: on-chain anchor for agent state, 2/3 guardian social recovery + timelock
  • CidRegistry: immutable bytes32 → IPFS CID mapping, three-tier resolution (local → MFS → on-chain)
  • Soul backup: AES-256-GCM encryption, incremental Merkle snapshots, carried by the Carrier network
  • Resurrection state machine: rebuild the agent from the latest snapshot and inject recovery context

Disaster-recovery guarantees

  • One uploading node disappears: fetchRemote retrieves from the remaining providers
  • All uploading nodes disappear: pushToK replicas and erasure shards remain on uninvolved nodes
  • Replicas stay missing: repairTick tops them back up to K
  • The source node is replaced: the CidRegistry anchor is unchanged and content stays addressable by CID

Outlook: a storage market

Today the writer stores for free and pushed-to nodes hold passively; the next step brings paid storage, quotas and service levels into the same PoSe settlement — the $MESH payment split (70% nodes / 20% treasury / 10% burned) is designed for it. No dates are promised.

PaliMesh · Decentralized storage for AI agents