20 lines
1.1 KiB
Text
20 lines
1.1 KiB
Text
The goal is basically to find a way to semi-automatically build
|
|
a relay structure for queues, in an attempt to supplement ActivityStreams
|
|
with a relay infrastructure to reduce maximum per-node load.
|
|
|
|
A good primer on the subject is https://gist.github.com/jdarcy/60107fe4e653819138396257df302eef
|
|
which introduces the problem the current RPC interface is causing
|
|
for ActivityStreams caching.
|
|
|
|
It appears to me that I basically want the combination of
|
|
* BitTorrent, but with queues instead of constant files (meaning that I need to
|
|
deal with queue positions and slices, and that the queue can grow on one end,
|
|
and vanish of the other)
|
|
* [episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md)
|
|
(libp2p's epidemic pubsub protocol, but with persistence/resume, so that
|
|
you don't loose all changes if you're offline (you just loose the ones which
|
|
were so old they scrolled out of the round-robin spool db))
|
|
|
|
One should also take note of the design of [Apache Kafka](https://kafka.apache.org/documentation/#design),
|
|
which uses disk-backed queue partitions, which are usually append-only.
|