Replication limiter

[pietgeursen]

Motivation:

manyverse needs the ability to limit the number or peers being replicated to improve performance of the app. This PR integrates the ssb-replication-limiter module I've written to help solve that problem.

Changes in this PR:

• adds a helper function getPeerAheadBy that the limiter can use to find out how far ahead a remote peer's feed is. The limiter uses this value to decide whether or not to change modes.
• change the request method from ebt.request (feedId, enableReplication) to ebt.request (feedId, enableReplication, [priority]) so the limiter can prioritise who to replicate first. See this PR for the changes to ssb-friends.
• adds and passes CI tests.
• exports the request function from the module. This wasn't done previously, I assume this was a bug because currently we're hooking the sbot.request function and it hasn't been needed.
• exposes methods to dynamically configure the limiter's threshold and maxNumConnections.
• uses config to set initial values for threshold and maxNumConnections

Using request(<feedId>, **false**) to pause replication.

Dominic said that using request like this might cause problems with eventual consistency. See this issue for more context.

Until ebt has a better pause feature I'm hoping we can use request like this.

[mmckegg]

About to start using this branch in Patchwork. I'll let ya'll know how it goes.

[mmckegg]

sorry @pietgeursen, made a bit of a mess of the commit history trying to get master merged in (and the merge conflicts properly resolved).

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

[christianbundy]

Anything I can do to help this along, or are you waiting from feedback from @staltz and @dominictarr?

[staltz]

I think this is part of a project that was cancelled, so I'm not sure that this code in this state is currently reflecting a fully functional code state that we want to include into master branch. I'm inclined to archiving this PR for future reference once we resume this work.

[christianbundy]

Cool, I would love to merge these improvements but if the PR isn't going to get merged then I'd prefer to just close it. I'm going to close both this and ssbc/ssb-friends#11, please re-open and/or ping me if I've done something rude/wrong.

❤️

[dominictarr]

if I recall correctly, this was intended to fix a problem with cpu usage, or slow syncing, but once it was ready it turned out it wasn't the problem - the problem was indexing.