Make sure processedOffset is only updated together with processedSeq

[arj03]

From #291 (comment). I believe this should fix off-by-x bugs in level indexes.

From that comment:

Look at this line and imagine we have added a new plugin. In this case we will reindex, but most plugins will just skip. BUT they will update their processedOffset thus lovering it compared to processedSeq. If you then kill the app while some of them have updated, but NOT keys or base then there will be a mismatch!

[arj03]

Please double and triple check this :)

[staltz]

Open question: how do we handle reindexing? Do I just deploy a reindex of all leveldb indexes in Manyverse? Could we instead do that in ssb-db2 by bumping the version somehow? On the other hand, version is specific to each individual leveldb plugin, so we would have to bump all of them and some of them aren't even in this module. It would be good (maybe? what do you think?) if there existed a "metaVersion" in the META section that catches all leveldb plugins.

[arj03]

Open question: how do we handle reindexing? Do I just deploy a reindex of all leveldb indexes in Manyverse?

I think that would be the safest

Could we instead do that in ssb-db2 by bumping the version somehow?

I think a general level version would be good for this yes. As a separate PR for sure :)

[github-actions]

Benchmark results

Part	Duration
add 1000 elements	657ms
add 1000 private box1 elements	1139ms
unbox 1000 private box1 elements first run	330ms
unbox 1000 private box1 elements second run	112ms
add 1000 private box2 elements	680ms
unbox 1000 private box2 elements first run	429ms
unbox 1000 private box2 elements second run	146ms
Migrate (+db1)	10964ms
Migrate (alone)	3646ms
Migrate (+db1 +db2)	7222ms
Migrate (+db2)	5972ms
Migrate continuation (+db2)	956ms
Memory usage without indexes	720.76 MB = 30.69 MB + etc
Initial indexing	881ms
Initial indexing maxCpu=86	4127ms
Initial indexing compat	663ms
Two indexes updating concurrently	719ms
key one initial	47ms
key two	2ms
key one again	1ms
reboot and key one again	50ms
latest root posts	660ms
latest posts	53ms
votes one initial	515ms
votes again	1ms
hasRoot	314ms
hasRoot again	1ms
author one posts	269ms
author two posts	14ms
dedicated author one posts	278ms
dedicated author one posts again	2ms
Maximum memory usage	818.37 MB = 50.17 MB + etc
Indexes folder size	12.76mb

[staltz]

Let's gooooo

[github-actions]

Benchmark results

Part	Duration
add 1000 elements	674ms
add 1000 private box1 elements	1532ms
unbox 1000 private box1 elements first run	480ms
unbox 1000 private box1 elements second run	164ms
add 1000 private box2 elements	849ms
unbox 1000 private box2 elements first run	587ms
unbox 1000 private box2 elements second run	193ms
Migrate (+db1)	16135ms
Migrate (alone)	5299ms
Migrate (+db1 +db2)	12148ms
Migrate (+db2)	7850ms
Migrate continuation (+db2)	1283ms
Memory usage without indexes	738.11 MB = 26.02 MB + etc
Initial indexing	1226ms
Initial indexing maxCpu=86	5423ms
Initial indexing compat	1045ms
Two indexes updating concurrently	948ms
key one initial	49ms
key two	2ms
key one again	2ms
reboot and key one again	64ms
latest root posts	674ms
latest posts	11ms
votes one initial	713ms
votes again	2ms
hasRoot	500ms
hasRoot again	2ms
author one posts	434ms
author two posts	23ms
dedicated author one posts	414ms
dedicated author one posts again	2ms
Maximum memory usage	815.43 MB = 46.46 MB + etc
Indexes folder size	12.76mb