Reference count virtual object property references for GC

[FUDCo]

Previously we treated any reference to an object in the value of a virtual object property as a permanent reference to that object, preventing its garbage collection forever. With this set of changes, these references are now counted, and any unreferenced objects become eligible for garbage collection, regardless of whether the reference is to a virtual object, a presence or a local (remotable) object.

[FUDCo]

I think this also will close #3149

[warner]

I'm still picking through the case analysis, but this seems to be a pretty solid implementation of the original approach. However, #3649 (comment) and #3649 (comment) , and our conversation about GC-provoked metering divergence whenever the refcounting code looks at the slotToVal WeakRefs, makes me think that we need a different approach.

The specific sequence I'm concerned about would be:

• virtual object VO1 is created, stored in a property of VO2, and also stored in a regular property of some normal object NO3
  • VO1 is therefore held by two pillars: the Representative Object in RAM via NO3, and a nonzero refcount via virtualized data on VO2
• NO3 is deleted or modified, making the Representative become UNREACHABLE
• some time passes, and organic GC either runs (moving it to COLLECTED) or not (leaving it at UNREACHABLE)
  • when liveslots/VOM realizes this is COLLECTED, the Representative pillar will drop
• now VO2 is modified, causing the VO1 refcount to drop to zero, causing the virtualized-data pillar to drop
  • this will invoke the wrapped-data setter, then updateReferenceCounts, removeReachableVref, decRefCount, setRefCount, and possibleVirtualObjectDeath
  • possibleVirtualObjectDeath starts with a call to getValForSlot, which is GC-sensitive (it can sense the difference between UNREACHABLE and COLLECTED, because it probes the WeakRef)
    • if metering is enabled at this point, the CPU meter will depend upon GC timing
  • if COLLECTED, it will do some syscalls: getRefCount/getExportStatus both use the vatstore, and if all three pillars have dropped, it will delete VO1 with more syscalls

So that's a recipe for not only metering variance that depends on the GC state, but also syscall variance.

I think the fix is to be more conservative with deletion. You made an excellent start by introducing possiblyDeadSet, and I think we should lean into that. Basically every time an export- or virtualized-data- pillar is dropped, the vref should be added to possiblyDeadSet, and nothing else should happen. Later, when processDeadSet() is run (i.e. during bringOutYourDead, when it's safe to probe the WeakRefs since we're unmetered), that can poll all three pillars to see if any are still standing (or have been re-raised by re-export, re-instantiation, or by being stashed in some new bit of virtualized data).

My original deadSet was an "accurate" set of COLLECTED/FINALIZED Presence vrefs: I removed vrefs from deadSet when re-importing them, so by the time processDeadSet looked at it, everything therein was for sure not kept alive by in-memory Objects (which is why it didn't probe slotToVal for each one). I'm now thinking that it would be cleaner to get rid of deadSet in favor of strictly using possiblyDeadSet, and not remove anything from it except for during processDeadSet, just as you're doing with the virtual objects in the code you've added previously.

This would parallel the way the kernel and comms GC works. The kernel (in kernelKeeper.js) has maybeFreeKrefs, and each time a refcount touches zero, the kref gets added to the set, and processRefcounts checks on everything therein. Comms has maybeFree (in src/vats/comms/state.js) and processMaybeFree.

So the task (maybe as part of this PR, maybe as a follow-on PR) is to change liveslots and VOM.js:

• delete deadSet, and change the code that added things to it to use possiblyDeadSet instead
• processDeadSet:
  • build a sorted list from possiblyDeadSet, iterate through it
  • for each vref, figure out the type, then check all its pillars
    • wr = slotToVal.get(vref); wr && wr.deref() tells you the Object pillar (Representative/Presence), and must be checked for each one
    • the VOM knows about some others
• remove the two places in liveslots.js which do deadSet.delete: we won't pre-emptively stop paying attention to the vref; we'll rely on processDeadSet to notice that it's been re-registered
• change VOM.js possiblyVirtualObjectDeath to not use getValForSlot (it's ok to keep using it for addReachableVref/removeReachableVref to retrieve the Remotable)

In general, the interface between processDeadSet and VOM.js needs to be one of "hey VOM, here's a vref that we need to check, and I've confirmed that there's no in-memory Object keeping it alive, please check the remaining pillars and delete if appropriate". So VOM.js is never checking the Object (Representative) pillar itself, nor probing slotToVal or the WeakRefs therein.

If the VOM drops a pillar itself (i.e. deletion of virtualized data), the vref is added to possiblyDeadSet and nothing else happens until processDeadSet comes around again to check up on it. This will cause some garbage to be dropped later than strictly necessary, however to maintain metering determinism we can't afford to ask the question (i.e. probe slotToVal's WeakRef) that would let us realize we're in that state.

[mhofman]

Related to #3649 (comment), I'm not sure I follow the current usage of WeakRefs. IMO those should be derefed only if we intend to use the target, not simply to sense if the target is collected or not, which is the job of the FinalizationRegistry.

In my mind, most code would consider the Representative to be available, until declared dead by bringOutYourDead. The only code that actually needs to bother if the representative has been collected or not is the one that returns a representative to user code when needed, aka returns the result from the makeKind constructor or the existing representative if it exists.

So my question is, what code outside of BOYD and the "get representative" need to know about the collected state of a representative, and why?

[warner]

Related to #3649 (comment), I'm not sure I follow the current usage of WeakRefs. IMO those should be derefed only if we intend to use the target, not simply to sense if the target is collected or not, which is the job of the FinalizationRegistry.

In my mind, most code would consider the Representative to be available, until declared dead by bringOutYourDead. The only code that actually needs to bother if the representative has been collected or not is the one that returns a representative to user code when needed, aka returns the result from the makeKind constructor or the existing representative if it exists.

So my question is, what code outside of BOYD and the "get representative" need to know about the collected state of a representative, and why?

@mhofman recently taught me something about FinalizationRegistry that I didn't know: callbacks are cancelled if you unregister the object. So if object A is registered, then collected (nominally queueing a finalizer callback), then fr.unregister() is called, the JS spec guarantees that you won't see that callback invoked. I'd thought unregistration was a slight performance improvement, but I didn't realize it could be used to yank something out of the queue and ensure that a callback was not invoked.

We need to defend against the "re-introduction case" (A is imported and registered, A is dropped, A is collected, A gets reimported before the finalizer gets to run, now if the finalizer runs it might delete the live slotToVal entry). Because I didn't think queued finalizer callbacks could be cancelled, I had processDeadSet poll the slotToVal WeakRef and skip it if the entry had been reintroduced.

I think @mhofman is probably right and we can simplify the GC implementation. I need to think through the approach, though. We still have the same REACHABLE -> UNREACHABLE -> COLLECTED -> FINALIZED -> GONE sequence, but if a vref is reintroduced while in the COLLECTED state (i.e. after polling the WeakRef would return undefined, but before the finalizer callback has run), we can unregister the vref and then re-register it with the new one, and then we don't need to worry about the callback ever getting run.

But, let's see, there's still a gap between the finalizer running and bringOutYourDead getting run. During that time, the vref is sitting in deadSet (or possiblyDeadSet, @FUDCo is updating/merging them in this PR and the next few). If it gets reintroduced during that time, we need to remove it from the deadset. The goal would be to maintain the invariant that vrefs on the deadset do not have in-memory Object representatives (Presence, Remotable, or Representative). As @mhofman suggests, the ultimate goal would be to avoid ever polling the WeakRef.

I think we could do that, but.. how should that interact with e.g. a virtual object which just lost one of the other two pillars? The (abstract) virtual object is kept alive by any of three pillars: an in-memory Object, an export to the kernel, or a reference from virtualized data. Call these the Object pillar, the "export" pillar, and the refcount pillar. To maintain our RAM footprint goals, the latter two aren't allowed to be tracked in RAM, else our memory footprint would grow with the number of living virtual objects, whereas we need it to only grow with the number of in-use virtual objects (i.e. those represented by live Representatives), which should be just a small subset.

Implementation wise, the VOM will notice when e.g. the refcount pillar is gone (the refcount drops to zero), and that needs to move the VO state one step closer to being deleted. At that point we need to know the status of the Object pillar. However, since the data-pillar can disappear during normal userspace activity (e.g. deleting a key of some virtual object), the behavior it causes must not depend upon GC timing, which means it can't directly look at something changed by the finalizers. That argues against polling the WeakRef too. So I think the approach that @FUDCo and I settled upon was to add vrefs to possiblyDeadSet any time a pillar is dropped, but not look at possiblyDeadSet until the safe time (bringOutYourDead), when metering is disabled and we're allowed to be sensitive to GC. At that point, it's safe to poll the WeakRef, if that's what we use to sense the presence/absence of the Object pillar. Even if a virtual object is dropped (by refcount) in some deterministic userspace way, we won't actually delete the object (and anything it references) until the next bringOutYourDead.

So instead of polling the WeakRef at that point, I guess we could afford to have an in-memory Set of all the vrefs for which we know Representatives exist (probably Presences too). We'd add a vref to the set upon creation of the Presence/Representative, and we'd remove them when the finalizer fires, and we'd rely upon unregister to prevent the finalizer from firing after reintroduction. The Set would only be read during bringOutYourDead.

The thing I can't get my head around is the polarity of that set. If we're polling the WeakRef, then I can see the right architecture:

• finalizer adds vref to possiblyDeadSet (Object pillar has fallen)
• dispatch.dropExport adds vref to possiblyDeadSet and deletes the DB key that tracks the export (export pillar falls)
• VOM decref adds vref to possiblyDeadSet as it writes the refcount DB key (refcount pillar falls)
• re-creating the Object, re-exporting, incref all update their own data sources
• bringOutYourDead walks possiblyDeadSet and polls all three pillars for each
  • polls slotToVal and its WeakRef to sense the Object pillar
  • queries DB for export
  • queries DB for refcount
• virtual object is deleted if all three are false

But if we track a Set of deleted Objects (populated by finalizer, removed by Object re-creation), what's the right way to sense the Object pillar? From within bringOutYourDead, if we see the vref in this Set, then the Object pillar is down, great. But when should we remove it from the set? And if the vref is not in the set, does that mean 1: there's stlll an Object around (and necessarily in the WeakRef), or 2: the Object is gone but we've removed the vref from the Set because we don't think we need it anymore? My hunch is that the decision of when to remove the vref from the set must be based upon its presence in the other two pillars.

So without deeper analysis, it feels to me like the easiest thing to do is to poll the WeakRef during bringOutYourDead, and use it as the sole source of truth for the Object pillar. Which means it doesn't matter whether the finalizer runs multiple times or after a re-introduction: the not-actually-dead vref will be added to possiblyDeadSet, but we'll discover the reintroduction when we poll the WeakRef in bringOutYourDead, and the VO will remain alive and unexamined until the next time one of its pillars drops.

[FUDCo]

So without deeper analysis, it feels to me like the easiest thing to do is to poll the WeakRef during bringOutYourDead, and use it as the sole source of truth for the Object pillar. Which means it doesn't matter whether the finalizer runs multiple times or after a re-introduction: the not-actually-dead vref will be added to possiblyDeadSet, but we'll discover the reintroduction when we poll the WeakRef in bringOutYourDead, and the VO will remain alive and unexamined until the next time one of its pillars drops.

What you describe here is what is actually implemented modulo the unlanded PRs (i.e., it's what we'll have once these PRs are in). The dance is otherwise very delicate, and I'm nervous about error cases slipping through if we try to be too clever. The issue, to me, is that the finalizer is triggered by the GC of the in-memory object, but the finalizer itself operates on the vref, which might, at the time it runs, now refer to a different in-memory object -- this is one reason for the "possibly" part of the possiblyDeadSet name (aside from the other two pillars that might be keeping the VO alive). The thing I can't wrap my own head around is that even if unregister works the way Mathieu points out that it does, it's still not obvious to me how to know when such an unregistration should be performed, since the WeakRef gets cleared by GC when nobody is looking. In particular, a resurrection by definition happens after the WeakRef is cleared, hence we could not use the object itself as the unregister token, which means we'd need a whole 'nother piece of machinery to allocate and keep track of unregister tokens.

[warner]

Yeah, the unregister token should always be the (string) vref. We'd unregister it when deserialization needs to re-introduce the Object (i.e. when it polls slotToVal, finds a WeakRef, but the WeakRef is empty).

And the finalizer callback is still probably best place to delete the now-useless slotToVal entry. Our current callback does:

• 1: check slotToVal and the WeakRef
• 2: avoids doing anything if the WeakRef is now alive (i.e. reintroduction)
• 3: delete the useless slotToVal if the the WeakRef is dead (freeing the old WeakRef)
• 4: add the vref to the might-be-dead-set so that when bringOutYourDead/processDeadSet runs, it know that an Object pillar might be gone

If we switch to relying upon unregistration nullifying pending finalizer callbacks, I think the main thing that would change is step 2. That WeakRef could only be alive if the vref was reintroduced, and the reintroduction code would unregister the finalizer, so the callback would never run. Therefore the callback should never observe the WeakRef to be empty, and we could skip step two, assume the WeakRef is empty, and unconditionally delete slotToVal.

But, 1: that doesn't help us make metering more consistent (polling the WeakRef makes code GC-sensitive, but we're already dependent upon having finalizers only run inside the unmetered box, because the computrons used by the callback would also be GC-sensitive), and 2: it's a cheap thing to check.

So, if we can feel confident in getting .unregister right, we could change that "exit if wr.deref() into an assert(!wr.deref()).

[FUDCo]

the unregister token should always be the (string) vref

Unregister tokens are required to be objects, so this won't work.

[warner]

Unregister tokens are required to be objects, so this won't work.

Oh. hm.

[mhofman]

the unregister token should always be the (string) vref

Unregister tokens are required to be objects, so this won't work.

Ah yeah, that's partially my fault as I found a bug in v8 with undefined values, and @erights ended up confirming the general feel in the room to just disallow non-object tokens. tc39/proposal-weakrefs#71 (comment)

That said the answer to your pickle is probably also in that thread: use the weakRef object as unregister token ;)

Regarding the broader issue, I'm still having a hard time following, since I'm not familiar enough with the actual implemenation yet.

Do I understand correctly that export and refcount info are stored in persistent storage? And Object is stored in memory only.

Would the following work (pseudo code, not using existing code):

const vrefToWeakRef = new Map();
const registry = new FinalizationRegistry((vref) => {
  vrefToWeakRef.set(vref, null);
});

const getRepresentative = (vref) => {
  let rep;
  let weakRef = vrefToWeakRef.get(vref);
  if (weakRef) {
    rep = weakRef.deref();
    if (rep) {
      return rep;
    } else {
      registry.unregister(weakRef);
      vrefToWeakRef.set(vref, null);
    }
  }
  rep = makeRepresentative(vref);
  weakRef = new WeakRef(rep);
  registry.register(rep, vref, weakRef);
  vrefToWeakRef.set(vref, weakRef);
};

const checkHasPillar = (vref) => {
  if (!vrefToWeakRef.has(vref) && refcount === 0 && !hasExport) {
    prune(vref);
  }
}

const bringOutYourDead = () => {
  for (const [vref, weakRef] of vrefToWeakRef.entries()) {
    if (!weakRef) {
      vrefToWeakRef.delete(vref);
      checkHasPillar(vref);
    }
  }
};

That way:

• When finalization runs, we basically mark the representative as dead, but we defer processing any estate
• When derefing the weakref, we cancel the previous notification of death, and mark as dead (to be consistent with finalizer), but immediately create a successor (thus erasing the death)
• The processing of estate only happens at known point (bringOutYourDead), until then the object pillar stays standing
• During estate processing, we check if there are any other claims (either an export or refcount pillar), and prune the vref if none exist
• When the export or refcount pillar fall, they check if the object is known about. it could be alive, collected, finalized, it doesn't matter, as long as it's known, it is considered to "exist". when none exist, we prune

[warner]

That said the answer to your pickle is probably also in that thread: use the weakRef object as unregister token ;)

Interesting..

Do I understand correctly that export and refcount info are stored in persistent storage? And Object is stored in memory only.

Correct, because our resource requirement is that RAM usage can be O(N) in the number of live Objects, but must be O(1) in the number of non-live objects: virtual objects and imports must only consume RAM when something in JS has a handle on it, not when they're offline in virtualized data of some sort.

Would the following work (pseudo code, not using existing code):

Roughly, yes. The register.unregister(weakRef) on re-introduction should ensure that we never clobber an existing Object here:

const registry = new FinalizationRegistry((vref) => {
  vrefToWeakRef.set(vref, null);
});

because that callback should never get run after a re-introduction.

This loop:

const bringOutYourDead = () => {
  for (const [vref, weakRef] of vrefToWeakRef.entries()) {

is too expensive, we use a set of possibly-killed vrefs instead of walking the entire table.

Let's see, with this approach, the REACHABLE/UNREACHABLE/COLLECTED/FINALIZED state transition digram looks basically like:

• nothing: introduction, Object created -> REACHABLE
• REACHABLE: userspace drops, -> UNREACHABLE
• UNREACHABLE: reintroduction, WeakRef.deref() full, re-use existing Object, -> REACHABLE
• UNREACHABLE: GC, finalizer queued, -> COLLECTED
• COLLECTED: reintroduction, WeakRef.deref() empty, unregister, finalizer cancelled, make new Object, -> REACHABLE
• COLLECTED: finalizer called, delete WeakRef, -> FINALIZED

To make it work properly, we'd need to integrate some pieces from the existing code, like:

• the finalizer should add the vref to possiblyDeadSet, where bringOutYourDead will poll those entries
• the finalizer should delete the vrefToWeakRef entry (actually named slotToVal), rather than waiting for later
  • the other code should tolerate that entry being missing

But yeah, I think that handles the reintroduction part properly.

[mhofman]

is too expensive, we use a set of possibly-killed vrefs instead of walking the entire table.

Yeah I knew I was gonna get dinged on that, but figured it'd be easy to reintroduce a separate "deadSet", which has the same content as slotToVal.entries().filter(([vref, weakRef]) => !weakRef).map(([vref]) => vref).

• the other code should tolerate that entry being missing

To make sure:

• the code that checks for existence of any pillar will need to test for presence in both slotToVal (for non finalized objects), AND possiblyDeadSet (which IMO we can call deadSet), for finalized objects which haven't yet been processed by bringOutYourDead , and thus for which the object pillar is still considered to exist. Alternative is to have finalizer just add to deadSet, and have bringOutYourDead remove all entries of deadSet from slotToVal.
• getRepresentative (or equivalent) will need to remove any existing entry in deadSet.

Basically everywhere vrefToWeakRef/slotToVal above was touched, so will deadSet.

[mhofman]

because that callback should never get run after a re-introduction.

Btw, we should test that the engines we actually run under are not broken re this. Here is what I wrote a couple years ago (before the switch to individual callbacks and registry name change): https://github.com/mhofman/tc39-weakrefs-shim/blob/3cd6ff1910bc884937396ab12d59e1d4b55e2932/src/tests/FinalizationGroup.shared.ts#L534-L572

Idea is to register 2 cells in registry for the same object, with the unregister token of the other as heldValue. First one called back unregisters the other. Wait a bit (e.g. make sure another new object gets collected), and make sure the callback never gets called more than once.

Edit: It's an easy one to mess up, SpiderMonkey did before I reported it to them: https://bugzilla.mozilla.org/show_bug.cgi?id=1600488
Edit2: And I just realized that the test262 I contributed for this still uses cleanupSome so it won't run. I probably should go and fix those.

[warner]

Ok, looks pretty good but a couple of changes to make.

I reviewed this by locally rebasing 3547-vom-weak-key-gc onto current trunk, then rebasing this PR's vom-vprop-ref-counting on top of that, then reading the result. You'll need to do something similar. You might consider making the recommended changes on this branch, then land this PR onto its parent branch (closing this PR), then rebasing the parent branch to current trunk, then landing the other PR.

[warner]

I think this slotToVal is redundant: vrefs only get here if they were in deadSet, which starts the loop empty and only grows with things that passed the if !getValForSlot check earlier. Also/although the difference between slotToVal.get (which returns true if there's an empty WeakRef) and getValForSlot (which returns false) makes that not so clear cut. But I think that's not a difference this code is supposed to care about.

We do need to make sure the dead WeakRef is cleaned up somewhere. I see a relevant slotToVal.delete in retireOneExport, but I'm not sure we're doing the cleanup for imported Presences. I bet that should be added to the finalizer callback.

[FUDCo]

I don't think that's right. The !getValForSlot check several lines above tests that there is no in-memory object (i.e., it's been GC'd and finalized), so when we reach this point the test of slotToVal is telling us that even though the WeakRef is cleared, the WeakRef itself is still there, i.e., there's still a mapping from the vref that might need to be cleaned up, which is what this code does. It is important that the finalizer callback only add to possiblyDeadSet and not do any kind of cleanup, else we might introduce visible nondeterminism.

[warner]

I don't know that it's worth addressing now, but as a performance improvement, I think we don't need to repeat the gcAndFinalize unless doMore tells us that an in-memory reference (e.g. a key of VOM.remotableRefCounts) was dropped. If possiblyDeadSet/possiblyRetiredSet have grown, we need to rescan them, but we have no reason to think that the RAM object graph has changed, so we don't need to spend the time doing another full GC sweep.

To implement that, I guess we'd add a flag like gcNeedsCollection, set it to true upon entry to scanForDeadObjects, and at the top of the loop do

if (gcNeedsCollection) {
  await gcAndFinalize();
  gcNeedsCollection = false;
}

and if maybeFree is true, set both doMore and gcNeedsCollection.

[FUDCo]

Seems plausible. Sounds like something that could be easy to mess up, so not for this round of changes, but we should make a note...

[warner]

This needs to take responsibility for deleting the dead WeakRef, so something like:

if (wr) {
  if (!wr.deref()) {
    slotToVal.delete(vref);
    addToPossiblyDeadSet(vref);
  }
} else {
  addToPossiblyDeadSet(vref);
}

except maybe less ugly.

And we should do a pass after this lands to incorporate @mhofman 's recommendations on unregistering during reintroduction, which should reduce the number of cases this callback observes.

[FUDCo]

No, per my comment above, I think that would totally break it.

[FUDCo]

I appear to have introduced some kind of weird Git singularity.

[warner]

Ok, #3968 has my (small) recommended changes.

To remind myself, there are still two items I want to investigate after this lands:

• retireOneExport will probe the WeakRef and possibly delete the slotToVal entry. Does that interfere or interact at all with finalizeDroppedImport doing its job? Do we need to ensure that this entry-deletion happens in only one place?
• In the possiblyDeadSet loop, the !deadSet.has test seems weird. I want to think about what happens if something becomes both unreachable and unrecognizable by the time processDeadSet looks at it: might we fail to retire it at the right time? I know we'll loop back around if anything gets added to possiblyRetiredSet, but there might be a pathway that causes us to miss the one opportunity to retire the vref.
• I wanted to change addRecognizableValue to addRecognizableVref, to save a slotToVal lookup (because everywhere it gets called, it does a lookup first). But the actual vref isn't returned by that initial lookup (only a vatStore key derived from the vref), thwarting my tiny performance improvement. We might look into making those lookup functions return the vref too, to avoid the extra lookup, but at that point it probably isn't worth it.
• The nested for..of+map() loops in possibleVirtualObjectDeath could be cleaner if we could use two map() calls and some form of any() to reduce the list-of-list-of-booleans. But I'm not sure JS has the kind of any() utility that I'm used to in Python (Promise.any doesn't help, maybe some form of sum()?).

[warner]

r+ with the supplied PR landed