(2.11) ADR-44: JetStream Asset Versioning in Metadata

[MauriceVanVeen]

Implements ADR-44

Initial addition of JetStream Asset Versioning, containing:

• generating stream and consumer metadata
  • created server version: _nats.created.server.version
  • created server API level: _nats.created.server.api_level
  • required server API level: _nats.server.require.api_level
• logging supported API level upon startup: [INF] API Level: 1

Note that stream and consumer metadata is only set/updated upon:

• creating a new stream/consumer
• updating a stream/consumer
  (created metadata will not be set for pre-existing assets, only required level will be updated)

Many tests are added ensuring that:

• restoring streams from backups doesn't set this metadata
  (if it doesn't exist, for example restoring a backup from a previous version)
• restarting a server (or upgrading) doesn't set this metadata
  (if it doesn't exist, for example due to upgrading)
• updating consumer PauseUntil ups or lowers required API level
• metadata is consistently reported through add, update and info requests
• only stream/consumer/meta leader determine the metadata, so that there can be no skew in metadata if followers were allowed to update as well
• users can't manually supply these _nats.> metadata fields, they will be overwritten to the appropriate values
• if the metadata is missing during an update, the metadata is preserved
  (for example an update by a client that doesn't know about metadata yet)

These tests check for both non-clustered R1 setups and clustered R3 setups.

This PR doesn't fully implement ADR-44. There will be follow-up PRs later on to add support for:

• dynamic metadata, like _nats.server.version and _nats.server.api_level that report the current server version and API level the asset lives on:
  (2.11) ADR-44: JetStream Dynamic Metadata #5857
• reporting server JS API level through jsz, varz and $JS.API.INFO (it is already reported in the logs during startup):
  (2.11) ADR-44: JetStream API Level in varz,jsz,statsz,$JS.API.INFO #5855
• etc.

There are some slight inconsistencies between the ADR and this PR. For example using the terms feature level, api_version and api_level that all mean the same and are all made consistent to be api_level. I'll correct the ADR with any changed details after this PR has been merged.

Signed-off-by: Maurice van Veen [email protected]

[ripienaar]

Behaviour wise this is pretty spot on

[derekcollison]

Do we need binary artifacts (the snapshots) store in GH? Can we just create them on the fly for the test?

Prefer not to have binary assets checked into GH if we can help it.

[derekcollison]

LGTM