Update usage information metrics for v2024.3

[matthew-white]

Metrics to add

• How many offline updates have been force-processed?
  • Should this include an offline update force-processed before the offline create? I.e., an offline update applied as a create?
    • Matt thinks ideally it would.
  • Moved from Update usage information metrics for v2024.2 #653
• How long does it take for an entire branch to be processed? Specifically: what is the max amount of time between when the first submission for a branch is received and when the last submission is processed?
  • Exclude branches where one of the updates was force-processed. We're more interested in the more normal case where a submission is put in the backlog just long enough for updates to be processed in the correct order. In other words, we expect the max amount of time to be less than 5 days (hopefully much less!).
  • Moved from Update usage information metrics for v2024.2 #653
• How many form definitions are XML-only and are not associated with an XLSForm?
  • When managed encryption is enabled for a project, Central will autogenerate new XML-only form definitions. It's a bug that XLSForms aren't carried forward (#729), but we don't want the bug to affect this metric. To work around the bug, we could exclude any form definition that uses encryption and whose version string contains [encrypted:. It'd also be OK just to exclude all form definitions that use encryption.
  • We may end up double-counting some autogenerated form definitions, e.g., if there's an XML-only form definition, and then we create a new version of it as part of #692. In that case, the user only uploaded a single XML-only form definition even though their database says that there are two. I think that level of double-counting is OK though. If we wanted to exclude form definitions created as part of #692, we could exclude definitions whose version string ends with [upgrade].
• Does the server use S3-compatible file storage? (1/0)
• Number of binary files (number of rows of the blobs table)
• Number of binary files currently stored in S3
• How many times has the user run reset-failed-to-pending for S3? (How many blobs.s3.failed-to-pending events are there in the audits table?)
• We break down entity updates by source (via submission vs. API). However, we don't break down entity creates in the same way: we just report the total number of entities. Let's add a few related metrics:
  • Number of Entities created via Submission
  • Number of Entities created via bulk upload
  • Number of Entities created via API

Metrics to remove

• Project has a description

Checklist

• Update the form version string in the Backend config.
• Publish the new form version on data.getodk.cloud.
• Update the usage report in Frontend.

Extra notes

• Is this the time to look into improving how the analytics queries are run? Query fewer analytics at once central-backend#1228

[ktuite]

I'm planning to add most of these at the top level of the metrics (and with no recent vs. total breakdown) because I feel like they are more operations-related and I put some of the existing offline entity metrics there already.

• num_offline_entity_submissions_force_processed (similar to num_offline_entity_submissions_reprocessed, compute via the same query)
• max_entity_branch_delay
• num_xml_only_form_defs (top level? don't care about per-project or recent/new?)
• uses_external_blob_store
• num_blob_files
• num_blob_files_on_s3
• num_reset_failed_to_pending_count
• datasets
  • num_entity_creates_sub (recent + total)
  • num_entity_creates_api (recent + total)
  • num_entity_creates_bulk (recent + total)

[matthew-white]

That breakdown looks good to me. 👍