Code coverage issues, performance and discrepancies

[Blebowski]

A follow-up of: #1150

I see that the run-time of simple test (the device_id as in the previous issue) differs a lot when coverage
is enabled vs. the case when it is disabled (e.g. by factor of 4).

Majority of time is consumed due to the elaboration. When coverage is enabled, the code is always generated
AOT. E.g. progress bar with coverage:

** Note: initialising [9ms +53600kB]
** Note: loading top-level unit [17ms +6192kB]
** Note: elaborating design [3921ms 1.1x +1326692kB]
** Note: dumping coverage data [15ms +160kB]
** Note: code generation for 9641 units [12342ms 4.6x +1583648kB]
** Note: linking shared library [566ms +0kB]

vs without coverage:

** Note: initialising [9ms +53760kB]
** Note: loading top-level unit [18ms +6196kB]
** Note: elaborating design [3850ms 1.1x +1312896kB]
... simulation already starts

In my case DEFAULT_JIT=true. I tried to modify nvc.c not to enforce AOT generation when coverage is enabled,
and with coverage and AOT not enforced, the elab time with coverage is more-less similar as without coverage.
Is there a reason why with coverage AOT is enforced ?

I am surprised by such huge difference, most of the code for the TB needs to be generated anyway for the TB.
Maybe the difference is the code for those huge constant arrays reference_data_set_* that are not used in
this test, so that is where the savings are from (we discussed this in VUnit/vunit#1036).

There is one other issue I get. When running without coverage (therefore with JIT), I get simulation log flooded
with:

** Warning: 0ms+0: NUMERIC_STD.TO_INTEGER: metavalue detected, returning 0
   Function TO_INTEGER [UNRESOLVED_UNSIGNED return NATURAL] at ../lib/ieee.08/numeric_std-body.vhdl:3025
   Process :tb_top_ctu_can_fd:dut:rx_buffer_inst:rx_buffer_ram_inst:rx_buf_ram_inst:_p0 at /WORK/ctu_can_fd_clean/src/common_blocks/inf_ram_wrapper.vhd:191

at many simulation times. This is expected since I drive DUT inputs to X besides SETUP+HOLD around
the edge when DUT should sample them, so the metavalues do propagate to many places. I know I can disable this
with --ieee-warnings. What is strange is that I do not get these warnings fired when the coverage is enabled
(and code is generated AOT). I would expect equal behavior regardless of the compilation mode.

[nickg]

Is there a reason why with coverage AOT is enforced ?

No, it's a mistake. I've fixed it.

I am surprised by such huge difference, most of the code for the TB needs to be generated anyway for the TB.

Well not really. The slow part is LLVM and in the JIT mode that only gets invoked when the code is executed frequently enough to be worthwhile. This simulation is so short that most of it can run just fine in the interpreter.

[Blebowski]

Thanks for looking into this.

Did you manage to reproduce the issue with logs ?

[nickg]

Did you manage to reproduce the issue with logs ?

Yes but it needs VUnit/vunit#1098.