JavaScript heap out of memory

[yashsway]

Describe the bug
When running jest with --env set to @happy-dom/jest-environment to run 812 tests (144 test suites), at the 81st test suite mark, the process runs out of memory. This issue does not occur in the previous configuration when running tests with jsdom.

To Reproduce
Steps to reproduce the behavior:

1. Run 800 or more jest tests with jest test --env=@happy-dom/jest-environment --maxWorkers=2 --silent
2. Wait for a stack trace error

<--- Last few GCs --->

[56985:0x7f9480040000]    92441 ms: Scavenge (reduce) 3954.2 (4105.1) -> 3954.0 (4105.6) MB, 2.3 / 0.0 ms  (average mu = 0.816, current mu = 0.679) allocation failure; 
[56985:0x7f9480040000]    93506 ms: Mark-sweep (reduce) 4028.9 (4180.3) -> 4001.1 (4160.4) MB, 651.8 / 0.0 ms  (average mu = 0.730, current mu = 0.647) allocation failure; GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10681ffb5 node::Abort() (.cold.1) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 2: 0x1052a0a29 node::Abort() [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 3: 0x1052a0c0e node::OOMErrorHandler(char const*, bool) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 4: 0x10542fcc3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 5: 0x1055f8975 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 6: 0x1055f7352 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 7: 0x1055e968a v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 8: 0x1055ea005 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
 9: 0x1055cbc4a v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
10: 0x1055c3674 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
11: 0x1055c4076 v8::internal::FactoryBase<v8::internal::Factory>::NewObjectBoilerplateDescription(int, int, int, bool) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
12: 0x105467d21 void v8::internal::ObjectLiteralBoilerplateBuilder::BuildBoilerplateDescription<v8::internal::Isolate>(v8::internal::Isolate*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
13: 0x1056f0e1a void v8::internal::interpreter::BytecodeGenerator::AllocateDeferredConstants<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Script>) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
14: 0x1056f0aef v8::internal::Handle<v8::internal::BytecodeArray> v8::internal::interpreter::BytecodeGenerator::FinalizeBytecode<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Script>) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
15: 0x105715f7c v8::internal::CompilationJob::Status v8::internal::interpreter::InterpreterCompilationJob::DoFinalizeJobImpl<v8::internal::Isolate>(v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Isolate*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
16: 0x105715dda v8::internal::interpreter::InterpreterCompilationJob::FinalizeJobImpl(v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Isolate*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
17: 0x1054e9362 v8::internal::CompilationJob::Status v8::internal::(anonymous namespace)::FinalizeSingleUnoptimizedCompilationJob<v8::internal::Isolate>(v8::internal::UnoptimizedCompilationJob*, v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Isolate*, std::__1::vector<v8::internal::FinalizeUnoptimizedCompilationData, std::__1::allocator<v8::internal::FinalizeUnoptimizedCompilationData>>*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
18: 0x1054e233f bool v8::internal::(anonymous namespace)::IterativelyExecuteAndFinalizeUnoptimizedCompilationJobs<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Handle<v8::internal::Script>, v8::internal::ParseInfo*, v8::internal::AccountingAllocator*, v8::internal::IsCompiledScope*, std::__1::vector<v8::internal::FinalizeUnoptimizedCompilationData, std::__1::allocator<v8::internal::FinalizeUnoptimizedCompilationData>>*, std::__1::vector<v8::internal::DeferredFinalizationJobData, std::__1::allocator<v8::internal::DeferredFinalizationJobData>>*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
19: 0x1054e1e99 v8::internal::Compiler::Compile(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*, v8::internal::CreateSourcePositions) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
20: 0x1054e26d1 v8::internal::Compiler::Compile(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
21: 0x1059fa66c v8::internal::Runtime_CompileLazy(int, unsigned long*, v8::internal::Isolate*) [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]
22: 0x105df7db9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/yashkadaru/.asdf/installs/nodejs/18.15.0/bin/node]

Expected behavior
All the tests should run with a pass/fail.

Device:

• MacOS Ventura 13.5.2, 32 GB 2667 MHz DDR4, Mac Pro 16in 2019
• Node v18.15.0 through asdf
• pnpm v8.0.0

Additional context
n/a

[capricorn86]

Thank you for reporting @yashsway! 🙂

The stack trace doesn't seem to say anything about what caused it inside Happy DOM, so it's impossible to reproduce based on that. I don't think the amount of tests is the reason.

Do you have a repo or some reproducible example?

[yashsway]

@capricorn86 I'm glad to help!

ah bummer :( I can't share the repo unfortunately. Hmm I'll have to figure out how to reproduce this separately. The only parameter I changed between running the tests one way or another was swapping out jsdom for happy-dom so I was sure that was the issue but maybe it's something else. I'll get back to you!

[capricorn86]

@yashsway I'm a bit worried about the memory leak.

Would it be possible to run a profiler on your tests?

1. Run node --inspect-brk ./node_modules/.bin/jest --env=@happy-dom/jest-environment --runInBand
2. Open DevTools in Chrome
3. Click on the green NodeJS icon
4. Go to source and select Play to make it continue
5. Go to Memory and run a profiler when you notice the memory starts go up, but doesn't go down again (it probably has to go over at least 1GB)

[capricorn86]

Another thing you can try is to look at tests that was running when it happened and try to run them in isolation.

I don't remember how it looks in Jest, but Vitest shows a list of all current tests and which ones that are running.

[olekshyr]

@capricorn86
Hi there, facing the same issue, I've made a screenshot of the JS heap. This is after about 10 test suites I guess. It was always rising and never had the chance to drop as it hit 4GB.

[illandril]

@capricorn86 I've created a repo to reproduce a leak: https://github.com/illandril/happy-dom-leak

My reproduction uses @testing-library/react, so it might only be some interaction between happy-dom and testing-library that leads to the leak (I haven't yet had time to investigate deeply to figure out specifically what is causing the leak, only that the exact same setup but swapping out happy-dom with jsdom works w/o a leak).

npm run test:d:happy and npm run test:d:jsdom runs the tests with happy-dom and jsdom respectively with the --detectLeaks flag - jest's leak detector complains for happy-dom but not jsdom.

npm run test:l:happy and npm run test:l:jsdom runs the tests with happy-dom and jsdom respectively with the --logHeapUsage flag - on my device, heap size starts at 38 MB for happy-dom and grows by ~2MB per test file, ending at 83 MB; for jsdom, heap starts higher (46 MB), but it levels out quickly at a peak of ~55 MB.

% npm run test:d:jsdom

> [email protected] test:d:jsdom
> node --expose-gc ./node_modules/.bin/jest --runInBand --detectLeaks --test-environment=jest-environment-jsdom

 PASS  src/z.test.tsx
...
 PASS  src/q.test.tsx

Test Suites: 26 passed, 26 total
Tests:       26 passed, 26 total
Snapshots:   0 total
Time:        6.724 s
Ran all test suites.

% npm run test:d:happy

> [email protected] test:d:happy
> node --expose-gc ./node_modules/.bin/jest --runInBand --detectLeaks --test-environment=@happy-dom/jest-environment

 FAIL  src/z.test.tsx
  ● Test suite failed to run

    EXPERIMENTAL FEATURE!
    Your test suite is leaking memory. Please ensure all references are cleaned.

    There is a number of things that can leak memory:
      - Async operations that have not finished (e.g. fs.readFile).
      - Timers not properly mocked (e.g. setInterval, setTimeout).
      - Keeping references to the global scope.

      at onResult (../node_modules/@jest/core/build/TestScheduler.js:150:18)
      at ../node_modules/@jest/core/build/TestScheduler.js:254:19
      at ../node_modules/emittery/index.js:363:13
          at Array.map (<anonymous>)
      at Emittery.emit (../node_modules/emittery/index.js:361:23)
...
Test Suites: 26 failed, 26 total
Tests:       0 total
Snapshots:   0 total
Time:        6.139 s

[illandril]

It looks like this has been fixed (at least for the project I was seeing this in) somewhere between 14.12.3 and 15.7.3 (most likely in all the refactoring that was done in 15.0.0, but I didn't go back and check each version in between to verify which one specifically fixed the issue)

[karpiuMG]

@yashsway please check the issue using the latest happy-dom. If issue doesn't exist anymore please close it