dts creation impossibly slow.

[datner]

I've been trying to migrate some project from unbuild to tsup.
Theres nothing special about it, it's not that big either.

unbuild was pretty fast, with 2-3s to chew the whole thing. dts included.
I've moved to tsup. bundling was amazingly fast, less than 60ms. But dts takes 5 seconds.
I do not resolve, it's a plain dts: true.

I've looked at the rollup-plugin-dts and saw that they the following:

The plugin works best with already existing .d.ts files generated by the typescript compiler from idiomatic code.

Ok, so lets do that. I run tsc with declaration: true, without a declaration map. It takes around 1s and done.
Now tsup takes around 20s to run dts before saying it's out of memory.

I thought there might be some multiple entry points shenanigans, so I tried reducing to only 1 entry point.
dts with no prior tsc - 1s. Relatively slow, but fine.
dts with tsc - crash after 20s. OOM.
dts with tsc only on the entry file - now dts takes 11s but resolves.... with the same exact result as it would if I haven't generated the d.ts beforehand.

How can this be? Why would it make things worse??
How can unbuild do it so fast with the same exact plugin, less optimized??? (they don't disable anything like tsup does)

I know this is a general typescript issue, but somehow tsup turns a slow operation into a full-on odyssey.
Possibly related issues: #840

I tried to work on this on my own, but I couldn't find the source of the issue.

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[coopbri]

Hey @datner, did you ever find a solution to this? I am encountering the same issue now that my project has reached a certain size and too am running into a (flaky, but always slow) out-of-memory error when enabling .d.ts generation via tsup.

[datner]

@coopbri I have not, I went down a rabbit hole of horrors and became a tsc hermit. I no longer wish to fight with bundling. I now use the vanilla tsc compiler for all my libs, only reaching for a simple babel transpilation step if I need to have both esm and cjs, like seen in this repo or a simple rollup config. I write my code in such a way that it's completely tree-shakeable and only need to work with my main app(s) bundlers which are a pain enough.
Funny enough, my builds now take 1-2s and have a smaller size after tree shaking, so it's a win

One day the pain will go away....

[coopbri]

Haha, no worries man! Thanks for detailing your process. I think I'll do much of the same for now (tsc hermit as you eloquently put 😄).

[datner]

@coopbri word of advice, set up self-imports if you want to bundle to esm and cjs since esm requires you used named imports or full paths (with extensions ) and cjs wants named imports and forbids extensions

import * as Foo from './Foo.js' // <-- breaks cjs
import * as Foo from './Foo' // <-- breaks esm
import * as Foo from "my-lib/Foo" // works for both

[coopbri]

Geat tip! Thank you!!

[coopbri]

@datner you may have seen this already, but I'm keeping a close eye on it: https://github.com/dudykr/stc (some discussion at swc-project/swc#657)

As an avid SWC fan, I'm really excited for this!

[datner]

@coopbri I'm quite satisfied with the performance of tsc surprisingly. But a better tsserver, especially one that actually conforms to the lsp protocol, is something I yearn for.

That said, I also like swc and have seen this project. It has nothing to see so I didn't know it was usable, I'll give it another glance. Thanks!

[maelp]

I have the same issue, dts does an out-of-memory, although it's a quite simple expressjs server where tsc has no issue generating types...

[tysonclugg]

This might be somewhat related to #920 (comment) and may have been fixed upstream in Node v21 and above, particularly if the heap was allocated using slower memory pages from swap. Can someone please test and confirm if this is the case?

[rawnly]

Hey, I just figure out a new solution to address the memory/slowlyness problem. Instead of using the standard tsup config, we use the javascript api with a custom glob in order to chunk the source files, we use the regular tsup build for the build, then for .d.ts we chunk the components into blocks of 15 elements.

import fs from "node:fs/promises";
import { glob } from "glob";
import { build } from "tsup";

function chunkArray(array, size) {
  const chunked_arr = [];
  let index = 0;
  while (index < array.length) {
    chunked_arr.push(array.slice(index, size + index));
    index += size;
  }
  return chunked_arr;
}

async function main() {
  try {
    await fs.rmdir("dist");
  } catch {
    console.warn("No dist folder found");
  }

  const files = await glob("src/components/*/index.tsx", {
    withFileTypes: true,
    ignore: [
      "src/**/*.test.{ts,tsx}",
      "src/**/*.spec.{ts,tsx}",
      "src/**/*.stories.{ts,tsx}",
    ],
  });

  const components = files.map((p) => p.parent?.name);

  console.log("[1/2] Building components");

  await build({
    format: ["esm"],
    dts: false,
    clean: true,
    sourcemap: true,
    target: "es2022",
    external: ["react", "react/jsx-runtime", "react-dom"],
    treeshake: true,
    minify: true,
    tsconfig: "tsconfig.build.json",
    entry: [
      "src/index.ts",
      "src/**/*@(ts|tsx)",
      "!playwright",
      "!src/**/*.spec.*",
      "!src/**/*.test.*",
      "!src/__tests__/**/*",
      "!src/**/*.stories.*",
      "!fixture",
    ],
  });

  console.log(
    `[2/2] Building declaration files for ${components.length} components`,
  );

  const matrix = chunkArray(components, 15);

  for (const group of matrix) {
    await build({
     // same config as the precedent one
      dts: {
        only: true,
      },
      tsconfig: "tsconfig.build.json",
      entry: group.flatMap((dir) => [
        `src/components/${dir}/**/*.{tsx,ts}`,
        `!src/components/${dir}/*.test.{tsx,ts}`,
        `!src/components/${dir}/*.spec.{tsx,ts}`,
        `!src/components/${dir}/*.stories.{tsx,ts}`,
        "!fixture",
      ]),
    });
  }

}

main();