[BUG] Cannot convert undefined or null to object on unpublished package

[grassick]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

When doing npm install and there is a package installed from github that has the same name as an unpublished package in npm, the error "Cannot convert undefined or null to object" appears. See npm/metavuln-calculator#7 (comment) and npm/metavuln-calculator#12

The security audit causes a crash in npm. We cannot easily rename internally managed packages simply to avoid colliding with an unpublished npm package.

Expected Behavior

I expected the install to succeed (as it did in npm version 6).

Steps To Reproduce

1. Create a package.json that references a github repository with the same name as an unpublished package. e.g.

  "dependencies": {
    "mwater-forms": "github:mWater/mwater-forms"
  },

2. With latest npm
3. Run npm install
4. See error: "Cannot convert undefined or null to object"

Environment

• npm: 8.3.2
• Node.js: v16.13.2
• OS Name: Mint 20.3
• System Model Name: Dell?
• npm config:

; "user" config from /home/clayton/.npmrc

//registry.npmjs.org/:_authToken = (protected) 

; node bin location = /home/clayton/.nvm/versions/node/v16.13.2/bin/node
; cwd = /home/clayton/dev/scratch/failnpm
; HOME = /home/clayton
; Run `npm config ls -l` to show all defaults.

[booooza]

Same here for npm install <tarball file>. Works with npm install <tarball file> --no-audit though...

[Krinkle]

Same here.

30 verbose stack TypeError: Cannot convert undefined or null to object
30 verbose stack     at Function.keys (<anonymous>)
30 verbose stack     at Deprecate.exec (/usr/local/lib/node_modules/npm/lib/commands/deprecate.js:55:12)
30 verbose stack     at async module.exports (/usr/local/lib/node_modules/npm/lib/cli.js:66:5)
31 verbose cwd /Users/krinkle/Development/mediawiki/extensions/VisualEditor/lib/ve
32 verbose Darwin 19.6.0
33 verbose argv "/usr/local/Cellar/node/17.4.0/bin/node" "/usr/local/bin/npm" "deprecate" "[email protected]" "Install by other means: https://www.mediawiki.org/wiki/VisualEditor"
34 verbose node v17.4.0
35 verbose npm  v8.3.1

[cody-lettau]

Is there a plan to fix this at any point in the near future or should we be using --no-audit to get around it?

[eliot-akira]

I'm seeing the same error, Cannot convert undefined or null to object.

In my case, --no-audit doesn't change the behavior, I still see the same error.

Here's the debug stack trace, in case it's useful.

TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at module.exports (~/n/lib/node_modules/npm/node_modules/npm-pick-manifest/lib/index.js:213:22)
    at RegistryFetcher.manifest (~/n/lib/node_modules/npm/node_modules/pacote/lib/registry.js:125:22)
    at async Arborist.[nodeFromEdge] (~/n/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:1107:19)
    at async Arborist.[buildDepStep] (~/n/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:976:11)
    at async Arborist.buildIdealTree (~/n/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:218:7)
    at async Promise.all (index 1)
    at async Arborist.reify (~/n/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:154:5)
    at async Install.exec (~/n/lib/node_modules/npm/lib/commands/install.js:145:5)
    at async module.exports (~/n/lib/node_modules/npm/lib/cli.js:78:5)

What's interesting is that the error is coming from npm-pick-manifest in my case, but for someone else who commented earlier, the same error occurred in npm/lib/commands/deprecate.js. I checked both places, and the errors have the same reason: doing Object.keys(packument.versions) when the variable packument.versions is undefined or null.

https://github.com/npm/npm-pick-manifest/blob/2c9d25f97663b72253828cb540d598c35b8920a7/lib/index.js#L213

cli/lib/commands/deprecate.js

Line 57 in ca756fd

Object.keys(packument.versions)

---

EDIT: I found that it was happening because the project where I ran npm install had a non-existing npm module in its package.json. (Not relevant but it was ts-transformer-unassert which apparently doesn't exist anymore.) When I remove that line, the error no longer occurs.

The bug is that somewhere packument.versions is being set to null or undefined when there's a non-existing package, causing errors downstream in other places where that variable is expected to be an object.

[dillonthompson]

are there any plans to fix this issue? the issue isn't with the npm install or npm ci it's actually with the npm audit command which is running during the former commands and that's what is breaking. very frustrating.

[AhmedBHameed]

The interesting thing that it is happening to me in docker-container only!! If i run npm install outside the docker, then it works normally?!

[ryaa]

This bug is very annoying. Any plans to fix it?

[JockVanDallas]

This bug is preventing us to use npm audit in ci pipelines

[jamie-pate]

For me it seems like this is a lockfileVersion upgrade issue.. upgrading from 1 to 2 (node 14 to 16, npm v6.14.18 to v8.19.4)

When inspecting node --inspect $(which npm) install using vscode auto-attach <3
It looks like the actual error that it's trying to print is
No matching version found for <some library> where is
"ftnt-devops-ci": "https://github.com/fortinet/ftnt-devops-ci/releases/download/1.1.7/ftnt-devops-ci-1.1.7.tgz",

This helps print the correct error:
/.nvm/versions/node/v16.13.2/lib/node_modules/npm/node_modules/npm-pick-manifest/index.js

-    versions: Object.keys(packument.versions),
+    versions: Object.keys(packument.versions || {}),

This worked for me to upgrade the package-lock.json from v1 to v2, replace ftnt-devops-ci... with your problem package.. maybe this can be shortened but i've already spent too long on this :D

 git checkout package-lock.json # restore your original package-lock
nvm use 16
npm install https://github.com/fortinet/ftnt-devops-ci/releases/download/1.1.7/ftnt-devops-ci-1.1.7.tgz
nvm use 14
npm install https://github.com/fortinet/ftnt-devops-ci/releases/download/1.1.7/ftnt-devops-ci-1.1.7.tgz
nvm use 16
npm install

[JPeer264]

We had the problem with a slightly different use case. For us we didn't had the issue with an unpublished package, but rather with using npm: as prefix. It can be reproduced by using following package.json and the, to date, latest npm@8 (v8.19.4):

{
  "name": "npm-bug",
  "dependencies": {
    "trim": "0.0.2",
    "my-trim": "npm:[email protected]"
  }
}

It has the same symptoms that it fails at the exact same position (and also won't fail when running npm i --no-audit). Appearantly it got fixed with [email protected] or to be more precise with @npmcli/[email protected]: 14c498d. I am not quite sure about the release process of npm, but this commit looks like it can be taken 1:1 to npm@8.

I would say this issue is related to this PR #6363 and this issue as well #5110