OSXFS specific issue #2296

alcohol · 2017-12-07T12:53:51Z

HOST

uname -a
Darwin macbookpro.local 17.2.0 Darwin Kernel Version 17.2.0: Fri Sep 29 18:27:05 PDT 2017; 
  root:xnu-4570.20.62~3/RELEASE_X86_64 x86_64 i386 MacBookPro11,4 Darwin

docker --version
Docker version 17.11.0-ce, build 1caf76c

FILES

package.json for reproduction:

{
  "name": "test",
  "private": true,
  "devDependencies": {
    "cucumber": "2.3.1"
  }
}

Dockerfile for reproduction:

FROM alpine:3.7

RUN apk add --no-cache --update git tini nodejs=8.9.1-r0

WORKDIR /workspace

ENTRYPOINT ["/sbin/tini", "--"]

REPRO

Repro steps host:

docker build --tag test-image .
docker run --rm -it -e HOME=/tmp -w /workspace -v $(pwd):/workspace test-image sh

Repro steps inside container:

npm install --verbose

OUTCOME

npm info it worked if it ends with ok
npm verb cli [ '/usr/bin/node', '/usr/bin/npm', 'install', '--verbose' ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session 8b372084f9346ee1
npm info lifecycle test@~preinstall: test@
npm http fetch GET 200 https://registry.npmjs.org/cucumber 128ms

...snipped...

npm info linkStuff [email protected]
npm verb linkBins [email protected]
npm verb linkBins [ { 'cucumber.js': './bin/cucumber.js',
npm verb linkBins     'cucumber-js': './bin/cucumber.js',
npm verb linkBins     cucumberjs: './bin/cucumber.js' },
npm verb linkBins   '/workspace/node_modules/.bin',
npm verb linkBins   false ]
npm verb linkMans [email protected]
npm verb unlock done using /tmp/.npm/_locks/staging-afa1b9ac6beb89c7.lock for /workspace/node_modules/.staging
npm verb stack Error: ENOENT: no such file or directory, chmod '/workspace/node_modules/cucumber/bin/cucumber.js'
npm verb cwd /workspace
npm verb Linux 4.9.60-linuxkit-aufs
npm verb argv "/usr/bin/node" "/usr/bin/npm" "install" "--verbose"
npm verb node v8.9.1
npm verb npm  v5.5.1
npm ERR! path /workspace/node_modules/cucumber/bin/cucumber.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/workspace/node_modules/cucumber/bin/cucumber.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm verb exit [ -2, true ]

npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/.npm/_logs/2017-12-06T16_40_19_127Z-debug.log

EXPECTED

Cucumber gets installed.

NOTES

Also applies to 1.3.3, 2.1.0, 2.2.0, 2.3.0 (did not test lower than this).

Version 3.1.0 does not seem to have this issue.

The text was updated successfully, but these errors were encountered:

matr369 · 2018-01-10T08:58:58Z

The same for me on linux machine. But sometimes it works.

nbigaouette-eai · 2018-01-16T16:57:53Z

Same kind of error here working with serverless (1.25.0).

uname -a
Darwin m.local 17.3.0 Darwin Kernel Version 17.3.0: Thu Nov  9 18:09:22 PST 2017; root:xnu-4570.31.3~1/RELEASE_X86_64 x86_64
docker --version
Docker version 17.12.0-ce, build c97c6d6

Minimal (non-working) example using docker:

mkdir docker-issue-2296
cd docker-issue-2296
docker run -it --rm -v $(PWD):/cwd -w /cwd node:9.4.0-alpine npm init

Accepting all default values, resulting in following package.json:

{
  "name": "cwd",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

Then install fails:

docker run -it --rm -v $(PWD):/cwd -w /cwd node:9.4.0-alpine npm install --verbose serverless
npm info it worked if it ends with ok
npm verb cli [ '/usr/local/bin/node',
npm verb cli   '/usr/local/bin/npm',
npm verb cli   'install',
npm verb cli   '--verbose',
npm verb cli   'serverless' ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session ae591a1f8d148d0f
npm http fetch GET 200 https://registry.npmjs.org/serverless 138ms
[...]
npm info linkStuff [email protected]
npm verb linkBins [email protected]
npm verb linkBins [ { serverless: './bin/serverless',
npm verb linkBins     slss: './bin/serverless',
npm verb linkBins     sls: './bin/serverless' },
npm verb linkBins   '/cwd/node_modules/.bin',
npm verb linkBins   false ]
npm verb linkMans [email protected]
npm verb unlock done using /root/.npm/_locks/staging-09696d3793f12899.lock for /cwd/node_modules/.staging
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

npm verb stack Error: ENOENT: no such file or directory, chmod '/cwd/node_modules/serverless/bin/serverless'
npm verb cwd /cwd
npm verb Linux 4.9.60-linuxkit-aufs
npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--verbose" "serverless"
npm verb node v9.4.0
npm verb npm  v5.6.0
npm ERR! path /cwd/node_modules/serverless/bin/serverless
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/cwd/node_modules/serverless/bin/serverless'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm verb exit [ -2, true ]

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-01-16T16_44_31_237Z-debug.log

The node_modules directory is empty:

tree -a node_modules/
node_modules/
└── .bin
    ├── esparse -> ../esprima/bin/esparse.js
    ├── esvalidate -> ../esprima/bin/esvalidate.js
    ├── js-yaml -> ../js-yaml/bin/js-yaml.js
    ├── json-refs -> ../json-refs/bin/json-refs
    ├── loose-envify -> ../loose-envify/cli.js
    ├── mime -> ../mime/cli.js
    ├── mkdirp -> ../mkdirp/bin/cmd.js
    ├── raven -> ../raven/bin/raven
    ├── rc -> ../rc/cli.js
    ├── rimraf -> ../rimraf/bin.js
    ├── seek-bunzip -> ../seek-bzip/bin/seek-bunzip
    ├── seek-table -> ../seek-bzip/bin/seek-bzip-table
    ├── semver -> ../semver/bin/semver
    ├── serverless -> ../serverless/bin/serverless
    ├── sha.js -> ../sha.js/bin.js
    ├── sls -> ../serverless/bin/serverless
    ├── slss -> ../serverless/bin/serverless
    ├── tabtab -> ../tabtab/bin/tabtab
    └── which -> ../which/bin/which

1 directory, 19 files

Uploading:

a gzipped version of the full log as serverless.log.gz: serverless.log.gz
a gzipped version of npm's own log:
2018-01-16T16_54_05_430Z-debug.log.gz

nbigaouette-eai · 2018-01-16T17:09:26Z

Interestingly enough, it works with a different docker image (using node v8.9.4):

rm -fr node_modules package-lock.json
docker run -it --rm -v $(PWD):/cwd -w /cwd node:8.9.4-alpine npm install serverless

> [email protected] postinstall /cwd/node_modules/spawn-sync
> node postinstall


> [email protected] postinstall /cwd/node_modules/serverless
> node ./scripts/postinstall.js

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
added 307 packages in 53.39s

EDIT: Talked to quickly; it now fails!

pcambra · 2018-04-13T04:05:40Z

I'm getting the same thing using d3 with docker for mac:

test | npm ERR! path /app/node_modules/d3-dsv/bin/dsv2dsv
test | npm ERR! code ENOENT
test | npm ERR! errno -2
test | npm ERR! syscall chmod
test | npm ERR! enoent ENOENT: no such file or directory, chmod '/app/node_modules/d3-dsv/bin/dsv2dsv'
test | npm ERR! enoent This is related to npm not being able to find a file.
test | npm ERR! enoent

Sometimes it fails on a different file:

 test | npm ERR! enoent ENOENT: no such file or directory, chmod '/app/node_modules/d3-dsv/bin/json2dsv'

npm ERR! enoent ENOENT: no such file or directory, chmod '/app/node_modules/d3-dsv/bin/dsv2json'

chipironcin · 2018-04-18T11:09:08Z

Same happens with cucumber@^3.2.0 in a Debian container running in MacOS High Sierra 10.13.4 with Docker version 18.03.0-ce

# npm install cucumber@^3.2.0
npm WARN enoent ENOENT: no such file or directory, open '/myVolume/projects/test/package.json'
npm WARN test No description
npm WARN test No repository field.
npm WARN test No README data
npm WARN test No license field.

npm ERR! path /myVolume/projects/test/node_modules/cucumber/bin/cucumber.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/myVolume/projects/test/node_modules/cucumber/bin/cucumber.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-04-18T11_04_59_446Z-debug.log

Docker info:

$ docker info
Containers: 6
 Running: 1
 Paused: 0
 Stopped: 5
Images: 21
Server Version: 18.03.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.87-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: FXOM:7AVH:WCUR:ISF3:ZZU7:5J3U:H3LN:EQ46:B65I:PTRW:7SGC:M7AC
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 42
 Goroutines: 61
 System Time: 2018-04-18T11:07:45.0285239Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

matr369 · 2018-05-23T06:57:34Z

try to use "npm install --unsafe-perm"

SalomonSmeke · 2018-05-24T20:24:35Z

+1 here. npm install --unsafe-perm didn't help.

:consistent didn't help.

gurucleff · 2018-08-13T20:43:37Z

any updates on this issue?

tomwidmer · 2018-08-24T13:25:28Z

I have a workaround for this: as well as the volume mount of the local project, additionally mount a named volume to the node_modules folder, so it isn't shared with the host (it isn't really desirable to share node_modules between the OS and the container anyway, since the OS is different). E.g. in my docker-compose file I have something like:

services:
  my-service:
    image: my-service:local
    build: ../my-service
    volumes:
      - ../my-service:/var/app
      - my_service_node_modules:/var/app/node_modules
volumes:
  my_service_node_modules:

Should also help with performance.

gsong · 2018-08-24T13:36:43Z

@tomwidmer When I tried this, I saw some weird behaviors when node_modules is also present in the host. Are you operating exclusively using Docker containers?

tomwidmer · 2018-08-27T17:12:51Z

@gsong I mostly just operate in the container - I use docker-compose exec <service> bash before doing any npm commands. However, I do run npm install outside the container to get the node_modules in place outside the container to help my IDE. Another option would be to copy node_modules out of the container and into the host folder manually with docker cp after installing/updating any modules.

I'd prefer a fix to the underlying problem of course.

cydergoth · 2018-09-10T16:22:54Z

+1 Same issue

Darwin JCrisp-MBP-TX.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

Docker version 18.06.0-ce, build 0ffa825

docker-robott · 2018-12-09T02:00:24Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

Zodiase · 2018-12-09T07:53:15Z

@docker-for-desktop-robot WTH?

Vanuan · 2018-12-20T19:17:28Z

Didn't understand what are those Mac users are complaining about. Got a MacBook, now I have this issue too :(

And I'm using yarn, not npm.
It looks like osxfs return NOENT if it can't keep up with firehose of files created by npm/yarn.

ashwinik001 · 2019-11-10T14:18:34Z

I am also running into the issue while installing d3, as @pcambra said. Any workaround?

SalomonSmeke · 2019-11-11T23:30:07Z

Workaround for me was to mount a NPM volume that is container-only. Install NPM there. Then copy the directory over to its final location within another volume that is container-only.

Basically: dont sync it.

ashwinik001 · 2019-11-13T18:12:11Z

/remove-lifecycle stale

docker-robott · 2020-02-11T01:00:13Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

docker-robott · 2020-06-26T03:06:17Z

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

nchazlett added the status/triage label Dec 12, 2017

alcohol mentioned this issue Dec 20, 2017

Bizarre docker specific issue (on OSX) cucumber/cucumber-js#984

Closed

nbigaouette-eai mentioned this issue Jan 16, 2018

Fails to install when going through Docker and volume serverless/serverless#4660

Closed

gsong mentioned this issue Apr 13, 2018

babel/lib/cli.js ENOENT during line ending conversion under docker osxfs npm/npm#19640

Open

1 task

jamesscottbrown mentioned this issue Nov 22, 2018

Unable to publish 0.2.0 Production Release to Docker Hub ove/ove-apps#45

Closed

docker-robott added the lifecycle/stale label Dec 9, 2018

docker-robott removed the lifecycle/stale label Nov 13, 2019

docker-robott added the lifecycle/stale label Feb 11, 2020

docker-robott closed this as completed Mar 12, 2020

docker-robott added the lifecycle/locked label Jun 26, 2020

docker locked and limited conversation to collaborators Jun 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OSXFS specific issue #2296

OSXFS specific issue #2296

alcohol commented Dec 7, 2017 •

edited

Loading

matr369 commented Jan 10, 2018

nbigaouette-eai commented Jan 16, 2018 •

edited

Loading

nbigaouette-eai commented Jan 16, 2018 •

edited

Loading

pcambra commented Apr 13, 2018 •

edited

Loading

chipironcin commented Apr 18, 2018

matr369 commented May 23, 2018

SalomonSmeke commented May 24, 2018

gurucleff commented Aug 13, 2018

tomwidmer commented Aug 24, 2018

gsong commented Aug 24, 2018

tomwidmer commented Aug 27, 2018

cydergoth commented Sep 10, 2018

docker-robott commented Dec 9, 2018

Zodiase commented Dec 9, 2018

Vanuan commented Dec 20, 2018 •

edited

Loading

ashwinik001 commented Nov 10, 2019

SalomonSmeke commented Nov 11, 2019

ashwinik001 commented Nov 13, 2019

docker-robott commented Feb 11, 2020

docker-robott commented Jun 26, 2020

OSXFS specific issue #2296

OSXFS specific issue #2296

Comments

alcohol commented Dec 7, 2017 • edited Loading

matr369 commented Jan 10, 2018

nbigaouette-eai commented Jan 16, 2018 • edited Loading

nbigaouette-eai commented Jan 16, 2018 • edited Loading

pcambra commented Apr 13, 2018 • edited Loading

chipironcin commented Apr 18, 2018

matr369 commented May 23, 2018

SalomonSmeke commented May 24, 2018

gurucleff commented Aug 13, 2018

tomwidmer commented Aug 24, 2018

gsong commented Aug 24, 2018

tomwidmer commented Aug 27, 2018

cydergoth commented Sep 10, 2018

docker-robott commented Dec 9, 2018

Zodiase commented Dec 9, 2018

Vanuan commented Dec 20, 2018 • edited Loading

ashwinik001 commented Nov 10, 2019

SalomonSmeke commented Nov 11, 2019

ashwinik001 commented Nov 13, 2019

docker-robott commented Feb 11, 2020

docker-robott commented Jun 26, 2020

alcohol commented Dec 7, 2017 •

edited

Loading

nbigaouette-eai commented Jan 16, 2018 •

edited

Loading

nbigaouette-eai commented Jan 16, 2018 •

edited

Loading

pcambra commented Apr 13, 2018 •

edited

Loading

Vanuan commented Dec 20, 2018 •

edited

Loading