Docker Image 'chown -R daemon:daemon' duplicates application layer #883
Comments
|
Seems this is caused by a very annoying docker issue that has had years of discussions and github issues opened but still hasn't been fixed. The current workaround is to perform chown/chmod locally, create a tar of the contents and then ADD the tar which docker will magically unpack, preserving permissions of the contents. Not sure if there will be any issues getting this to work since a user (probably) won't have the |
|
There have already been some discussions on gitter on this topic ( I couldn't find unfortunately ). For docker we have some options to optimize this (e.g. splitting the app jars from the lib jars to enable caching). However this needs time, testing and verification. So if you have good solution with proper research we I'm happy to discuss this on a pull request from you :) cc @fiadliel |
|
You'll want to follow moby/moby#30110. If you don't have volumes, it should be reasonably safe to just remove the |
|
Thanks, I'll keep an eye on it, an alternative is the "squash" command (when it becomes stable). But I would disagree that it's safe to remove USER/chown - the proof lies in decades of escapes from chroots on Unix systems (and Docker containers are just chroots on steroids). |
|
Personally, I am using https://github.com/goldmann/docker-squash, which gets rid of the "root-owned files"-layer altogether. |
|
hey people, I did a quick and dirty hack to improve layer caching on our projects that basically copies the libraries in one layer and the project files in another layer and completely skips the chown.. I think that the chown could be less problematic if it was also separated in one command for the libs and one for the projects but didn't test it. See this comparison of new/old images:
before the change we would need to upload 2 layers of 44.4MB every time we do a release, now (if no dependencies change) we would only be uploading 412Kb 🎉 here is a gist with the changes: https://gist.github.com/ivantopo/8dbc581723458a1614be3c6c11ab90da @muuki88 would you be interested in a PR? |
|
Thanks @ivantopo for sharing this 😍 I guess we have to start at some point and your gist was the best looking and working solution I have seen so far. As a first step I would like to add this as a separate archetype plugin, so people can turn it on if they feel brave enough. Also I would keep the chowing in the first place. Or introduce an setting If you need any guidance, feel free to ask. Also don't hesitate to open a pull request early and we discuss details there . Thanks again and looking forward to this 😎 |
|
@muuki88 Just saw this issue. Here's another idea we ended up using. The idea is to map libraries and their transitive dependencies into multiple layers that can even be shared between different services :) The only downside: it's using |
|
Thanks @mosche for sharing :) It looks interesting, but I guess would be to specific for native-packager. Creating shared docker images from a single build is something more special. I would also like to avoid a dependency on sbt-dependency-graph. Splitting application and dependencies into two layers is a performance optimization ( that's really what it's all about ) which yields a hopefully small amount of complexity. And must be explicitly enabled. Creating multiple layers from regex and a 3rd party plugin, won't be maintainable with our limited resources. Still this is pretty crazy what you can do with this :D |
|
Update: moby/moby#34263 has a fix for this now |
|
Thanks for sharing @NeQuissimus ❤️ |
Fixes sbt#1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should keep the file ownership by Docker's `root`, and use `chmod` to grant access to `daemon`. The challenge is calling `chmod` without incurring the fs layer overhead (sbt#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute`: chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Here's an example to change these settings: ```scala import com.typesafe.sbt.packager.docker._ dockerPermissionStrategy := DockerPermissionStrategy.Run dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as sbt#1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0.
Fixes sbt#1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should keep the file ownership by Docker's `root`, and use `chmod` to grant access to `daemon`. The challenge is calling `chmod` without incurring the fs layer overhead (sbt#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute` (default): chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Some application will require writing files to the working directory. In that case the setting should be changed as follows: ```scala import com.typesafe.sbt.packager.docker.DockerChmodType dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as sbt#1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0.
Fixes sbt#1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should use `chmod` to default to read-only access unless the build user opts into writable working directory. The challenge is calling `chmod` without incurring the fs layer overhead (sbt#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute` (default): chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Some application will require writing files to the working directory. In that case the setting should be changed as follows: ```scala import com.typesafe.sbt.packager.docker.DockerChmodType dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as sbt#1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0.
Fixes sbt#1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should use `chmod` to default to read-only access unless the build user opts into writable working directory. The challenge is calling `chmod` without incurring the fs layer overhead (sbt#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute` (default): chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Some application will require writing files to the working directory. In that case the setting should be changed as follows: ```scala import com.typesafe.sbt.packager.docker.DockerChmodType dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as sbt#1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0.
Fixes sbt#1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should use `chmod` to default to read-only access unless the build user opts into writable working directory. The challenge is calling `chmod` without incurring the fs layer overhead (sbt#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute` (default): chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Some application will require writing files to the working directory. In that case the setting should be changed as follows: ```scala import com.typesafe.sbt.packager.docker.DockerChmodType dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as sbt#1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0.
* Validate Docker packaging * implement dockerPermissionStrategy Fixes #1189 This implements a non-root Docker container that's safer by default and compatible with Red Hat OpenShift. Current `ADD --chown=daemon:daemon opt /opt` nominally implements non-root image, but by giving ownership of the working directory to the `daemon` user, it reduces the safety. Instead we should use `chmod` to default to read-only access unless the build user opts into writable working directory. The challenge is calling `chmod` without incurring the fs layer overhead (#883). [Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) can be used to pre-stage the files with desired file permissions. This adds new `dockerPermissionStrategy` setting which decides how file permissions are set for the working directory inside the Docker image generated by sbt-native-packager. The strategies are: - `DockerPermissionStrategy.MultiStage` (default): uses multi-stage Docker build to call chmod ahead of time. - `DockerPermissionStrategy.None`: does not attempt to change the file permissions, and use the host machine's file mode bits. - `DockerPermissionStrategy.Run`: calls `RUN` in the Dockerfile. This has regression on the resulting Docker image file size. - `DockerPermissionStrategy.CopyChown`: calls `COPY --chown` in the Dockerfile. Provided as a backward compatibility. For `MultiStage` and `Run` strategies, `dockerChmodType` is used in addition to call `chmod` during Docker build. - `DockerChmodType.UserGroupReadExecute` (default): chmod -R u=rX,g=rX - `DockerChmodType.UserGroupWriteExecute`: chmod -R u=rwX,g=rwX - `DockerChmodType.SyncGroupToUser`: chmod -R g=u - `DockerChmodType.Custom`: Custom argument provided by the user. Some application will require writing files to the working directory. In that case the setting should be changed as follows: ```scala import com.typesafe.sbt.packager.docker.DockerChmodType dockerChmodType := DockerChmodType.UserGroupWriteExecute ``` During `docker:stage`, Docker package validation is called to check if the selected strategy is compatible with the deteted Docker version. This fixes the current repeatability issue reported as #1187. If the incompatibility is detected, the user is advised to either upgrade their Docker, pick another strategy, or override the `dockerVersion` setting. `daemonGroup` is set to `root` instead of copying the value from the `daemonUser` setting. This matches the semantics of `USER` as well as OpenShift, which uses gid=0. * improve the names in file-permission scritped test * add comment on globalSettings
sbt-native-packager versions 1.2.0-M5 and 1.1.4
Affer adding application contents to a docker image, another layer is created with command:
This touches every file in the application and creates new layer that has the same size as the application. In my case, after adding ~130mb layer containing my app, another 130mb layer is added as a result of touching every file in the my app layer.
docker historyshows:Wondering if this is necessary and/or if it could be done before adding the files from the application layer to the container.
The text was updated successfully, but these errors were encountered: