Skip to content

Commit

Permalink
rust: Add option not to cache path = cargo dependencies
Browse files Browse the repository at this point in the history
Due to Earthly's layer cache code added with `COPY` (even with `--keep-ts`) can
end up with timestamps (`mtime`) corresponding to the point of creation of the
cache entry, not the current time.

However on a following build the `target` mount cache may contain builds from
other branches, with different code for those dependencies, which have a newer
`mtime`. In this case `cargo` will think it can use the cached dependency
instead of rebuilding (because the code appears older than the cached entry
under `target`).

This only affects `path` dependencies since other dependencies are versioned
(either from crates.io or from git dependencies) so cargo's cache key will
differ.

This should become unnecessary with rust-lang/cargo#14136
  • Loading branch information
ijc committed Aug 16, 2024
1 parent 7544585 commit 2d37277
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions rust/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ VERSION 0.8
# Arguments:
# - cache_prefix: Overrides cache prefix for cache IDS. Its value is exported to the build environment under the entry: $EARTHLY_CACHE_PREFIX. By default ${EARTHLY_TARGET_PROJECT_NO_TAG}#${OS_RELEASE}#earthly-cargo-cache
# - keep_fingerprints (false): Instructs the following +CARGO calls to not remove the Cargo fingerprints of the source packages. Use only when source packages have been COPYed with --keep-ts option.
# - do_not_cache_path_based_dependencies (false): Instructs the following +CARGO calls to remove any path based dependencies from the target cache.
# - sweep_days (4): +CARGO uses cargo-sweep to clean build artifacts that haven't been accessed for this number of days.
INIT:
FUNCTION
Expand All @@ -28,6 +29,10 @@ INIT:
ARG keep_fingerprints=false
ENV EARTHLY_KEEP_FINGERPRINTS=$keep_fingerprints

# $EARTHLY_DO_NOT_CACHE_PATH_BASED_DEPENDENCIES
ARG do_not_cache_path_based_dependencies
ENV EARTHLY_DO_NOT_CACHE_PATH_BASED_DEPENDENCIES=$do_not_cache_path_based_dependencies

# $EARTHLY_SWEEP_DAYS
ARG sweep_days=4
ENV EARTHLY_SWEEP_DAYS=$sweep_days
Expand Down Expand Up @@ -62,6 +67,9 @@ CARGO:
RUN --mount=$EARTHLY_RUST_CARGO_HOME_CACHE --mount=$EARTHLY_RUST_TARGET_CACHE \
set -e; \
cargo $args; \
if [ "$EARTHLY_DO_NOT_CACHE_PATH_BASED_DEPENDENCIES" = "true" ]; then \
cargo metadata --format-version=1 --no-deps | jq -r '.packages[] | select(.id | startswith("path+file://")) | .name' | xargs -I{} cargo clean -p {}; \
fi; \
cargo sweep -r -t $EARTHLY_SWEEP_DAYS; \
cargo sweep -r -i; \
$EARTHLY_FUNCTIONS_HOME/copy-output.sh "$output";
Expand Down

0 comments on commit 2d37277

Please sign in to comment.