Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the filecaching #198

Open
fredrikekre opened this issue Dec 14, 2023 · 0 comments
Open

Rework the filecaching #198

fredrikekre opened this issue Dec 14, 2023 · 0 comments
Assignees

Comments

@fredrikekre
Copy link
Member

Some initial exploration of the data from #196 suggest that registries have a way to "stick" in the cache. Some datapoints (aggregation over all deployed servers):

Resource type Total size (GB) Total number of files Average size (MB)
registries 585 70500 8.3
packages 89 73800 1.2
artifacts 1000 26400 37.9

In particular it seems wasteful to store so many registries in particular since they are quite large (7x the size of the average package). Registries are also a bit different in that all(?) requests want the latest one, so caching something like the 10 latest should be more than sufficient. This would free up space for packages and artifacts instead, which could potentially reduce network usage between package servers and storage servers.

The major reason causing this (I think) is that we now serve existing files from nginx directly but this cache hit is not recorded on the julia resulting in all resources have num_accessed = 1.

Removing the registries from the equation as suggested above should help quite a bit, but might also be worth trying some way to communicate the cache hits from nginx to julia.

@fredrikekre fredrikekre self-assigned this Dec 14, 2023
fredrikekre added a commit that referenced this issue Dec 21, 2023
As described in #198, the file cache is unaware of cache hits for files
served directly by nginx. This patch notifies PkgServer.jl about nginx
cache hits by sending a mirrored request to the internal /notify
endpoint. The original request URI is sent through the `X-Original-URI`
header and is used to record the cache hit. In addition, since any cache
hits are for files already existing in the cache we can record the size
as bytes sent by nginx in a Prometheus counter.
fredrikekre added a commit that referenced this issue Dec 21, 2023
As described in #198, the file cache is unaware of cache hits for files
served directly by nginx. This patch notifies PkgServer.jl about nginx
cache hits by sending a mirrored request to the internal /notify
endpoint. The original request URI is sent through the `X-Original-URI`
header and is used to record the cache hit. In addition, since any cache
hits are for files already existing in the cache we can record the size
as bytes sent by nginx in a Prometheus counter.
fredrikekre added a commit that referenced this issue Dec 27, 2023
As described in #198, the file cache is unaware of cache hits for files
served directly by nginx. This patch notifies PkgServer.jl about nginx
cache hits by sending a mirrored request to the internal /notify
endpoint. The original request URI is sent through the `X-Original-URI`
header and is used to record the cache hit. In addition, since any cache
hits are for files already existing in the cache we can record the size
as bytes sent by nginx in a Prometheus counter.
fredrikekre added a commit that referenced this issue Dec 27, 2023
As described in #198, the file cache is unaware of cache hits for files
served directly by nginx. This patch notifies PkgServer.jl about nginx
cache hits by sending a mirrored request to the internal /notify
endpoint. The original request URI is sent through the `X-Original-URI`
header and is used to record the cache hit. In addition, since any cache
hits are for files already existing in the cache we can record the size
as bytes sent by nginx in a Prometheus counter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant