Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prunerepo and DNF5 #3615

Closed
xsuchy opened this issue Jan 30, 2025 · 8 comments
Closed

prunerepo and DNF5 #3615

xsuchy opened this issue Jan 30, 2025 · 8 comments
Assignees
Labels

Comments

@xsuchy
Copy link
Member

xsuchy commented Jan 30, 2025

Prunerepo still uses DNF3
https://pagure.io/prunerepo/blob/6cafec4a70f93aa87778c3dc3b96d3de7cb0de02/f/prunerepo/helpers.py#_177-179

We done that because of
https://pagure.io/prunerepo/issue/18

DNF3 is slower than DNF5, we should revisit this.

@github-project-automation github-project-automation bot moved this to Needs triage in CPT Kanban Jan 30, 2025
@nikromen nikromen moved this from Needs triage to In Progress in CPT Kanban Feb 3, 2025
@praiskup
Copy link
Member

praiskup commented Feb 8, 2025

$ dnf repoquery --repofrompath=prunerepo_query,pwd --disablerepo='*' --enablerepo prunerepo_query --refresh --quiet --qf '%{location}\n'

dnf5-5.2.8.1-3.fc41.x86_64

Seems like %location / --location is still giving us an empty output.

@praiskup
Copy link
Member

There's a work-around for rpm-software-management/dnf5#2048, it no longer blocks us.

But we need to update the pair_srpm_rpm.py to no longer use import dnf, but import libdnf5. Not sure, yet.

@praiskup praiskup moved this from In 3 months to In Progress in CPT Kanban Feb 11, 2025
@praiskup praiskup removed the blocked label Feb 11, 2025
@praiskup
Copy link
Member

@xsuchy xsuchy added the blocked label Feb 17, 2025
@xsuchy
Copy link
Member Author

xsuchy commented Feb 17, 2025

Needs to be build and deployed to production.

@praiskup
Copy link
Member

After a prior team ACK, deployed to prod. The repo pruner is running now. Going to close tomorrow.

@praiskup
Copy link
Member

Hmm, the script I run manually ended after ~2hours, that is a bit too fast? Monitoring another run.

@praiskup
Copy link
Member

praiskup commented Feb 20, 2025

[2025-02-16 03:46:01,481][  INFO][PID:1473868][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[2025-02-17 03:25:01,234][  INFO][PID:2393753][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[2025-02-18 03:23:02,317][  INFO][PID:1827606][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[2025-02-18 11:03:44,800][  INFO][PID:748139][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[2025-02-19 03:39:02,195][  INFO][PID:3097089][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[2025-02-20 03:35:02,552][  INFO][PID:2864581][copr_prune_results][frontend.py:_send_attempt:70] Sending GET request to frontend URL - https://copr.fedorainfracloud.org/backend/chroots-prunerepo-status/
[root@copr-be ~][PROD]# grep PID:1473868 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-16 12:30:15,501][  INFO][PID:1473868][copr_prune_results][copr_prune_results.py:run:191] --------------------------------------------
[root@copr-be ~][PROD]# grep PID:2393753 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-17 12:54:27,259][  INFO][PID:2393753][copr_prune_results][copr_prune_results.py:run:191] --------------------------------------------
[root@copr-be ~][PROD]# grep PID:1827606 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-18 03:49:17,136][  INFO][PID:1827606][copr_prune_results][copr_prune_results.py:run:172] Pruning tasks are delegated to background workers, waiting.
[root@copr-be ~][PROD]# grep PID:748139 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-18 14:51:12,642][  INFO][PID:748139][copr_prune_results][copr_prune_results.py:run:191] --------------------------------------------
[root@copr-be ~][PROD]# grep PID:3097089 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-19 07:05:09,684][  INFO][PID:3097089][copr_prune_results][copr_prune_results.py:run:191] --------------------------------------------
[root@copr-be ~][PROD]# grep PID:2864581 /var/log/copr-backend/pruner.log | tail -n1
[2025-02-20 07:51:37,868][  INFO][PID:2864581][copr_prune_results][copr_prune_results.py:run:191] --------------------------------------------

These data show the last prunerepo starts, and when particular PIDs finished. The first 2025-02-18 was canceled by me, and the following one was running the updated prunerepo (but benefited from kernel filesystem caches of the canceled run). Anyway, we are down from ~9h to ~4.5h. Nice!

Sorry for the edit. Firefox/GitHub ate my well-edited comment, and submitted previous version.

@praiskup praiskup moved this from In Progress to Done in CPT Kanban Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

2 participants