Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

understanding disc usage #8433

Closed
henfri opened this issue Sep 28, 2024 · 18 comments
Closed

understanding disc usage #8433

henfri opened this issue Sep 28, 2024 · 18 comments
Labels

Comments

@henfri
Copy link
Contributor

henfri commented Sep 28, 2024

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Of course!

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

Describe the problem you're observing.

My backup-disc is full, although much larger than the source disc.
I would like to understand, what is using my backup-disc space.
borg info is too top-level (giving per - archive information only) and also is difficult to read as it provides too much information.

Is there no way to get a better overview? I could imagine for example, that I accidentally backed up some often changing DB or VM file and would not know, how to identify that.

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Sep 28, 2024

Did you forget to add a call to borg compact to your script when upgrading to borg >= 1.2?

You can compact after each borg prune (or borg delete), but it also works if you compact less often (just avoid running out of space: only borg compact actually frees space).

@henfri
Copy link
Contributor Author

henfri commented Sep 28, 2024

Hallo Thomas,

I did run borg compact manually, but the issue persists.
I need to check if borgmatic runs it automatically.

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

Well, you could use borg mount (and a file manager) or borg list to look what's inside your backups, to check if you backed up frequently changing stuff that should not get backed up and then adjust your excludes accordingly.

If you have logs from your backup runs (borg create --list --stats), having a look into these might also be interesting (especially if stats show a sudden jump in the deduplicated size of the freshly created archive).

Due to the deduplication it is hard to find out otherwise (e.g. borg info) what the issue is.

@henfri
Copy link
Contributor Author

henfri commented Sep 28, 2024

Hello,

hm, do you think that this is something that could be added as a feature in future?
Of course I did not keep such logs.

Using borg mount, I would really have to go through folder by folder, as I do not really have a suspect.

Greetings,
Hendrik

@henfri
Copy link
Contributor Author

henfri commented Sep 28, 2024

I am now running borg info in a script and process the json output.

Archive name: homeserver-documents-2020-04-30T19:07:03 | Original size: 1.23 TB | Compressed size: 1.20 TB | Deduplicated size: 791.66 MB
Archive name: localhost-documents-2020-05-10T19:07:02 | Original size: 1.23 TB | Compressed size: 1.20 TB | Deduplicated size: 707.45 MB
Archive name: homeserver-documents-2020-05-31T19:07:02 | Original size: 1.23 TB | Compressed size: 1.20 TB | Deduplicated size: 746.08 MB
Archive name: homeserver-documents-2020-06-05T19:07:03 | Original size: 1.23 TB | Compressed size: 1.20 TB | Deduplicated size: 793.54 MB
Archive name: homeserver-documents-2020-08-28T19:07:02 | Original size: 1.37 TB | Compressed size: 1.34 TB | Deduplicated size: 1.58 GB
Archive name: homeserver-documents-2020-10-31T19:07:04 | Original size: 1.38 TB | Compressed size: 1.34 TB | Deduplicated size: 1.70 GB
Archive name: homeserver-documents-2020-11-27T20:27:15 | Original size: 1.38 TB | Compressed size: 1.34 TB | Deduplicated size: 1.42 GB
Archive name: homeserver-documents-2020-12-07T19:07:03 | Original size: 1.38 TB | Compressed size: 1.34 TB | Deduplicated size: 1.43 GB
Archive name: homeserver-documents-2021-01-31T19:07:02 | Original size: 1.54 TB | Compressed size: 1.43 TB | Deduplicated size: 2.84 GB
Archive name: homeserver-documents-2021-02-27T09:27:31 | Original size: 1.50 TB | Compressed size: 1.42 TB | Deduplicated size: 2.59 GB
Archive name: homeserver-documents-2021-03-30T19:07:05 | Original size: 1.42 TB | Compressed size: 1.34 TB | Deduplicated size: 3.35 GB
Archive name: homeserver-documents-2021-04-30T19:07:03 | Original size: 1.42 TB | Compressed size: 1.34 TB | Deduplicated size: 2.24 GB
Archive name: homeserver-documents-2021-05-31T19:07:04 | Original size: 1.43 TB | Compressed size: 1.35 TB | Deduplicated size: 2.38 GB
Archive name: homeserver-documents-2021-06-30T19:07:05 | Original size: 1.47 TB | Compressed size: 1.39 TB | Deduplicated size: 2.73 GB
Archive name: homeserver-documents-2021-07-31T19:07:05 | Original size: 1.48 TB | Compressed size: 1.39 TB | Deduplicated size: 4.01 GB
Archive name: homeserver-documents-2021-08-31T19:07:06 | Original size: 1.52 TB | Compressed size: 1.44 TB | Deduplicated size: 2.70 GB
Archive name: homeserver-documents-2021-09-30T19:07:02 | Original size: 1.53 TB | Compressed size: 1.44 TB | Deduplicated size: 2.85 GB
Archive name: homeserver-documents-2021-10-31T19:07:02 | Original size: 1.54 TB | Compressed size: 1.46 TB | Deduplicated size: 2.99 GB
Archive name: homeserver-documents-2021-11-30T19:07:03 | Original size: 1.54 TB | Compressed size: 1.46 TB | Deduplicated size: 3.06 GB
Archive name: homeserver-documents-2021-12-31T19:07:03 | Original size: 1.56 TB | Compressed size: 1.47 TB | Deduplicated size: 3.13 GB
Archive name: homeserver-documents-2022-01-30T20:22:00 | Original size: 1.56 TB | Compressed size: 1.47 TB | Deduplicated size: 3.23 GB
Archive name: homeserver-documents-2022-02-28T19:07:06 | Original size: 1.58 TB | Compressed size: 1.49 TB | Deduplicated size: 3.14 GB
Archive name: homeserver-documents-2022-03-21T19:07:05 | Original size: 1.58 TB | Compressed size: 1.49 TB | Deduplicated size: 3.57 GB
Archive name: homeserver-documents-2022-04-30T19:07:04 | Original size: 1.64 TB | Compressed size: 1.55 TB | Deduplicated size: 3.58 GB
Archive name: homeserver-documents-2022-05-31T20:47:07 | Original size: 1.65 TB | Compressed size: 1.55 TB | Deduplicated size: 3.74 GB
Archive name: homeserver-documents-2022-06-30T19:07:04 | Original size: 1.65 TB | Compressed size: 1.55 TB | Deduplicated size: 3.65 GB
Archive name: homeserver-documents-2022-07-31T19:07:02 | Original size: 1.67 TB | Compressed size: 1.56 TB | Deduplicated size: 3.45 GB
Archive name: homeserver-documents-2022-08-31T19:07:02 | Original size: 1.67 TB | Compressed size: 1.56 TB | Deduplicated size: 3.52 GB
Archive name: homeserver-documents-2022-09-30T19:07:02 | Original size: 1.68 TB | Compressed size: 1.57 TB | Deduplicated size: 3.84 GB
Archive name: homeserver-documents-2022-10-31T19:07:02 | Original size: 1.69 TB | Compressed size: 1.57 TB | Deduplicated size: 4.09 GB
Archive name: homeserver-documents-2022-11-30T19:07:02 | Original size: 1.69 TB | Compressed size: 1.57 TB | Deduplicated size: 3.99 GB
Archive name: homeserver-documents-2022-12-31T19:07:02 | Original size: 1.69 TB | Compressed size: 1.57 TB | Deduplicated size: 3.65 GB
Archive name: homeserver-documents-2023-01-08T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 3.43 GB
Archive name: homeserver-documents-2023-01-15T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 3.20 GB
Archive name: homeserver-documents-2023-01-22T19:07:01 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 3.05 GB
Archive name: homeserver-documents-2023-01-29T19:07:01 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 2.75 GB
Archive name: homeserver-documents-2023-01-31T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 2.63 GB
Archive name: homeserver-documents-2023-02-05T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 3.20 GB
Archive name: homeserver-documents-2023-02-12T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 2.74 GB
Archive name: homeserver-documents-2023-02-14T19:07:02 | Original size: 1.73 TB | Compressed size: 1.61 TB | Deduplicated size: 2.87 GB
Archive name: homeserver-documents-2023-03-19T19:07:01 | Original size: 1.71 TB | Compressed size: 1.61 TB | Deduplicated size: 3.83 GB
Archive name: homeserver-documents-2023-03-26T19:07:02 | Original size: 1.71 TB | Compressed size: 1.61 TB | Deduplicated size: 3.01 GB
Archive name: homeserver-documents-2023-03-31T19:07:02 | Original size: 1.71 TB | Compressed size: 1.61 TB | Deduplicated size: 3.08 GB
Archive name: homeserver-documents-2023-04-09T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 2.64 GB
Archive name: homeserver-documents-2023-04-16T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.96 GB
Archive name: homeserver-documents-2023-04-20T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.38 GB
Archive name: homeserver-documents-2023-04-21T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.22 GB
Archive name: homeserver-documents-2023-04-22T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.19 GB
Archive name: homeserver-documents-2023-04-23T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.11 GB
Archive name: homeserver-documents-2023-04-24T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.12 GB
Archive name: homeserver-documents-2023-04-25T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.19 GB
Archive name: homeserver-documents-2023-04-26T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.40 GB
Archive name: homeserver-documents-2023-04-27T19:07:02 | Original size: 1.73 TB | Compressed size: 1.63 TB | Deduplicated size: 1.55 GB

If I see it correctly, each of the backups did add ~1.5GB, right?

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

The way to interpret that is rather like:

Such an archive references NOW about 700MB .. 4GB which are not used by any of the other archives NOW. If you delete any of these, the numbers will change.

@ThomasWaldmann
Copy link
Member

Related: #71

@henfri
Copy link
Contributor Author

henfri commented Sep 29, 2024

Thanks for working on the analysis feature and for your explaination.

@henfri henfri closed this as completed Sep 29, 2024
@henfri
Copy link
Contributor Author

henfri commented Oct 20, 2024

Hello,

I need to come back to this.
I did run borg compact and I did clean up lots of backups. With that, I had about 700GB free. That was three weeks ago.
Now the 5TB drive is full again.

Archive name: ARCHIV_DO_NOT_DELETE-2017-04-17-171027 | Original size: 2.09 TB | Compressed size: 1.97 TB | Deduplicated size: 142.16 GB
Archive name: 'homeserver-2017-06-29-190702' | Original size: 1.52 TB | Compressed size: 1.50 TB | Deduplicated size: 49.39 MB
Archive name: 'homeserver-2017-07-17-190701' | Original size: 933.49 GB | Compressed size: 931.47 GB | Deduplicated size: 20.95 MB
Archive name: 'homeserver-2017-12-25-190702' | Original size: 1.55 TB | Compressed size: 1.54 TB | Deduplicated size: 72.94 MB
Archive name: LaptopHendrik-documents-2018-03-10T17:55:23 | Original size: 125.04 GB | Compressed size: 84.39 GB | Deduplicated size: 27.85 GB
Archive name: homeserver-documents-2018-11-30T19:07:02 | Original size: 1.64 TB | Compressed size: 1.59 TB | Deduplicated size: 3.24 GB
Archive name: Fotos_von_Backupplatte | Original size: 191.66 GB | Compressed size: 191.25 GB | Deduplicated size: 9.89 GB
Archive name: homeserver-documents-2019-12-29T12:16:24 | Original size: 1.47 TB | Compressed size: 1.45 TB | Deduplicated size: 4.40 GB
Archive name: homeserver-documents-2020-12-07T19:07:03 | Original size: 1.38 TB | Compressed size: 1.34 TB | Deduplicated size: 29.85 GB
Archive name: homeserver-documents-2021-12-31T19:07:03 | Original size: 1.56 TB | Compressed size: 1.47 TB | Deduplicated size: 6.30 GB
Archive name: homeserver-documents-2022-06-30T19:07:04 | Original size: 1.65 TB | Compressed size: 1.55 TB | Deduplicated size: 4.36 GB
Archive name: homeserver-documents-2022-12-31T19:07:02 | Original size: 1.69 TB | Compressed size: 1.57 TB | Deduplicated size: 10.32 GB
Archive name: homeserver-documents-2023-07-02T09:56:40 | Original size: 1.27 TB | Compressed size: 1.20 TB | Deduplicated size: 74.04 GB
Archive name: homeserver-documents-2024-09-19T12:15:03 | Original size: 0 B | Compressed size: 0 B | Deduplicated size: 628 B

If I add all the values in "Deduplicated Size", I get about 300GB.
What could be the reason?

Best regards,
Hendrik

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Oct 20, 2024

What you added up is the space only used by one (not by multiple) archive(s).

But of course there is a lot of overlapping use which occupies the remainder of the space.

@henfri
Copy link
Contributor Author

henfri commented Oct 20, 2024

Hello Thomas,
thanks for your reply.
Sorry, I do not get it. When I add up the space used only by one archive, and do that for all, including the first one, I should have the total, I thought.
Hm, looking at the first backup, I see that that is not the case...
So, the last column only shows what is exclusive to this backup and not what was added in this backup.
Original size: 2.09 TB | Compressed size: 1.97 TB | Deduplicated size: 142.16 GB

But should not the Compressed Size of this first backup an the sum of the Deduplicated sizes of the other backups add up to the total space?

In any case, what I do not get: In the last three weeks, borg wrote 680GiB. Why?

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

borg-deduplication-space-usage

@henfri
Copy link
Contributor Author

henfri commented Jan 8, 2025

Hello again,

thanks. I understand your graphic.
However, I still have an issue with this repository.
Accidentally, I did backup way too much. Now the drive is full.
I deleted the last backup and executed compact.

  517  borg delete  ./borg/::homeserver-documents-2024-09-19T12:15:03
  518  borg compact ./borg/

I did not recover the space.
Is there a way of a more thorough compact?

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

See the docs page of borg compact, esp. the threshold option:

https://borgbackup.readthedocs.io/en/stable/usage/compact.html

@henfri
Copy link
Contributor Author

henfri commented Jan 9, 2025

Thanks for your reply, Thomas.
I understand that a segment will only be compacted -by default- if more than 10% space will be saved.
But should I not have created many segments (I have doubled the size of the repository by that mistakes) that are new, so that a compact would remove 100% of the space?

Apart from that, I now get the message:

borg compact --progress --threshold 3 --verbose ./borg/
segment 73400 not found, but listed in compaction data
segment 74423 not found, but listed in compaction data
segment 74463 not found, but listed in compaction data
segment 74869 not found, but listed in compaction data
segment 93284 not found, but listed in compaction data
(and many more)
Compacting segments  37%

Is that cause for concern?

In the end, this did not bring the success I expected:
compaction freed about 238.51 MB repository space.
The Repository before my mistake was 4TB, now it is 8TB..

Greetings,
Hendrik

@ThomasWaldmann
Copy link
Member

2 possibilities:

  • multiple backups reference that additional 4TB. as long as you don't delete them all, the space won't be freed.
  • maybe there is an issue with the compaction hints (these log messages at least indicate it's not in a perfect state, although the messages might be harmless). did you run a borg check --repair recently?

@henfri
Copy link
Contributor Author

henfri commented Jan 10, 2025

Hello,

there may have been further backups, but none that I did not delete (the latest is from 2022, I had 4TB also in October).
I did not run borg check --repair - should I?

Best regards,
Hendrik

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Jan 10, 2025

Not sure, but maybe it helps, just try it. If you have a lot of archives in there, you can use --repository-only, so it finishes faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants