Is it possible to use a custom type of DB?

[Twi-Hard]

I was wondering if anybody knows a method of using postgres for the archive files either directly or indirectly? I use gallery-dl on several different computers, and I've had to pick which computer handles which site because I have to keep the sqlite database local to that computer. I've tried putting in on a nfs share but that just makes it lock up. I use postgres for everything else already and it works over the network so all of the computers could contribute to the same database. The way I have things set up, I really depend on those archive files and usually can't use gallery-dl without them without running into trouble. Does anybody have any ideas? Thanks :)

[mikf]

Using SQLite3 for anything database related is currently hardcoded.

If it's just archives, it might be relatively easy to add an option to support backends other than SQLite.

[mikf]

@Twi-Hard
I made an attempt at adding PostgreSQL support for archives: 841bc9f

I've tested it on my local machine and it seems to work, but I had absolutely no experience using PostgreSQL before, so expect a few bugs and inefficient database usage. You need to tell me what needs improvements / changes, because I don't know any better myself.

edit: I should mention archive needs to be a PostgreSQL Connection URI and psycopg needs to be installed for this to work.

[Twi-Hard]

Thank you so much :)
Unfortunately, it would get EXTREMELY out of control if I had to make a separate DB for each site. Is it possible to name the table after the category name instead of "archive"? That would be a HUGE help. Some of my categories have dots in the names (custom boorus) so those would have to be replaced with underscores.
Also, you might want to add the new package to the optional dependencies on the readme

[Twi-Hard]

Now that I thought about it some more, selecting a table name (optionally) would be optimal. The default could potentially be the category name. The issue I thought of is I have several archive files usually. Sometimes it's just an archive for media and an archive for jsons but sometimes I have ones for different subcategories or ones for postprocessors (like exec postprocessors for example).

[mikf]

Technically. archive-prefix is meant to differentiate between sites/categories/etc, but adding a new archive-table option would be possible.

And now that you mention it: I completely forgot about post processor archives ...

[Twi-Hard]

The thing with archive-prefix is wouldn't that still add everything to the same table? I assumed that could cause performance issues over time if it all piles up in one table. Regardless, I think the archive-table option existing would be really good. :)

[Twi-Hard]

To give context of how big that table would be if it wasn't multiple tables, I have 4.9GB of archive files lol

[mikf]

Post processor archives can now use PostgreSQL databases: fd4de02

I've also added an archive-table option, which is interpreted as a format string and lets you choose an arbitrary table name to store IDs in. The name, when used in an SQL query, is wrapped in double quotes ", so it can contain any character (except ", which are replaced with _)

[kattjevfel]

Been busy converting all my separate sqlite databases into a single psql database with archive-table="{category}", and so far it's working great!