Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unlogged table doesn't survive after compute node restart. #1222

Closed
lubennikovaav opened this issue Feb 5, 2022 · 8 comments
Closed

Unlogged table doesn't survive after compute node restart. #1222

lubennikovaav opened this issue Feb 5, 2022 · 8 comments
Assignees
Labels
c/compute Component: compute, excluding postgres itself m/bug-bash-feb22 Moment: during the Bug Bash in Feb 22 p/cloud Product: Neon Cloud t/bug Issue Type: Bug

Comments

@lubennikovaav
Copy link
Contributor

lubennikovaav commented Feb 5, 2022

CREATE UNLOGGED TABLE unlog (
    id integer PRIMARY KEY,
    val integer NOT NULL
)
WITH (fillfactor='50');
 
INSERT INTO unlog (id, val)
SELECT i, 0
FROM generate_series(1, 10000) AS i;
 
VACUUM (ANALYZE) unlog;

Then stop and start a compute node and the table exists, but the file is not found:

autumn-rain-110164=> \d
               List of relations
 Schema | Name  | Type  |        Owner         
--------+-------+-------+----------------------
 public | unlog | table | lubennikovaav@zenith
(1 row)

autumn-rain-110164=> select * from unlog ;
ERROR:  could not open file "base/16385/24576": No such file or directory

autumn-rain-110164=>  
INSERT INTO unlog (id, val)
SELECT i, 0
FROM generate_series(1, 10000) AS i;
ERROR:  could not open file "base/16385/24576": No such file or directory

UPD: the behaviour is expceted and documented, what remains to be fixed here is that a user should not receive an error message, they should simply see an empty (ulogged) table instead of:

ERROR: could not open file "base/16385/24576": No such file or directory

@lubennikovaav lubennikovaav added the m/bug-bash-feb22 Moment: during the Bug Bash in Feb 22 label Feb 5, 2022
@antons-antons
Copy link
Contributor

antons-antons commented Feb 5, 2022 via email

@kelvich
Copy link
Contributor

kelvich commented Feb 8, 2022

+1, IMO we should bring back old behavior where we just ignored UNLOGGED

@knizhnik
Copy link
Contributor

I think that Zenith users may want to have some kind of "local" tables: tables which will be stored locally at compute nodes and so provide fast access.
Certainly we can and should improve speed of retrieving pages from pageserver so that access to "normal" tables will not be significantly slower than to "local" tables. But at least now it is not true.

It is more natural to use temporary tables as local tables. But them can not be shared between backends. So unlogged tables may play this role of local tables.

Certainly current behavior is unacceptable. We should either treat each node restart as abnormal postgres termination and clear all unlogged tables.
Either really ignore UNLOGGED qualifier and treat them as normal tables.

@stepashka stepashka added c/storage Component: storage p/cloud Product: Neon Cloud labels Feb 17, 2022
@stepashka
Copy link
Member

we're not sure hwo we want to support this, but we'd like the user to get to know that our behaviour is different through the troubleshooting page and an addition to our extension that @kelvich has im mind

@stepashka
Copy link
Member

stepashka commented Apr 14, 2022

confirm that we have this documented and send to backlog @stepashka

@hlinnaka
Copy link
Contributor

hlinnaka commented Oct 8, 2022

A user ran into this yesterday. Not sure what we want to do here, but that error is clearly not ok.

@knizhnik
Copy link
Contributor

knizhnik commented Oct 8, 2022

The use has replaced normal table with unlogged because he was not satisfied by speed of delete.
So if we just ignore "unlogged" qualifier, it doesn't solve the problem.
We can patch Postgres in the place where this error is produced and check if table is unlogged and if so, doesn't report error and reinitialize the table.
But... It once again is not a solution because unlogged table is removed on each node restart. Which is more similar with behavior of temporary tables and once again doesn't satisfy user's needs.

Other solutions?
Somehow maintain persistent storage at compute (for example using EBS)?

@stepashka stepashka removed this from the 2022/06 milestone Oct 20, 2022
@stepashka stepashka added the t/bug Issue Type: Bug label Oct 20, 2022
@vadim2404 vadim2404 assigned vadim2404 and knizhnik and unassigned vadim2404 Nov 17, 2022
@shanyp shanyp added c/compute Component: compute, excluding postgres itself and removed c/storage Component: storage labels Feb 23, 2023
arssher pushed a commit that referenced this issue Feb 24, 2023
Instead of trying to create missing files on the way, send init fork contents as
main fork from pageserver during basebackup. Add test for that. Call
put_rel_drop for init forks; previously they weren't removed. Bump
vendor/postgres to revert previous approach on Postgres side.

Co-authored-by: Arseny Sher <[email protected]>

ref neondatabase/postgres#264
ref neondatabase/postgres#259
ref #1222
arssher pushed a commit that referenced this issue Feb 24, 2023
Instead of trying to create missing files on the way, send init fork contents as
main fork from pageserver during basebackup. Add test for that. Call
put_rel_drop for init forks; previously they weren't removed. Bump
vendor/postgres to revert previous approach on Postgres side.

Co-authored-by: Arseny Sher <[email protected]>

ref neondatabase/postgres#264
ref neondatabase/postgres#259
ref #1222
arssher pushed a commit that referenced this issue Feb 24, 2023
Instead of trying to create missing files on the way, send init fork contents as
main fork from pageserver during basebackup. Add test for that. Call
put_rel_drop for init forks; previously they weren't removed. Bump
vendor/postgres to revert previous approach on Postgres side.

Co-authored-by: Arseny Sher <[email protected]>

ref neondatabase/postgres#264
ref neondatabase/postgres#259
ref #1222
@vadim2404
Copy link
Contributor

the fix is in main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/compute Component: compute, excluding postgres itself m/bug-bash-feb22 Moment: during the Bug Bash in Feb 22 p/cloud Product: Neon Cloud t/bug Issue Type: Bug
Projects
None yet
Development

No branches or pull requests

8 participants