Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

auto regress to std fs backend if io_uring is not usable on Linux #7116

Closed
Tracked by #4744
skyzh opened this issue Mar 13, 2024 · 8 comments · Fixed by #7120
Closed
Tracked by #4744

auto regress to std fs backend if io_uring is not usable on Linux #7116

skyzh opened this issue Mar 13, 2024 · 8 comments · Fixed by #7120
Assignees
@problame
Labels
c/storage/pageserver Component: storage: pageserver t/bug Issue Type: Bug

Comments

@skyzh
Copy link
Member

skyzh commented Mar 13, 2024 •
edited
Loading

Steps to reproduce

Run page server on Docker for macOS OrbStack. io_uring cannot be used. Page server starts normally, and will only error when new tenants are attached.

Probably it would be good to run some quick kernel version checks before deciding to use io_uring backend by default.

Expected result

Error at an early stage or automatically regress to std fs backend.

Actual result

Environment

Logs, links
@skyzh skyzh added t/bug Issue Type: Bug c/storage/pageserver Component: storage: pageserver labels Mar 13, 2024
@problame problame self-assigned this Mar 13, 2024
@problame
Copy link
Contributor

Will take a look tomorrow.

Out of curiosity, what kernel version and distribution are running in the Docker VM?

cat /etc/os-release
uname -a

@skyzh
Copy link
Member Author

skyzh commented Mar 13, 2024

Inside the page server container,

neon@add5f88ee4ab:~$ cat /etc/os-release
uname -a
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Linux add5f88ee4ab 6.5.13-orbstack-00122-g57b8027e2387 #1 SMP Tue Feb  6 07:48:26 UTC 2024 aarch64 GNU/Linux

I'm on OrbStack Pro trial and it's really easy to use 😅 except I don't know what modifications they did to the kernel.

@skyzh
Copy link
Member Author

skyzh commented Mar 13, 2024

-> https://github.com/orbstack/linux-macvirt

@arpad-m
Copy link
Member

arpad-m commented Mar 13, 2024

Seems related: moby/moby#39396

@arpad-m
Copy link
Member

arpad-m commented Mar 13, 2024 •
edited
Loading

esp: moby/moby#39396 (comment)

edit: so the support for io_uring was added, and then removed again due to security concerns: containerd/containerd#9048

@arpad-m
Copy link
Member

arpad-m commented Mar 13, 2024

I suppose if we want auto-detection, kernel versions won't do. We'll have to check by trying out the io_uring API directly.

@problame
Copy link
Contributor

I already do feature-checking in tokio-epoll-uring, just need to turn it into a public API

problame added a commit to neondatabase/tokio-epoll-uring that referenced this issue Mar 14, 2024
@problame
lifecycle: expose LaunchResult enum
492ae34 
refs neondatabase/neon#7116
@problame problame mentioned this issue Mar 14, 2024
Merged
problame added a commit to neondatabase/tokio-epoll-uring that referenced this issue Mar 14, 2024
@problame
lifecycle: expose LaunchResult enum (#49)
342ddd1 
refs neondatabase/neon#7116
@problame problame mentioned this issue Mar 14, 2024
Merged
@problame
Copy link
Contributor

PR is out: #7120

@problame problame mentioned this issue Mar 14, 2024
Closed
problame added a commit that referenced this issue Mar 15, 2024
@problame
tokio-epoll-uring: fallback to std-fs if not available & not explicit…
60f3000 
…ly requested (#7120)

fixes #7116

Changes:

- refactor PageServerConfigBuilder: support not-set values
- implement runtime feature test
- use runtime feature test to determine `virtual_file_io_engine` if not
explicitly configured in the config
- log the effective engine at startup
- drive-by: improve assertion messages in `test_pageserver_init_node_id`

This needed a tiny bit of tokio-epoll-uring work, hence bumping it.
Changelog:

```
    git log --no-decorate --oneline --reverse 868d2c42b5d54ca82fead6e8f2f233b69a540d3e..342ddd197a060a8354e8f11f4d12994419fff939
    c7a74c6 Bump mio from 0.8.8 to 0.8.11
    4df3466 Bump mio from 0.8.8 to 0.8.11 (#47)
    342ddd1 lifecycle: expose `LaunchResult` enum (#49)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@problame problame

Labels
c/storage/pageserver Component: storage: pageserver t/bug Issue Type: Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tokio-epoll-uring: fallback to std-fs if not available & not explicitly requested neondatabase/neon
3 participants
@problame @skyzh @arpad-m