Symlinks with inaccessible targets cause filemanager to not list anything in the same directory

[rcthomas]

Description

It seems that in jupyter_core, in is_file_hidden_posix() an exception is raised if it comes across a symlink where the target path is not accessible to the user (PermissionError). This has the effect of stopping the file contents manager in JupyterLab 3 from displaying the contents of a directory containing such links.

The traceback looks like the following, where the symlink points to a directory that is not accessible to user1:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tornado/web.py", line 1704, in _execute
    result = await result
  File "/usr/local/lib/python3.8/dist-packages/jupyter_server/services/contents/handlers.py", line 110, in get
    model = await ensure_async(self.contents_manager.get(
  File "/usr/local/lib/python3.8/dist-packages/jupyter_server/services/contents/filemanager.py", line 381, in get
    model = self._dir_model(path, content=content)
  File "/usr/local/lib/python3.8/dist-packages/jupyter_server/services/contents/filemanager.py", line 288, in _dir_model
    if self.allow_hidden or not is_file_hidden(os_path, stat_res=st):
  File "/usr/local/lib/python3.8/dist-packages/jupyter_core/paths.py", line 297, in is_file_hidden_posix
    stat_res = os.stat(abs_path)
PermissionError: [Errno 13] Permission denied: '/home/user1/subdir/secret'

Reproduce

1. Have user 1 create a directory that user 2 cannot see, and have user 2 link to the directory created by user 1
2. Have user 2 do something like touch a b c in the same directory as their symlink
3. Have user 2 start up JupyterLab 3 with jupyter_server
4. Browsing to the directory containing the symlink cause the file listing to freeze
5. Logs will show traceback similar to above

Expected behavior

The file listing should just omit the disallowed link since but list everything else that can normally be listed.
I think this means that jupyter_server should handle the exception. See also jupyter/jupyter_core#224

Context

• Operating System and version: ubuntu:focal
• Browser and version: Chrome 90.0.4430.93
• Jupyter Server version: 1.6.4

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[kevin-bates]

Hi @rcthomas - thank you for moving this issue here and thank you for the great instructions for reproducing the issue. Unfortunately, I'm unable to get stat() calls to raise EACCES at all, nor can I reproduce this.

Here's what I did to model your instructions...

As root, I created a directory named protected in user 2's home directory with mode 0700

$ ls -ld protected/
drwx------ 2 root root 6 Apr 28 17:58 protected/


As user2 (gateway), I created a sibling directory named unprotected, within which I created a symbolic link to ../protected and touched files a, b, and c.

$ ls -l unprotected/
total 0
-rw-rw-r-- 1 gateway gateway  0 Apr 28 18:01 a
-rw-rw-r-- 1 gateway gateway  0 Apr 28 18:01 b
-rw-rw-r-- 1 gateway gateway  0 Apr 28 18:01 c
lrwxrwxrwx 1 gateway gateway 12 Apr 28 18:01 mylink -> ../protected


Attempts to navigate into protected result in the expected behavior, as do attempts to follow the symlink

$ cd protected
-bash: cd: protected: Permission denied
$ cd mylink
-bash: cd: mylink: Permission denied


However, os.stat() (or os.lstat()) will not raise EACCES

$ python
Python 3.8.0 (default, Nov  6 2019, 21:49:08) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.stat('/home/gateway/protected')
os.stat_result(st_mode=16832, st_ino=8413, st_dev=64768, st_nlink=2, st_uid=0, st_gid=0, st_size=6, st_atime=1619657951, st_mtime=1619657936, st_ctime=1619657963)
>>> os.stat('/home/gateway/unprotected/mylink')
os.stat_result(st_mode=16832, st_ino=8413, st_dev=64768, st_nlink=2, st_uid=0, st_gid=0, st_size=6, st_atime=1619657951, st_mtime=1619657936, st_ctime=1619657963)
>>> os.lstat('/home/gateway/unprotected/mylink')
os.stat_result(st_mode=41471, st_ino=268690318, st_dev=64768, st_nlink=1, st_uid=1012, st_gid=1012, st_size=12, st_atime=1619658080, st_mtime=1619658078, st_ctime=1619658078)

Also, Lab does not show protected or the symlink (mylink) in unprotected and the logs are clean of tracebacks

Do you happen to be using a mounted filesystem by chance?

Regardless, this doesn't change what should happen regarding reworking _dir_model(), but it just makes changes a little more difficult to validate.

[rcthomas]

This was first observed on a mounted filesystem, yes (GPFS) but I was able to reproduce it in a Docker container on my laptop, with paths inside the container. The traceback I included comes from that Docker attempt.

But, both my setups are using JupyterHub to launch JupyterLab. And, I realized in both places I have set:

JUPYTERHUB_SINGLEUSER_APP=jupyter_server.serverapp.ServerApp

Removing that from my Docker setup made things better. Does that change anything for you in terms of being able to reproduce the JupyterLab behavior?

[kevin-bates]

The GPFS and Docker FS clues make perfect sense as we've experienced these kinds of issues previously - mostly related to securely writing out the kernel's connection file.

It's strange that the removal of JUPYTERHUB_SINGLEUSER_APP caused things to behave better. Its removal restores the use of Notebook as the server (with Lab < 3) yet the two bases (notebook and jupyter_server) are close to identical. We should probably capture versions (via pip freeze) and/or image names (if they vary).
Are any custom content managers in use?

[rcthomas]

I've created a complete Dockerfile-based reproducer. The process helped me notice there's another variable I missed, permissions on at least the directory enclosing the target. The Dockerfile includes commands to fiddle with permissions with a single line that can change the behavior. In this case, if you create the target directory in a directory that is itself accessible to the user, there is no problem in any case. It doesn't matter if jupyter_server is being used or not.

But if you create the inaccessible target directory in a directory that is also not accessible to the user, then you get a PermissionDenied whether you use jupyter_server or not. I think this is in line with your expectations since the codes are close to identical. The full traceback is different, but it's the same error. However, the file manager issue I described only happens for the jupyter_server case.

See my reproducer below. I've left out the cmd setting to make jupyter-labhub the default, and tested both /tree and /lab for the notebook backend. There are 4 tests. The first 2 exhibit the error/problem because the target directory is in a directory that itself is not readable to the user. The latter 2 do not exhibit the problem because we comment out a chmod and rebuild the image. Sorry this is so fiddly:

0. Build: docker build -t jl3:latest .
1. docker run -it --rm -p 8000:8000 jl3:latest jupyterhub
   Log in as user1/user1, then navigate into the "dir" directory. You will see a, b, c but no "secret." However note that there is a traceback in the log, with PermissionDenied (happens for /tree or /lab)
2. docker run -it --rm -p 8000:8000 -e JUPYTERHUB_SINGLEUSER_APP=jupyter_server.serverapp.ServerApp jl3:latest jupyterhub
   Log in as user1/user1, navigate into the "dir" directory. It will not show a, b, c or "secret." It will look like nothing happened in JupyterLab. Different traceback in the log, but still PermissionDenied
3. Edit the Dockerfile to comment out chmod o-rwx /srv, the line that walls off the secret dir, then rebuild and repeat 1.
   You will observe correct behavior (lists a, b, c but not secret) and no error for either /tree or /lab.
4. Repeat 2 with the new image.
   Again, correct behavior, no error.

Thanks for looking into this, I hope this points in a helpful direction.

Dockerfile

FROM ubuntu:focal
WORKDIR /srv

ENV DEBIAN_FRONTEND noninteractive
ENV LANG C.UTF-8

RUN \
    apt-get update          &&  \
    apt-get upgrade --yes   &&  \
    apt-get install --yes       \
        --no-install-recommends \
        npm                     \
        python3-pip             \
        python3-setuptools

RUN \
    pip3 install            \
        --no-cache-dir      \
        jupyterlab          \
        jupyterhub

RUN \
    npm install -g configurable-http-proxy

RUN \
    adduser -q --gecos "" --disabled-password user1     && \
    echo user1:user1 | chpasswd

# For latter 2 tests, comment out "chmod o-rwx /srv"

RUN \
    mkdir /srv/secret           &&  \
    chmod o-rwx /srv/secret     &&  \
    chmod o-rwx /srv            &&  \
    mkdir /home/user1/dir       &&  \
    cd /home/user1/dir          &&  \
    touch a b c                 &&  \
    ln -s /srv/secret secret    &&  \
    chown -R user1:user1 /home/user1/dir

For completeness here's pip3 freeze for the image

alembic==1.5.8
anyio==2.2.0
argon2-cffi==20.1.0
async-generator==1.10
attrs==20.3.0
Babel==2.9.1
backcall==0.2.0
bleach==3.3.0
certifi==2020.12.5
certipy==0.1.3
cffi==1.14.5
chardet==4.0.0
cryptography==3.4.7
decorator==5.0.7
defusedxml==0.7.1
deprecation==2.1.0
entrypoints==0.3
greenlet==1.0.0
idna==2.10
ipykernel==5.5.3
ipython==7.22.0
ipython-genutils==0.2.0
jedi==0.18.0
Jinja2==2.11.3
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.12
jupyter-core==4.7.1
jupyter-packaging==0.9.2
jupyter-server==1.6.4
jupyter-telemetry==0.1.0
jupyterhub==1.4.0
jupyterlab==3.0.14
jupyterlab-pygments==0.1.2
jupyterlab-server==2.5.0
Mako==1.1.4
MarkupSafe==1.1.1
mistune==0.8.4
nbclassic==0.2.7
nbclient==0.5.3
nbconvert==6.0.7
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.3.0
oauthlib==3.1.0
packaging==20.9
pamela==1.0.0
pandocfilters==1.4.3
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
prometheus-client==0.10.1
prompt-toolkit==3.0.18
ptyprocess==0.7.0
pycparser==2.20
Pygments==2.8.1
pyOpenSSL==20.0.1
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
python-editor==1.0.4
python-json-logger==2.0.1
pytz==2021.1
pyzmq==22.0.3
requests==2.25.1
ruamel.yaml==0.17.4
ruamel.yaml.clib==0.2.2
Send2Trash==1.5.0
six==1.15.0
sniffio==1.2.0
SQLAlchemy==1.4.11
terminado==0.9.4
testpath==0.4.4
tomlkit==0.7.0
tornado==6.1
traitlets==5.0.5
urllib3==1.26.4
wcwidth==0.2.5
webencodings==0.5.1

[kevin-bates]

Thanks again for the great information!

So, it looks like this was essentially resolved in Notebook via jupyter/notebook#4670 to address recursive symlinks. I think we should simply add errno.EACCES to the check to prevent this kind of thing dumped to the log:

[W 2021-04-29 19:46:12.255 SingleUserNotebookApp filemanager:295] Unknown error checking if file '/home/user1/dir/secret' is hidden
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/jupyter_server/services/contents/filemanager.py", line 288, in _dir_model
        if self.allow_hidden or not is_file_hidden(os_path, stat_res=st):
      File "/usr/local/lib/python3.8/dist-packages/jupyter_core/paths.py", line 297, in is_file_hidden_posix
        stat_res = os.stat(abs_path)
    PermissionError: [Errno 13] Permission denied: '/home/user1/dir/secret'

I have the PR ported to jupyter_server and the patch for checking EACCES in place, just need to finish the paperwork (sometime this afternoon). I've also pushed the candidate fix in an image: https://hub.docker.com/repository/docker/kbates/jl3 (docker pull kbates/jl3:handle-eaccess).

You should find that Jupyter Server's log output is clean, while Notebook's will still produce the logged exception above (but relative to notebook code).