OpenMPI security paranoia about ptrace #3397

claudevms · 2018-07-21T20:21:37Z

Please fill out the below information:

Your Windows build number: (Type ver at a Windows Command Prompt)
Windows 10 Pro
Version: 1803
OS Build 17134.18
Ubuntu
16.04.4 LTS
xenial
What you're doing and what's happening: (Copy&paste specific commands and their output, or include screen shots)

Running LAMMPS software inside WSL at the bash command prompt
mike@Tak: mpirun -np 32 --hostfile hostfile /home/mike/bin/sxlmp_umpi -in pdms50_script.in

What's wrong / what should be happening instead:
The following message is generated at the start of execution. This did not happen prior to the 1803 build

WARNING: Linux kernel CMA support was requested via the btl_vader_single_copy_mechanism MCA variable, but CMA support is not available due to restrictive ptrace settings.

The vader shared memory BTL will fall back on another single-copy mechanism if one is available. This may result in lower performance.
Local host: Tak

What should happen:

The above message shouldn't be displayed. Does this require building OpenMPI a different way or
is there some security "feature" that needs to be disabled in WSL?
There should be some easy to follow instructions to turn off the restrictive ptrace settings.
I shouldn't have to settle for bad performance because some security guy is paranoid. Security should
be opt-in not opt-out.

Possible rant:

Just a short note: Many of us with compute servers don't care about security. That's what "air gaps" are for. I have yet to find how Microsoft plans to distribute Sperctre and Meltdown to everyone whether they want it or not. Many of us need a tool to see if that crap was installed and hurting performance
and instructions to remove it if found.

By the way:

THANK YOU FOR WSL!!! This is the best way for me to build LAMMPS and run it on my servers and
analysis nodes. I get access to Linux and Windows software!!!

Strace of the failing command, if applicable: (If some_command is failing, then run strace -o some_command.strace -f some_command some_args, and link the contents of some_command.strace in a gist here).
For WSL launch issues, please collect detailed logs.

See our contributing instructions for assistance.

The text was updated successfully, but these errors were encountered:

NablaCFD · 2018-08-30T17:25:29Z

Microsoft Windows 10 Pro [Version 10.0.17134.228]
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
++++++++++++++++++++++++++++++++++++++++
I encounter the same problem.

type the following COMMAND in bash:
mpirun -np 3 test_openmpi

WARNING: Linux kernel CMA support was requested via the
btl_vader_single_copy_mechanism MCA variable, but CMA support is
not available due to restrictive ptrace settings.

The vader shared memory BTL will fall back on another single-copy
mechanism if one is available. This may result in lower performance.

Local host: workstation

[workstation:04636] [[3228,0],0] ORTE_ERROR_LOG: Data unpack would read past end of buffer in file util/show_help.c at line 507
Hello: rank 1, world: 3
Hello: rank 0, world: 3
Hello: rank 2, world: 3
[workstation:04636] 1 more process has sent help message help-btl-vader.txt / cma-permission-denied
[workstation:04636] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

here is the file: test_openmpi.cpp
cat test_openmpi.cpp
/* Put this text inside hello.c file */
#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
int rank;
int world;

MPI_Init(NULL, NULL);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &world);
printf("Hello: rank %d, world: %d\n",rank, world);
MPI_Finalize();
}

I have tested it using the OpenMPI-3.1.1 and OpenMPI-3.1.2 compiled by myself, the above problem always happens.

Of course I can use the following option to turn off this warning, but I do not sure whether it will affect the performance. Thus, I think it should be solved by a developer from either WSL or OpenMPI.

mpirun --mca btl_vader_single_copy_mechanism none -np 3 test_openmpi

+++++++++++++++++++++++++++++++++++++++++
In addition:
I agree the above author's suggestion, in fact we care more about the performance than the safety. So maybe if Win10 can give the more switches/options to tune the performance and turn off the safety options if they will influent the performance.

This link may be useful to someone.
open-mpi/ompi#4948

Thanks in advance.
QL

Shweppie · 2018-09-01T17:56:28Z

Same issue... Try the following:
https://groups.io/g/OpenHPC-users/topic/openmpi_and_shared_memory/16489081?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,16489081

$ echo 0 > /proc/sys/kernel/yama/ptrace_scope
or
$ sudo echo 0 > /proc/sys/kernel/yama/ptrace_scope
or
$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

claudevms · 2018-09-03T21:26:09Z

Would be nice to permanently configure the system so I don't have to do this. Looks l8ke Linux projects are using 1 as a default as well. Much thanks for WSL!!!

…

On Sat, Sep 1, 2018, 10:56 AM Shweppie ***@***.***> wrote: Same issue... Try the following: $ echo 0 > /proc/sys/kernel/yama/ptrace_scope or $ sudo echo 0 > /proc/sys/kernel/yama/ptrace_scope or $ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3397 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AdNyby4XtSY1qDytfWskOuDWB0mD0v7vks5uWspWgaJpZM4VZwDE> .

therealkenc · 2018-09-03T22:12:31Z

Yeah 1 appears to be the default on unadulterated Ubuntu. Maybe change the issue title to "change default ptrace_scope to 1 if that's really the concrete ask we're talking about here. [Someone will suggest making it a wsl.conf thing, and then I'll feel sad inside.]

claudevms · 2018-09-03T22:23:26Z

I will change it in wsl.conf to 0! That is what I need!!! Thank you!!! A value of 1 hurts OpenMPI performance

…

On Mon, Sep 3, 2018, 3:12 PM therealkenc ***@***.***> wrote: Yeah 1 appears to be the default on unadulterated Ubuntu. Maybe change the issue title to "change default ptrace_scope to 1if that's really the concrete ask we're talking about here. [Someone will suggest making it awsl.conf` thing, and then I'll feel sad inside.] — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3397 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AdNyb267W1hMNBCcxFMpuuKbjtt_BFNJks5uXalYgaJpZM4VZwDE> .

KristianHolsheimer · 2020-01-23T01:00:17Z

If you'd like this setting to be persistent, then you need to edit the config file /etc/sysctl.d/10-ptrace.conf instead, e.g.

$ sudo vim /etc/sysctl.d/10-ptrace.conf

and edit the line that says

kernel.yama.ptrace_scope = 1

to

kernel.yama.ptrace_scope = 0

N.B. This was taken from this answer on Stack Overflow.

therealkenc · 2020-01-23T01:27:17Z

/etc/sysctl.d/10-ptrace.conf

Nothing reads that file on WSL (nothing starts systemd-sysctl.service), de-facto landing zone #994.

KristianHolsheimer · 2020-01-23T01:59:23Z

Ah I see. Right never mind then

lokapal · 2021-02-21T13:12:56Z

Well, how to set kernel.yama.ptrace_scope to 0 permanently? There is no possibility to set kernel variable in wsl.conf (at least this is not documented) and WSL just ignores rc.local and /etc/sysctl.d files....

therealkenc · 2021-02-22T18:27:55Z

Test and set it in .bashrc or .profile is the de-facto standard operating procedure right now. Use sudoers to avoid the password prompt if you like.

yiqiangjizhang · 2021-03-01T19:07:17Z

Yeah 1 appears to be the default on unadulterated Ubuntu. Maybe change the issue title to "change default ptrace_scope to 1 if that's really the concrete ask we're talking about here. [Someone will suggest making it a wsl.conf thing, and then I'll feel sad inside.]

How did you change it? I have the same issue but I can't manage to figure out how to set it to 0

yiqiangjizhang · 2021-03-01T19:07:33Z

I will change it in wsl.conf to 0! That is what I need!!! Thank you!!! A value of 1 hurts OpenMPI performance
…
On Mon, Sep 3, 2018, 3:12 PM therealkenc @.***> wrote: Yeah 1 appears to be the default on unadulterated Ubuntu. Maybe change the issue title to "change default ptrace_scope to 1if that's really the concrete ask we're talking about here. [Someone will suggest making it awsl.conf` thing, and then I'll feel sad inside.] — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3397 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AdNyb267W1hMNBCcxFMpuuKbjtt_BFNJks5uXalYgaJpZM4VZwDE .

How did you change it? I have the same issue but I can't manage to figure out how to set it to 0

yiqiangjizhang · 2021-03-01T21:01:46Z

Same issue... Try the following:
https://groups.io/g/OpenHPC-users/topic/openmpi_and_shared_memory/16489081?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,16489081

$ echo 0 > /proc/sys/kernel/yama/ptrace_scope
or
$ sudo echo 0 > /proc/sys/kernel/yama/ptrace_scope
or
$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

Solve it using the last command! (I am using WSL 1). Just copy and paste it into your terminal

therealkenc · 2021-03-02T20:01:22Z

ptrace_scope was added to WSL1 in 17046. For WSL2, #4286.

therealkenc mentioned this issue Oct 3, 2018

could not execute .exe file beyond windows root #3588

Closed

TheodoreGalanos mentioned this issue Oct 24, 2018

Support Windows 10 native installation ladybug-tools/butterfly#271

Open

lucagl mentioned this issue Apr 7, 2020

Mysterious segmentation fault rarely observed lucagl/KMC_pattern#12

Closed

yiqiangjizhang mentioned this issue Mar 1, 2021

Open MPI throws error while unlinking internal file open-mpi/ompi#8531

Closed

therealkenc added the fixedinreleasebuild label Mar 2, 2021

therealkenc closed this as completed Mar 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenMPI security paranoia about ptrace #3397

OpenMPI security paranoia about ptrace #3397

claudevms commented Jul 21, 2018

NablaCFD commented Aug 30, 2018 •

edited

Loading

Shweppie commented Sep 1, 2018 •

edited

Loading

claudevms commented Sep 3, 2018 via email

therealkenc commented Sep 3, 2018 •

edited

Loading

claudevms commented Sep 3, 2018 via email

KristianHolsheimer commented Jan 23, 2020

therealkenc commented Jan 23, 2020

KristianHolsheimer commented Jan 23, 2020

lokapal commented Feb 21, 2021

therealkenc commented Feb 22, 2021

yiqiangjizhang commented Mar 1, 2021

yiqiangjizhang commented Mar 1, 2021

yiqiangjizhang commented Mar 1, 2021

therealkenc commented Mar 2, 2021 •

edited

Loading

OpenMPI security paranoia about ptrace #3397

OpenMPI security paranoia about ptrace #3397

Comments

claudevms commented Jul 21, 2018

The vader shared memory BTL will fall back on another single-copy mechanism if one is available. This may result in lower performance. Local host: Tak

NablaCFD commented Aug 30, 2018 • edited Loading

type the following COMMAND in bash: mpirun -np 3 test_openmpi

Local host: workstation

Shweppie commented Sep 1, 2018 • edited Loading

claudevms commented Sep 3, 2018 via email

therealkenc commented Sep 3, 2018 • edited Loading

claudevms commented Sep 3, 2018 via email

KristianHolsheimer commented Jan 23, 2020

therealkenc commented Jan 23, 2020

KristianHolsheimer commented Jan 23, 2020

lokapal commented Feb 21, 2021

therealkenc commented Feb 22, 2021

yiqiangjizhang commented Mar 1, 2021

yiqiangjizhang commented Mar 1, 2021

yiqiangjizhang commented Mar 1, 2021

therealkenc commented Mar 2, 2021 • edited Loading

The vader shared memory BTL will fall back on another single-copy mechanism if one is available. This may result in lower performance.
Local host: Tak

NablaCFD commented Aug 30, 2018 •

edited

Loading

type the following COMMAND in bash:
mpirun -np 3 test_openmpi

Shweppie commented Sep 1, 2018 •

edited

Loading

therealkenc commented Sep 3, 2018 •

edited

Loading

therealkenc commented Mar 2, 2021 •

edited

Loading