conda-forge-standard (linux): python3 spkg-configure.m4 rejects conda's python3

[mkoeppe]

Checking whether SageMath should install SPKG python3...
checking whether any of bzip2 xz libffi is installed as or will be installed as SPKG... no
checking for python3 >= 3.6.0, < 3.10.0 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core... 
checking ... whether /opt/conda/bin/python3 is good... no, this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://github.com/sagemath/sage-prod/issues/31132; to use it anyway, use ./configure --with-python=/opt/conda/bin/python3
checking ... whether /opt/conda/bin/python3 is good... no, this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://github.com/sagemath/sage-prod/issues/31132; to use it anyway, use ./configure --with-python=/opt/conda/bin/python3

configure: to try to use a different system python, use ./configure --with-python=/path/to/python
configure: no suitable system package found for SPKG python3

Critical for Sage 9.3 because it is a regression of platform support

CC: @isuruf @videlec @dimpase

Component: build: configure

Issue created by migration from https://trac.sagemath.org/ticket/31539

[mkoeppe]

comment:2

On this system, python3 -m sysconfig gives
LDFLAGS = "-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib"

It is the -L/opt/conda/lib that our configure is unhappy about.

[mkoeppe]

comment:3

Moving to 9.4, as 9.3 has been released.

[mkoeppe]

comment:5

@isuruf wrote in https://groups.google.com/g/sage-devel/c/ZCjFpJGD-JA/m/5hOi6hcZBwAJ:

Looks like the check for misconfigured python is a little bit too extreme.
If sysconfig's CFLAGS includes any path in the environment's CFLAGS,
then it shouldn't be considered misconfigured.

From the viewpoint of user installable Python packages, it is actually misconfigured, for the same reasons that led to the introduction of this check in #31132.
The -L options are in the front of the command line, and take precedence over any package-provided -L options that come in from the use of setuptools.

[mkoeppe]

comment:7

From isuruf in #31132 comment:77:

-I can't be stripped, but -L can be stripped by setting LDSHARED env variable. This would fix the conda-forge issue as conda-forge only use -L and doesn't use -I.

[mkoeppe]

comment:8

A previous attempt to use LDSHARED were not successful in #29408.
But of course in the meantime the Xcode python3 has changed (it no longer seems to use xcrun) so this may be worth trying if it can be done without breaking another platform.

Has there been a discussion in conda-forge regarding this misconfiguration of python3?

[isuruf]

comment:9

Has there been a discussion in conda-forge regarding this misconfiguration of python3?

Nope, but the use-case we have in sage is really a rare case because we usually don't want to override a library in a conda environment. Removing -L will cause more harm than good.

Something like below should give the LDSHARED to use and ldflags to append to LDFLAGS.

import sysconfig
import os
import shlex

orig_ldshared = sysconfig.get_config_var("LDSHARED")
orig_cc = sysconfig.get_config_var("CC")
if 'LDSHARED' in os.environ:
    orig_ldshared = os.environ['LDSHARED']
elif 'CC' in os.environ and orig_ldshared.startswith(orig_cc):
    orig_ldshared = os.environ['CC'] + orig_ldshared[len(orig_cc):]

orig_ldshared = shlex.split(orig_ldshared)
new_ldshared = shlex.join(filter(lambda x: not x.startswith("-L"), orig_ldshared))
append_ldflags = shlex.join(filter(lambda x: x.startswith("-L"), orig_ldshared))

print(new_ldshared)
print(append_ldflags)

[isuruf]

comment:10

@mkoeppe, would you be able to help with this? I don't have experience with writing
autoconf scripts. Only experience in patching them. ;)

[dimpase]

comment:12

Not clear from the ticket description whether this is a failure to accept a correct configuration,
or there is more to fix than just this.

If the former, we can just waive Conda's Python3 python through, if we knew how to distinguish it.

Is https://wiki.sagemath.org/Conda up to date?

[mkoeppe]

comment:13

Replying to @dimpase:

Not clear from the ticket description

... it should be clear from the comments above

[mkoeppe]

Outdated; we have updated the check for -L so that it accepts conda's configuration