Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] detect non-default dynamic symbols in check_dynamic_dependencies.py #5610

Merged
merged 1 commit into from
Nov 30, 2022

Conversation

jameslamb
Copy link
Collaborator

While updating #5252 to the new CI image proposed in guolinke/lightgbm-ci-docker#29, I found that the Linux regular and Linux swig jobs were failing with the following.

Traceback (most recent call last):
File "/__w/1/s/helpers/check_dynamic_dependencies.py", line 55, in
check_dependencies(Path(sys.argv[1]).read_text(encoding='utf-8'))
File "/__w/1/s/helpers/check_dynamic_dependencies.py", line 30, in check_dependencies
assert len(versions) > 1
AssertionError

I was able to reproduce this locally in a container too.

how to reproduce this (click me)
docker run --rm -ti \
    -v $PWD/artifacts:/artifacts \
    -e AZURE=true \
    -e BUILD_DIRECTORY=/vsts/lightgbm \
    -e BUILD_SOURCESDIRECTORY=/vsts/lightgbm \
    -e BUILD_ARTIFACTSTAGINGDIRECTORY=/artifacts \
    -e COMPILER=gcc \
    -e CONDA_ENV=test-env \
    -e LGB_VER=3.3.3.99 \
    -e OS_NAME=linux \
    -e PRODUCES_ARTIFACTS=true \
    -e PYTHON_VERSION=3.10 \
    -e SETUP_CONDA=false \
    -e TASK=swig \
    -e OMP_NUM_THREADS=4 \
    -e POCL_MAX_PTHREAD_COUNT=4 \
    --workdir=/vsts/lightgbm \
    lightgbm/vsts-agent:manylinux_2_28_x86_64-dev \
    bash

export PATH=/opt/miniforge/bin:"${PATH}"

git clone \
    --recursive \
    https://github.com/jgiannuzzi/LightGBM.git \
    --branch linux-gpu-wheel \
    .

.ci/setup.sh
mkdir $BUILD_DIRECTORY/build
cd $BUILD_DIRECTORY/build
cmake -DUSE_SWIG=ON ..

make -j4

objdump -T $BUILD_DIRECTORY/lib_lightgbm.so > $BUILD_DIRECTORY/objdump.log
objdump -T $BUILD_DIRECTORY/lib_lightgbm_swig.so >> $BUILD_DIRECTORY/objdump.log
python $BUILD_DIRECTORY/helpers/check_dynamic_dependencies.py $BUILD_DIRECTORY/objdump.log

I found that this is happening because helpers/check_dynamic_dependencies.py ignores non-default symbol versions in the output from objdump.

From the objdump docs (link)

-T
Print the dynamic symbol table entries of the file....

...If the version is the default version to be used when resolving unversioned references to the symbol then it's displayed as is, otherwise it's put into parentheses.

In other words, that script ignores versions in objdump -T lib_lightgbm.so output like the following:

0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) strcpy
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) fflush
0000000000000000      DO *UND*	0000000000000000 (GLIBC_2.2.5) stderr
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) fprintf
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) snprintf
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) memmove

Because it expects to only see stuff like this

0000000000000000      DF *UND*	0000000000000000 GLIBC_2.2.5 strcpy
0000000000000000      DF *UND*	0000000000000000 GLIBC_2.2.5 fflush
0000000000000000      DO *UND*	0000000000000000 GLIBC_2.2.5 stderr
0000000000000000      DF *UND*	0000000000000000 GLIBC_2.2.5 fprintf
0000000000000000      DF *UND*	0000000000000000 GLIBC_2.2.5 snprintf
0000000000000000      DF *UND*	0000000000000000 GLIBC_2.2.5 memmove

This PR fixes that.

@jameslamb
Copy link
Collaborator Author

Whenever you have time, @guolinke or @shiyu1994 can you please review this? I think it'll help with #5252.

@jameslamb jameslamb merged commit ae0cd5d into master Nov 30, 2022
@jameslamb jameslamb deleted the fix/check-dynamic-dependencies branch November 30, 2022 15:01
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants