Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark fread's init function as static #6328

Merged
merged 3 commits into from
Jul 31, 2024

Conversation

rtobar
Copy link
Contributor

@rtobar rtobar commented Jul 31, 2024

The function isn't used elsewhere, and making it publicly accessible opens the door for runtime linking issues -- where the function is served by other libraries exposing the same function. This was seen in a HPC cluster with software built with spack:

0  0x00001555513d8ce0 in init () from /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5
1  0x00001555433f46ba in parse_double_extended (...) at fread.c:819
2  0x00001555433f3e97 in detect_types (...) at fread.c:1203
3  0x00001555433f7959 in freadMain (...) at fread.c:1852
4  0x00001555433fd84d in freadR (...) at fRead.c:217

Marking it as static prevents these issues.

@rtobar rtobar requested a review from MichaelChirico as a code owner July 31, 2024 04:35
@MichaelChirico
Copy link
Member

the trace you gave doesn't actually show any errors, can you please be sure to include the full relevant context?

@rtobar
Copy link
Contributor Author

rtobar commented Jul 31, 2024

@MichaelChirico sorry, in my haste I didn't explain further.

The backtrace above is from a segfault. This occurs whenever trying to read a CSV file with Inf in it, triggering the call to init, which was resolved to be another library's. The other library's initsurely expected completely different arguments and context, and thus its unexpected invocation results in a segfault.

Making the function as static ensures its usage isn't dynamically resolved, and doesn't depend on other libraries already loaded in the process.

@rtobar
Copy link
Contributor Author

rtobar commented Jul 31, 2024

@MichaelChirico for more context, now that I'm at my laptop:

$> Rscript -e 'data.table::fwrite(list(x = Inf), "test.csv"); data.table::fread("test.csv")'

 *** caught segfault ***
address 0x15052c1dbce0, cause 'invalid permissions'

Traceback:
 1: data.table::fread("test.csv")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)

$> R -d gdb
...
> data.table::fwrite(list(x = Inf), "test.csv")
> data.table::fread("test.csv")

Program received signal SIGSEGV, Segmentation fault.
0x00001555513d8ce0 in init () from /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5
Missing separate debuginfos, use: zypper install glibc-locale-base-debuginfo-2.31-150300.41.1.x86_64 krb5-debuginfo-1.19.2-150400.3.3.1.x86_64 libbrotlicommon1-debuginfo-1.0.7-3.3.1.x86_64 libbrotlidec1-debuginfo-1.0.7-3.3.1.x86_64 libcom_err2-debuginfo-1.46.4-150400.3.3.1.x86_64 libcurl4-debuginfo-7.79.1-150400.5.15.1.x86_64 libidn2-0-debuginfo-2.2.0-3.6.1.x86_64 libjansson4-debuginfo-2.9-1.24.x86_64 libjson-c3-debuginfo-0.13-3.3.1.x86_64 libkeyutils1-debuginfo-1.6.3-5.6.1.x86_64 libldap-2_4-2-debuginfo-2.4.46-150200.14.11.2.x86_64 libnghttp2-14-debuginfo-1.40.0-6.1.x86_64 libnsl2-debuginfo-1.2.0-2.44.x86_64 libnss_nis2-debuginfo-3.0-150000.3.3.1.x86_64 libopenssl1_1-debuginfo-1.1.1l-150400.7.22.1.x86_64 libpcre1-debuginfo-8.45-150000.20.13.1.x86_64 libpsl5-debuginfo-0.20.1-150000.3.3.1.x86_64 libselinux1-debuginfo-3.1-150400.1.69.x86_64 libssh4-debuginfo-0.9.6-150400.1.5.x86_64 libtirpc3-debuginfo-1.2.6-150300.3.17.1.x86_64 libunistring2-debuginfo-0.9.10-1.1.x86_64 libzstd1-debuginfo-1.5.0-150400.1.71.x86_64 sssd-common-debuginfo-2.5.2-150400.4.11.1.x86_64
(gdb) bt
#0  0x00001555513d8ce0 in init () from /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5
#1  0x00001555433f46ba in parse_double_extended (ctx=0x7ffffffeb7b0) at fread.c:819
#2  0x00001555433f3e97 in detect_types (pch=pch@entry=0x7ffffffeba18, ncol=ncol@entry=1, bumped=bumped@entry=0x7ffffffebb50, type=<optimized out>) at fread.c:1203
#3  0x00001555433f7959 in freadMain (_args=...) at fread.c:1852
#4  0x00001555433fd84d in freadR (inputArg=<optimized out>, isFileNameArg=<optimized out>, sepArg=<optimized out>, decArg=<optimized out>, quoteArg=<optimized out>, headerArg=0x423fc0, nrowLimitArg=0x2c62af0, skipArg=0x2e89018, NAstringsArg=0x2c62a48, stripWhiteArg=0x2c62818, skipEmptyLinesArg=0x2c627a8, 
    fillArg=0x2c627e0, showProgressArg=0x424030, nThreadArg=0x2e38940, verboseArg=0x2e39078, warnings2errorsArg=0x423ff8, logical01Arg=0x2c62690, selectArg=0x421bd0, dropArg=0x421bd0, colClassesArg=0x421bd0, integer64Arg=0x2c628c0, encodingArg=0x2c62850, keepLeadingZerosArgs=0x2c62620, noTZasUTC=0x2e88fa8)
    at freadR.c:217
#5  0x0000155554f73308 in R_doDotCall (fun=<optimized out>, nargs=nargs@entry=24, cargs=cargs@entry=0x7ffffffec1a0, call=call@entry=0x2a2b100) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/dotcode.c:998
#6  0x0000155554f73bf8 in do_dotcall (call=0x2a2b100, op=<optimized out>, args=<optimized out>, env=<optimized out>) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/dotcode.c:1551
#7  0x0000155554fb92c4 in bcEval (body=<optimized out>, rho=<optimized out>, useCache=<optimized out>) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/eval.c:7446
#8  0x0000155554fd1710 in Rf_eval (e=0x2a0db80, rho=rho@entry=0x2a3c420) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/eval.c:1013
#9  0x0000155554fd3a62 in R_execClosure (call=call@entry=0x29dd0d8, newrho=newrho@entry=0x2a3c420, sysparent=<optimized out>, rho=rho@entry=0x459648, arglist=arglist@entry=0x2a3cca8, op=op@entry=0x29dcea8)
    at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/eval.c:2187
#10 0x0000155554fd5022 in Rf_applyClosure (call=call@entry=0x29dd0d8, op=<optimized out>, arglist=arglist@entry=0x2a3cca8, rho=rho@entry=0x459648, suppliedvars=<optimized out>)
    at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/eval.c:2113
#11 0x0000155554fd1848 in Rf_eval (e=e@entry=0x29dd0d8, rho=rho@entry=0x459648) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/eval.c:1140
#12 0x0000155555012091 in Rf_ReplIteration (rho=rho@entry=0x459648, savestack=savestack@entry=0, browselevel=browselevel@entry=0, state=state@entry=0x7ffffffee930) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/main.c:262
#13 0x0000155555012430 in R_ReplConsole (rho=0x459648, savestack=0, browselevel=0) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/main.c:314
#14 0x00001555550124c8 in run_Rmainloop () at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/main.c:1200
#15 0x0000155555012582 in Rf_mainloop () at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/main.c:1207
#16 0x0000000000400d5b in main (ac=<optimized out>, av=<optimized out>) at /scratch/pawsey0001/spack/setonix/2024.05/software/spack/build_stage/spack-stage-r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/spack-src/src/main/Rmain.c:29

@ben-schwen
Copy link
Member

Cannot reproduce on my system. Could you also provide the output of sessionInfo()

@MichaelChirico
Copy link
Member

Cannot reproduce on my system. Could you also provide the output of sessionInfo()

IIUC this is .so is not loaded by R (?): libsci_gnu_82_mpi.so

and hitting this type of issue IME only happens under certain build environments...

@rtobar
Copy link
Contributor Author

rtobar commented Jul 31, 2024

@ben-schwen This is not easily reproducible because it requires a certain context for it to occur.

Before continuing with the rest of the message, I'm curious whether there's any objection to the change itself. Even if this wasn't causing issues (it is), marking private functions within a module as static is a good idea so they don't pollute the global symbol namespace, specially if they have very generic names.

The key insight to the problem is that the the segfault trace shows the init symbol as coming from libsci_gnu_82_mpi.so.5 instead of fread.c. The symbol in the former library is an object, not even a function, so it makes sense that when trying to execute it you get a segfault with cause 'invalid permissions'.

This is happening in an HPC cluster with SuSE Linux Enterprise Server 15 SP4. The R binary is linked against Cray's libsci as its LAPACK library, which has an init object:

$> ldd $R_HOME/bin/exec/R
        linux-vdso.so.1 (0x00007ffe19185000)
        libR.so => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libR.so (0x000014ca66303000)
        libRblas.so => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libRblas.so (0x000014ca669b0000)
        libxpmem.so.0 => /opt/cray/xpmem/default/lib64/libxpmem.so.0 (0x000014ca66992000)
        libdl.so.2 => /lib64/libdl.so.2 (0x000014ca660ff000)
        libgomp.so.1 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libgomp.so.1 (0x000014ca65eb8000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x000014ca65c95000)
        libc.so.6 => /lib64/libc.so.6 (0x000014ca658a0000)
        libmpifort_gnu_91.so.12 => /opt/cray/pe/lib64/libmpifort_gnu_91.so.12 (0x000014ca6561b000)
        libmpi_gnu_91.so.12 => /opt/cray/pe/lib64/libmpi_gnu_91.so.12 (0x000014ca628a7000)
        libsci_gnu_82_mpi.so.5 => /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5 (0x000014ca62101000)
        libsci_gnu_82.so.5 => /opt/cray/pe/lib64/libsci_gnu_82.so.5 (0x000014ca5e866000)
        libdsmml.so.0 => /opt/cray/pe/lib64/libdsmml.so.0 (0x000014ca5e65b000)
        libgfortran.so.5 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libgfortran.so.5 (0x000014ca5e190000)
        libquadmath.so.0 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libquadmath.so.0 (0x000014ca5df4b000)
        libm.so.6 => /lib64/libm.so.6 (0x000014ca5dc00000)
        libreadline.so.8 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/readline-8.2-mhb24t7gmcozaoczpii54hbciquoe6qa/lib/libreadline.so.8 (0x000014ca66934000)
        libpcre2-8.so.0 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/pcre2-10.42-ttrdyjd7btn4yfs3ifv4slf6mgtf5mkm/lib/libpcre2-8.so.0 (0x000014ca668d1000)
        liblzma.so.5 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/xz-5.4.1-3fbzjvrdj53pqdh3uei75ocs7icmxznu/lib/liblzma.so.5 (0x000014ca6689b000)
        libbz2.so.1.0 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/bzip2-1.0.8-w4dnqgslj6ehbw2ztjzr3ba6cnlpnylz/lib/libbz2.so.1.0 (0x000014ca66881000)
        libz.so.1 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/zlib-ng-2.1.4-vfqglebtahodddms7yc3rfu5ia6vexjs/lib/libz.so.1 (0x000014ca66854000)
        librt.so.1 => /lib64/librt.so.1 (0x000014ca5d9f7000)
        libicuuc.so.67 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/icu4c-67.1-bsctjrdirw3hu5txezrgnw6u3yufu6wz/lib/libicuuc.so.67 (0x000014ca5d7ec000)
        libicui18n.so.67 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/icu4c-67.1-bsctjrdirw3hu5txezrgnw6u3yufu6wz/lib/libicui18n.so.67 (0x000014ca5d4b7000)
        /lib64/ld-linux-x86-64.so.2 (0x000014ca667d6000)
        libfabric.so.1 => /opt/cray/libfabric/1.15.2.0/lib64/libfabric.so.1 (0x000014ca5d1c4000)
        libatomic.so.1 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libatomic.so.1 (0x000014ca5cfbb000)
        libpmi.so.0 => /opt/cray/pe/lib64/libpmi.so.0 (0x000014ca5cdb9000)
        libpmi2.so.0 => /opt/cray/pe/lib64/libpmi2.so.0 (0x000014ca5cb97000)
        libgcc_s.so.1 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libgcc_s.so.1 (0x000014ca5c978000)
        libncursesw.so.6 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/ncurses-6.4-ef6i23whkcmdnioiexcowxxvrdby3mhe/lib/libncursesw.so.6 (0x000014ca66813000)
        libtinfow.so.6 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/ncurses-6.4-ef6i23whkcmdnioiexcowxxvrdby3mhe/lib/libtinfow.so.6 (0x000014ca5c92d000)
        libicudata.so.67 => /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/icu4c-67.1-bsctjrdirw3hu5txezrgnw6u3yufu6wz/lib/libicudata.so.67 (0x000014ca5ae17000)
        libstdc++.so.6 => /opt/cray/pe/gcc/12.2.0/snos/lib64/libstdc++.so.6 (0x000014ca5a9f5000)
        libcxi.so.1 => /usr/lib64/libcxi.so.1 (0x000014ca5a7d0000)
        libcurl.so.4 => /usr/lib64/libcurl.so.4 (0x000014ca5a731000)
        libjson-c.so.3 => /usr/lib64/libjson-c.so.3 (0x000014ca5a521000)
        libpals.so.0 => /opt/cray/pe/lib64/libpals.so.0 (0x000014ca5a319000)
        libnghttp2.so.14 => /usr/lib64/libnghttp2.so.14 (0x000014ca5a0f1000)
        libidn2.so.0 => /usr/lib64/libidn2.so.0 (0x000014ca59ed4000)
        libssh.so.4 => /usr/lib64/libssh.so.4 (0x000014ca59c66000)
        libpsl.so.5 => /usr/lib64/libpsl.so.5 (0x000014ca59a54000)
        libssl.so.1.1 => /usr/lib64/libssl.so.1.1 (0x000014ca599b5000)
        libcrypto.so.1.1 => /usr/lib64/libcrypto.so.1.1 (0x000014ca59675000)
        libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x000014ca59423000)
        libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x000014ca591cf000)
        liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x000014ca58fc0000)
        libzstd.so.1 => /usr/lib64/libzstd.so.1 (0x000014ca58c90000)
        libbrotlidec.so.1 => /usr/lib64/libbrotlidec.so.1 (0x000014ca58a84000)
        libjansson.so.4 => /usr/lib64/libjansson.so.4 (0x000014ca58876000)
        libunistring.so.2 => /usr/lib64/libunistring.so.2 (0x000014ca584f3000)
        libjitterentropy.so.3 => /usr/lib64/libjitterentropy.so.3 (0x000014ca582ec000)
        libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x000014ca58013000)
        libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x000014ca57dfb000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000014ca57bf7000)
        libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x000014ca579e8000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x000014ca577d0000)
        libsasl2.so.3 => /usr/lib64/libsasl2.so.3 (0x000014ca575b3000)
        libbrotlicommon.so.1 => /usr/lib64/libbrotlicommon.so.1 (0x000014ca57392000)
        libkeyutils.so.1 => /usr/lib64/libkeyutils.so.1 (0x000014ca5718d000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x000014ca56f64000)
        libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x000014ca56cdb000)

$> objdump -t /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5 | grep '\<init\>'
0000000000784ce0 g     O .bss   0000000000000004              init
0000000000022e90 g     F .init  0000000000000000              _init

Turning on LD_DEBUG=bindings shows this symbol binding in action:

$> RD_DEBUG=bindings R -e 'data.table::fread("test.csv")' &> ld-debug-all.log
...
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libR.so [0]: normal symbol `Rf_isReal'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libR.so [0]: normal symbol `Rf_isMatrix'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0]: normal symbol `coerceAs'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0]: normal symbol `colnamesInt'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5 [0]: normal symbol `init'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0]: normal symbol `ansMsg'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libR.so [0]: normal symbol `Rf_mkCharCE'
    119820:     binding file /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so [0] to /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libR.so [0]: normal symbol `Rf_error'

I've been trying locally to reproduce this with a dummy, minimal library using a mix of LD_PRELOAD and LD_BIND_NOW but unsuccessful so far; I'll post something if I can come up with it.

I can also confirm that in the original system the proposed patch resolves the issue.

For completeness, in case it's helpful, this is the sessionInfo() output after loading data.table:

> sessionInfo()
Failed to query server: Transport endpoint is not connected
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: SUSE Linux Enterprise Server 15 SP4

Matrix products: default
BLAS:   /software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib/libRblas.so 
LAPACK: /opt/cray/pe/libsci/23.09.1.1/GNU/10.3/x86_64/lib/libsci_gnu_82.so.5.0;  LAPACK version 3.10.1

locale:
 [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8    
 [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8   
 [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C       

time zone: Australia/Perth
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.15.4

loaded via a namespace (and not attached):
[1] compiler_4.3.0
Warning message:
In system("timedatectl", intern = TRUE) :
  running command 'timedatectl' had status 1

@MichaelChirico
Copy link
Member

I'm curious whether there's any objection to the change itself.

for myself, I am just not deeply familiar with any tradeoffs presented by the choice to mark the routine static. LLM tells me this is the right approach though: https://g.co/gemini/share/08a28441079e

OTOH, "namespacing" the function as 'fread_init' instead of the generic 'init' would work without needing to consider whether static is appropriate

@rtobar
Copy link
Contributor Author

rtobar commented Jul 31, 2024

Another detail that clarifies things further: not only the R binary and libR library link to libsci, but data.table.so does too (albeit not the offending one, but a slightly different one, note the final mp instead of mpi):

$> objdump -x /software/projects/pawsey0119/rtobar/setonix/2024.05/r/4.3/data.table/libs/data_table.so  | grep NEEDED
  NEEDED               libz.so.1
  NEEDED               libR.so
  NEEDED               libsci_gnu_82_mp.so.5
  NEEDED               libdl.so.2
  NEEDED               libxpmem.so.0
  NEEDED               libm.so.6
  NEEDED               libpthread.so.0
  NEEDED               libgomp.so.1
  NEEDED               libc.so.6

I'm not sure how that dependency is put there, because when compiling the package I can't see any references there:

> install.packages("data.table")
...
cc -shared -L/software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib -L/software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib -Wl,-rpath,/software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -L/software/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/r-4.3.0-llieqbuwjngu7buqaftswodfq3wx65dc/rlib/R/lib -lR

@MichaelChirico yes, static makes the symbol visible only within that one file, preventing it being exported and having to be resolved (that's the efficiency Gemini refers to), while avoiding name collisions like it happened here. I would suggest to do this instead of namespacing the function name, since a rename would still keep the function publicly visible, when it isn't necessary (and still leaves the door open for a potential clash-- if anything, you'd want to namespace it more specifically like data_table_fread_init) .

@MichaelChirico
Copy link
Member

TBF there are only two clashing routines on all GitHub:

https://github.com/search?q=lang%3Ac+%2F%5Cbfread_init%2F&type=code

and this is the first time the issue comes up in 6 years:

5049e56

so I think it's unlikely it'll cause issues.

anyway I'm swayed that static is the way to go, can you please add a NEWS entry under NOTES and credit yourself?

The function isn't used elsewhere, and making it publicly accessible
opens the door for runtime linking issues -- where the function is
served by other libraries exposing the same function. This was seen in a
HPC cluster with software built with spack:

0  0x00001555513d8ce0 in init () from /opt/cray/pe/lib64/libsci_gnu_82_mpi.so.5
1  0x00001555433f46ba in parse_double_extended (...) at fread.c:819
2  0x00001555433f3e97 in detect_types (...) at fread.c:1203
3  0x00001555433f7959 in freadMain (...) at fread.c:1852
4  0x00001555433fd84d in freadR (...) at fRead.c:217

Signed-off-by: Rodrigo Tobar <[email protected]>
@rtobar
Copy link
Contributor Author

rtobar commented Jul 31, 2024

Thanks @MichaelChirico, I've added an entry under NOTES pointing to this PR. Thanks for the time and consideration!

Copy link
Member

@MichaelChirico MichaelChirico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for investigating + fixing!

@MichaelChirico MichaelChirico merged commit 488bdd2 into Rdatatable:master Jul 31, 2024
4 of 5 checks passed
@rtobar rtobar deleted the static-init branch July 31, 2024 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants