ABI for float builtins depends on target features

[beetrees]

When LLVM needs to call a builtin (such as __truncdfhf2 or __addtf3), it will use whatever ABI is the default for the target features of the function that is currently being compiled. This means that on targets where the ABI of float types is target-feature dependant (e.g. the ABI of half changes of 32-bit x86 depending on whether sse2 is enabled, and the ABI of fp128 changes on PowerPC depending on whether vsx is enabled), LLVM will call the same builtin function using two different incompatible ABIs.

The simplest solution to this seems to be extending the target-abi module flag to allow frontends to specify which float ABI gets used for function calls independent of the target features of a function. Other solutions include allowing the frontend to override the names of builtin functions on a per-function-being-compiled level.

Related Rust issues: rust-lang/rust#131819, rust-lang/rust#125109

[RalfJung]

The simplest solution to this seems to be extending the target-abi module flag to allow frontends to specify which float ABI gets used for function calls independent of the target features of a function.

I would argue this is a per-call-site decision, see also #70563.

IIUC, the most problematic situation here is code that is generally built for i386 without any SSE support, but then has some fast-paths for when SSE2 is available. That code will presumably be linked with a compiler_rt built without SSE support, but float operations inside the functions that have SSE2 enabled will expect the callee to use the SSE register. If call could encode which ABI it should use, then libcall lowering could synthesize calls that use the right registers, ignoring the target features enabled for the current function.

[nikic]

@RalfJung For calls to builtins, there is usually no call at the IR level. The call is inserted as part of legalizing a (non-call) IR operation.

[RalfJung]

Isn't that happening on some different IR? I'm not familiar with the lower ends of LLVM -- I heard there's Machine IR there.

If this is never in any IR it should be easier -- the legalization "just" has to pass through the information which ABI to use, to make it clear that this should not be adjusted to whatever the target features of the current function are. (I'm sure it's not actually easy, the devil is always in the details.)

[arsenm]

The legalization call will be inserted in SelectionDAG or gMIR

[tgross35]

For reference both gcc and clang just reject _Float16 with -mno-sse2 , and neither seem to support __float128 or _Float128 on PowerPC.

[beetrees]

Both GCC and Clang support _Float128/__float128 on PowerPC when vsx is enabled. Both require the -mfloat128 option to enable it: Clang will automatically enable the vsx feature when -mfloat128 is used whereas GCC will emit an error if -mfloat128 is used and vsx is not enabled.