1.11 enablement

[maleadt]

No description provided.

[codecov]

Codecov Report

Patch coverage: 86.95% and project coverage change: +10.29% 🎉

Comparison is base (77247f7) 73.14% compared to head (e49a782) 83.44%.

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #497       +/-   ##
===========================================
+ Coverage   73.14%   83.44%   +10.29%     
===========================================
  Files          23       23               
  Lines        3039     3069       +30     
===========================================
+ Hits         2223     2561      +338     
+ Misses        816      508      -308     

Files Changed	Coverage Δ
src/validation.jl	91.84% <84.84%> (-0.42%)	⬇️
src/spirv.jl	93.10% <90.00%> (+15.53%)	⬆️
src/metal.jl	68.66% <100.00%> (-0.06%)	⬇️

... and 10 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[maleadt]

julia: /workspace/srcdir/llvm-project/llvm/lib/ExecutionEngine/Orc/Core.cpp:2915: llvm::Error llvm::orc::ExecutionSession::OL_notifyResolved(llvm::orc::MaterializationResponsibility&, const SymbolMap&): Assertion `I != MR.SymbolFlags.end() && "Resolving symbol outside this responsibility set"' failed.

[105124] signal (6.-6): Aborted
in expression starting at /home/tim/Julia/pkg/GPUCompiler/test/native.jl:1
unknown function (ip: 0x7fe7e248e83c)
raise at /usr/lib/libc.so.6 (unknown line)
abort at /usr/lib/libc.so.6 (unknown line)
unknown function (ip: 0x7fe7e24263db)
__assert_fail at /usr/lib/libc.so.6 (unknown line)
llvm::orc::ExecutionSession::OL_notifyResolved(llvm::orc::MaterializationResponsibility&, llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol> > const&) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::RTDyldObjectLinkingLayer::onObjLoad(llvm::orc::MaterializationResponsibility&, llvm::object::ObjectFile const&, llvm::RuntimeDyld::MemoryManager&, llvm::RuntimeDyld::LoadedObjectInfo&, std::map<llvm::StringRef, llvm::JITEvaluatedSymbol, std::less<llvm::StringRef>, std::allocator<std::pair<llvm::StringRef const, llvm::JITEvaluatedSymbol> > >, std::set<llvm::StringRef, std::less<llvm::StringRef>, std::allocator<llvm::StringRef> >&) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::Error llvm::detail::UniqueFunctionBase<llvm::Error, llvm::object::ObjectFile const&, llvm::RuntimeDyld::LoadedObjectInfo&, std::map<llvm::StringRef, llvm::JITEvaluatedSymbol, std::less<llvm::StringRef>, std::allocator<std::pair<llvm::StringRef const, llvm::JITEvaluatedSymbol> > > >::CallImpl<llvm::orc::RTDyldObjectLinkingLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >)::{lambda(llvm::object::ObjectFile const&, llvm::RuntimeDyld::LoadedObjectInfo&, std::map<llvm::StringRef, llvm::JITEvaluatedSymbol, std::less<llvm::StringRef>, std::allocator<std::pair<llvm::StringRef const, llvm::JITEvaluatedSymbol> > >)#1}>(void*, llvm::object::ObjectFile const&, llvm::RuntimeDyld::LoadedObjectInfo&, std::map<llvm::StringRef, llvm::JITEvaluatedSymbol, std::less<llvm::StringRef>, std::allocator<std::pair<llvm::StringRef const, llvm::JITEvaluatedSymbol> > >&) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::jitLinkForORC(llvm::object::OwningBinary<llvm::object::ObjectFile>, llvm::RuntimeDyld::MemoryManager&, llvm::JITSymbolResolver&, bool, llvm::unique_function<llvm::Error (llvm::object::ObjectFile const&, llvm::RuntimeDyld::LoadedObjectInfo&, std::map<llvm::StringRef, llvm::JITEvaluatedSymbol, std::less<llvm::StringRef>, std::allocator<std::pair<llvm::StringRef const, llvm::JITEvaluatedSymbol> > >)>, llvm::unique_function<void (llvm::object::OwningBinary<llvm::object::ObjectFile>, std::unique_ptr<llvm::RuntimeDyld::LoadedObjectInfo, std::default_delete<llvm::RuntimeDyld::LoadedObjectInfo> >, llvm::Error)>) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::RTDyldObjectLinkingLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ObjectTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::IRCompileLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
LLVMOrcIRTransformLayerEmit at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
LLVMOrcIRTransformLayerEmit at /home/tim/Julia/depot/packages/LLVM/Od0DH/lib/15/libLLVM_h.jl:4790 [inlined]
emit at /home/tim/Julia/depot/packages/LLVM/Od0DH/src/orcv2.jl:253 [inlined]
#3 at /home/tim/Julia/pkg/GPUCompiler/test/definitions/native.jl:252 [inlined]
ThreadSafeContext at /home/tim/Julia/depot/packages/LLVM/Od0DH/src/executionengine/ts_module.jl:15
#2 at /home/tim/Julia/pkg/GPUCompiler/test/definitions/native.jl:245
JuliaContext at /home/tim/Julia/pkg/GPUCompiler/src/driver.jl:47
materialize at /home/tim/Julia/pkg/GPUCompiler/test/definitions/native.jl:235
unknown function (ip: 0x7fe7e1bc7e25)
_jl_invoke at /home/tim/Julia/src/julia/src/gf.c:2870 [inlined]
ijl_apply_generic at /home/tim/Julia/src/julia/src/gf.c:3071
__materialize at /home/tim/Julia/depot/packages/LLVM/Od0DH/src/orcv2.jl:292
unknown function (ip: 0x7fe7e1bc60dc)
llvm::orc::MaterializationTask::run() at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
void llvm::detail::UniqueFunctionBase<void, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> > >::CallImpl<void (*)(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >)>(void*, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >&) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ExecutionSession::dispatchTask(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ExecutionSession::dispatchOutstandingMUs() at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ExecutionSession::OL_completeLookup(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, std::shared_ptr<llvm::orc::AsynchronousSymbolQuery>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::InProgressFullLookupState::complete(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ExecutionSession::OL_applyQueryPhase1(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, llvm::Error) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::ExecutionSession::lookup(llvm::orc::LookupKind, std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags> > > const&, llvm::orc::SymbolLookupSet, llvm::orc::SymbolState, llvm::unique_function<void (llvm::Expected<llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol> > >)>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::LazyCallThroughManager::resolveTrampolineLandingAddress(unsigned long, llvm::unique_function<void (unsigned long) const>) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
void llvm::detail::UniqueFunctionBase<void, unsigned long, llvm::unique_function<void (unsigned long) const> >::CallImpl<llvm::orc::LocalLazyCallThroughManager::init<llvm::orc::OrcX86_64_SysV>()::{lambda(unsigned long, llvm::unique_function<void (unsigned long) const>)#1} const>(void*, unsigned long, llvm::unique_function<void (unsigned long) const>&) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
llvm::orc::LocalTrampolinePool<llvm::orc::OrcX86_64_SysV>::reenter(void*, void*) at /home/tim/Julia/src/julia/build/dev/usr/bin/../lib/libLLVM-15jl.so (unknown line)
unknown function (ip: 0x7fe7e2650043)
unknown function (ip: 0x7fe7e2238fb5)
unknown function (ip: 0x7fe7e1bcc096)
_jl_invoke at /home/tim/Julia/src/julia/src/gf.c:2870 [inlined]
ijl_apply_generic at /home/tim/Julia/src/julia/src/gf.c:3071
call_delayed at /home/tim/Julia/pkg/GPUCompiler/test/definitions/native.jl:380

@vchuravy Any quick thoughts before I get familiar with Orc again?

[vchuravy]

The two things I would check is.

1. Are we using the JuliaOJIT interface
2. It sounds like the symbol set the MR is responsible for and the symbols we define through emit here

   GPUCompiler.jl/test/definitions/native.jl

   Line 234 in 5dbe729

   function materialize(mr)

Is different. I vaguely recall discussing this with Lang, but I am unsure what the right answer is...

[maleadt]

I didn't find the issue, and got fed up with LazyCodegen being a test-only thing that no package uses (Enzyme.jl vendors its own copy), no meaningful tests use, breaks all the time (due to the regular changes to upstream Julia's JIT), and does illegal/unsupported stuff (with world ages) that nobody seems to care to fix. I've demoted it to an example for now (sorry @vchuravy), and will probably disable that example for whatever version/configuration of Julia it doesn't support. If anybody wants to maintain it, feel free, but at this point it might be better to put it in a separate package that focusses on integrating GPUCompiler.jl/LLVM.jl/Julia's JIT.

There's also an issue with GCN's non-0 stack addrspace: Emitting code for box_bool, which returns a Type{Bool}, calls into https://github.com/JuliaLang/julia/blob/4768d7ab8fefc8e86c6f9cde09bc10e5a2f72e9b/src/cgutils.cpp#L410-L415 to emit the Bool part of Expr(:invoke, (::Type{InexactError})(Symbol, Any, Any) from (::Type{InexactError})(Symbol, Any, Any), Base.InexactError, :(:Bool), Base.Bool, Core.Argument(n=2)). On e.g. PTX that emits a bitcast from smallp = i8* getelementptr inbounds (i8, i8* @small_typeof, i32 192) to ppjlvalue = {}**, which is fine, but on GCN smallp is i8 addrspace(1)* getelementptr inbounds (i8, i8 addrspace(1)* @small_typeof, i32 192). That results in an illegal bitcast, as an a addrspacecast is required here. @jpsamaroo Can you look into this? I'll be disabling GCN testing for now on relevant configurations.

[vchuravy]

It was intentionally a test only ting since I didn't want to add the burden to GPUCompiler to test the full-fledged monster in Enzyme.

It is a test case for deferred codegen, that actually check that the resolving code is executable.

[vchuravy]

cc: @wsmoses