-
Notifications
You must be signed in to change notification settings - Fork 13k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[llvm] Support llvm::Any across shared libraries on windows #108051
Conversation
@llvm/pr-subscribers-clang-analysis @llvm/pr-subscribers-llvm-transforms Author: Thomas Fransham (fsfod) ChangesThis is part of the effort to support for enabling plugins on windows by adding better support for building llvm as a DLL. The export macros used here were added in #96630 Since shared library symbols aren't deduplicated across multiple libraries on windows like Linux we have to manually explicitly import and export I also removed the only use of llvm::Any for LoopNest that only existed in debug code and there also doesn't seem to be any code creating Full diff: https://github.com/llvm/llvm-project/pull/108051.diff 6 Files Affected:
diff --git a/llvm/include/llvm/Analysis/LazyCallGraph.h b/llvm/include/llvm/Analysis/LazyCallGraph.h
index e7fd18967d9bed..55060f506c3330 100644
--- a/llvm/include/llvm/Analysis/LazyCallGraph.h
+++ b/llvm/include/llvm/Analysis/LazyCallGraph.h
@@ -34,6 +34,7 @@
#ifndef LLVM_ANALYSIS_LAZYCALLGRAPH_H
#define LLVM_ANALYSIS_LAZYCALLGRAPH_H
+#include "llvm/ADT/Any.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/PointerIntPair.h"
@@ -1308,6 +1309,10 @@ class LazyCallGraphDOTPrinterPass
static bool isRequired() { return true; }
};
+#ifdef _WIN32
+extern template struct LLVM_TEMPLATE_ABI
+ Any::TypeId<const LazyCallGraph::SCC *>;
+#endif
} // end namespace llvm
#endif // LLVM_ANALYSIS_LAZYCALLGRAPH_H
diff --git a/llvm/include/llvm/IR/PassInstrumentation.h b/llvm/include/llvm/IR/PassInstrumentation.h
index 9fcc2d5957a30c..c41c287f5f1e99 100644
--- a/llvm/include/llvm/IR/PassInstrumentation.h
+++ b/llvm/include/llvm/IR/PassInstrumentation.h
@@ -50,10 +50,11 @@
#define LLVM_IR_PASSINSTRUMENTATION_H
#include "llvm/ADT/Any.h"
+#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/FunctionExtras.h"
#include "llvm/ADT/SmallVector.h"
-#include "llvm/ADT/DenseMap.h"
#include "llvm/IR/PassManager.h"
+#include "llvm/Support/Compiler.h"
#include <type_traits>
#include <vector>
@@ -61,6 +62,15 @@ namespace llvm {
class PreservedAnalyses;
class StringRef;
+class Module;
+class Loop;
+class Function;
+
+#ifdef _WIN32
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Module *>;
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Function *>;
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Loop *>;
+#endif
/// This class manages callbacks registration, as well as provides a way for
/// PassInstrumentation to pass control to the registered callbacks.
diff --git a/llvm/lib/Analysis/LazyCallGraph.cpp b/llvm/lib/Analysis/LazyCallGraph.cpp
index e6bf8c9cbb289f..9d74bce98122bb 100644
--- a/llvm/lib/Analysis/LazyCallGraph.cpp
+++ b/llvm/lib/Analysis/LazyCallGraph.cpp
@@ -37,6 +37,10 @@ using namespace llvm;
#define DEBUG_TYPE "lcg"
+#ifdef _WIN32
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const LazyCallGraph::SCC *>;
+#endif
+
void LazyCallGraph::EdgeSequence::insertEdgeInternal(Node &TargetN,
Edge::Kind EK) {
EdgeIndexMap.try_emplace(&TargetN, Edges.size());
diff --git a/llvm/lib/IR/PassInstrumentation.cpp b/llvm/lib/IR/PassInstrumentation.cpp
index 0c4e7698d9fa87..134990eee26988 100644
--- a/llvm/lib/IR/PassInstrumentation.cpp
+++ b/llvm/lib/IR/PassInstrumentation.cpp
@@ -17,6 +17,12 @@
namespace llvm {
+#ifdef _WIN32
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Module *>;
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Function *>;
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Loop *>;
+#endif
+
void PassInstrumentationCallbacks::addClassToPassName(StringRef ClassName,
StringRef PassName) {
ClassToPassName.try_emplace(ClassName, PassName.str());
diff --git a/llvm/lib/Transforms/Scalar/LoopPassManager.cpp b/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
index a4f2dbf9a58289..cc1921d61791a4 100644
--- a/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
+++ b/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
@@ -273,8 +273,6 @@ PreservedAnalyses FunctionToLoopPassAdaptor::run(Function &F,
llvm::any_cast<const LoopNest *>(&IR));
const Loop **LPtr = llvm::any_cast<const Loop *>(&IR);
const Loop *L = LPtr ? *LPtr : nullptr;
- if (!L)
- L = &llvm::any_cast<const LoopNest *>(IR)->getOutermostLoop();
assert(L && "Loop should be valid for printing");
// Verify the loop structure and LCSSA form before visiting the loop.
diff --git a/llvm/unittests/IR/PassBuilderCallbacksTest.cpp b/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
index 6230aed7b7119b..9aad6e3ca91255 100644
--- a/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
+++ b/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
@@ -298,8 +298,6 @@ template <> std::string getName(const Any &WrappedIR) {
return (*F)->getName().str();
if (const auto *const *L = llvm::any_cast<const Loop *>(&WrappedIR))
return (*L)->getName().str();
- if (const auto *const *L = llvm::any_cast<const LoopNest *>(&WrappedIR))
- return (*L)->getName().str();
if (const auto *const *C =
llvm::any_cast<const LazyCallGraph::SCC *>(&WrappedIR))
return (*C)->getName();
|
@llvm/pr-subscribers-llvm-analysis Author: Thomas Fransham (fsfod) ChangesThis is part of the effort to support for enabling plugins on windows by adding better support for building llvm as a DLL. The export macros used here were added in #96630 Since shared library symbols aren't deduplicated across multiple libraries on windows like Linux we have to manually explicitly import and export I also removed the only use of llvm::Any for LoopNest that only existed in debug code and there also doesn't seem to be any code creating Full diff: https://github.com/llvm/llvm-project/pull/108051.diff 6 Files Affected:
diff --git a/llvm/include/llvm/Analysis/LazyCallGraph.h b/llvm/include/llvm/Analysis/LazyCallGraph.h
index e7fd18967d9bed..55060f506c3330 100644
--- a/llvm/include/llvm/Analysis/LazyCallGraph.h
+++ b/llvm/include/llvm/Analysis/LazyCallGraph.h
@@ -34,6 +34,7 @@
#ifndef LLVM_ANALYSIS_LAZYCALLGRAPH_H
#define LLVM_ANALYSIS_LAZYCALLGRAPH_H
+#include "llvm/ADT/Any.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/PointerIntPair.h"
@@ -1308,6 +1309,10 @@ class LazyCallGraphDOTPrinterPass
static bool isRequired() { return true; }
};
+#ifdef _WIN32
+extern template struct LLVM_TEMPLATE_ABI
+ Any::TypeId<const LazyCallGraph::SCC *>;
+#endif
} // end namespace llvm
#endif // LLVM_ANALYSIS_LAZYCALLGRAPH_H
diff --git a/llvm/include/llvm/IR/PassInstrumentation.h b/llvm/include/llvm/IR/PassInstrumentation.h
index 9fcc2d5957a30c..c41c287f5f1e99 100644
--- a/llvm/include/llvm/IR/PassInstrumentation.h
+++ b/llvm/include/llvm/IR/PassInstrumentation.h
@@ -50,10 +50,11 @@
#define LLVM_IR_PASSINSTRUMENTATION_H
#include "llvm/ADT/Any.h"
+#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/FunctionExtras.h"
#include "llvm/ADT/SmallVector.h"
-#include "llvm/ADT/DenseMap.h"
#include "llvm/IR/PassManager.h"
+#include "llvm/Support/Compiler.h"
#include <type_traits>
#include <vector>
@@ -61,6 +62,15 @@ namespace llvm {
class PreservedAnalyses;
class StringRef;
+class Module;
+class Loop;
+class Function;
+
+#ifdef _WIN32
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Module *>;
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Function *>;
+extern template struct LLVM_TEMPLATE_ABI Any::TypeId<const Loop *>;
+#endif
/// This class manages callbacks registration, as well as provides a way for
/// PassInstrumentation to pass control to the registered callbacks.
diff --git a/llvm/lib/Analysis/LazyCallGraph.cpp b/llvm/lib/Analysis/LazyCallGraph.cpp
index e6bf8c9cbb289f..9d74bce98122bb 100644
--- a/llvm/lib/Analysis/LazyCallGraph.cpp
+++ b/llvm/lib/Analysis/LazyCallGraph.cpp
@@ -37,6 +37,10 @@ using namespace llvm;
#define DEBUG_TYPE "lcg"
+#ifdef _WIN32
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const LazyCallGraph::SCC *>;
+#endif
+
void LazyCallGraph::EdgeSequence::insertEdgeInternal(Node &TargetN,
Edge::Kind EK) {
EdgeIndexMap.try_emplace(&TargetN, Edges.size());
diff --git a/llvm/lib/IR/PassInstrumentation.cpp b/llvm/lib/IR/PassInstrumentation.cpp
index 0c4e7698d9fa87..134990eee26988 100644
--- a/llvm/lib/IR/PassInstrumentation.cpp
+++ b/llvm/lib/IR/PassInstrumentation.cpp
@@ -17,6 +17,12 @@
namespace llvm {
+#ifdef _WIN32
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Module *>;
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Function *>;
+template struct LLVM_EXPORT_TEMPLATE Any::TypeId<const Loop *>;
+#endif
+
void PassInstrumentationCallbacks::addClassToPassName(StringRef ClassName,
StringRef PassName) {
ClassToPassName.try_emplace(ClassName, PassName.str());
diff --git a/llvm/lib/Transforms/Scalar/LoopPassManager.cpp b/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
index a4f2dbf9a58289..cc1921d61791a4 100644
--- a/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
+++ b/llvm/lib/Transforms/Scalar/LoopPassManager.cpp
@@ -273,8 +273,6 @@ PreservedAnalyses FunctionToLoopPassAdaptor::run(Function &F,
llvm::any_cast<const LoopNest *>(&IR));
const Loop **LPtr = llvm::any_cast<const Loop *>(&IR);
const Loop *L = LPtr ? *LPtr : nullptr;
- if (!L)
- L = &llvm::any_cast<const LoopNest *>(IR)->getOutermostLoop();
assert(L && "Loop should be valid for printing");
// Verify the loop structure and LCSSA form before visiting the loop.
diff --git a/llvm/unittests/IR/PassBuilderCallbacksTest.cpp b/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
index 6230aed7b7119b..9aad6e3ca91255 100644
--- a/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
+++ b/llvm/unittests/IR/PassBuilderCallbacksTest.cpp
@@ -298,8 +298,6 @@ template <> std::string getName(const Any &WrappedIR) {
return (*F)->getName().str();
if (const auto *const *L = llvm::any_cast<const Loop *>(&WrappedIR))
return (*L)->getName().str();
- if (const auto *const *L = llvm::any_cast<const LoopNest *>(&WrappedIR))
- return (*L)->getName().str();
if (const auto *const *C =
llvm::any_cast<const LazyCallGraph::SCC *>(&WrappedIR))
return (*C)->getName();
|
Can we make another pass over this PR? |
@@ -273,8 +273,6 @@ PreservedAnalyses FunctionToLoopPassAdaptor::run(Function &F, | |||
llvm::any_cast<const LoopNest *>(&IR)); | |||
const Loop **LPtr = llvm::any_cast<const Loop *>(&IR); | |||
const Loop *L = LPtr ? *LPtr : nullptr; | |||
if (!L) | |||
L = &llvm::any_cast<const LoopNest *>(IR)->getOutermostLoop(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an any_cast<const LoopNest *>
in an assertion left above.
Though generally I don't get why you're removing the LoopNest support here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess i missed the line above, but unless i was misunderstood layers of template template instantiation i found no code creating Any with LoopNest when i checked before.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To add some more context it looks like LoopPassManager::runSinglePass would be the path a LoopNest Any could be created but it decayed to a Loop from a overload of getLoopFromIR that takes a LoopNest and returns a Loop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, so the code intentionally passes the outermost loop instead of the LoopNest to pass instrumentation. In that case these changes are fine.
81203df
to
d336c45
Compare
✅ With the latest revision this PR passed the C/C++ code formatter. |
#include "llvm/ADT/STLExtras.h" | ||
#include "llvm/ADT/STLFunctionalExtras.h" | ||
#include "llvm/ADT/SmallVector.h" | ||
#include "llvm/Support/Errc.h" | ||
#include "llvm/Support/Error.h" | ||
|
||
namespace clang { | ||
namespace dataflow { | ||
class NoopLattice; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this forward declaration placed in this file, when the NoopLattice is defined in its own file? Regardless, if it does belong here, please comment to explain the intent. I'm a code owner and yet I don't understand what this is doing and how/what I should do to prevent it from breaking in the case of changes. IIUC, this change is Window-specific and seems fragile, making it all the more critical to clearly document.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its used as type in a extern template declaration below, The extern template declaration needs to be seen by code using any_cast with NoopLattice to allow external code to use the same address for the Any: Id for NoopLattice for shared library builds on windows , but i guess it really could go in the NoopLattice header. I can make whatever changes you think would be best.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry -- I was asking about the whole block, not just the forward decl. Yes, please move it the NoopLattice header. Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LLVM changes LGTM
@@ -273,8 +273,6 @@ PreservedAnalyses FunctionToLoopPassAdaptor::run(Function &F, | |||
llvm::any_cast<const LoopNest *>(&IR)); | |||
const Loop **LPtr = llvm::any_cast<const Loop *>(&IR); | |||
const Loop *L = LPtr ? *LPtr : nullptr; | |||
if (!L) | |||
L = &llvm::any_cast<const LoopNest *>(IR)->getOutermostLoop(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, so the code intentionally passes the outermost loop instead of the LoopNest to pass instrumentation. In that case these changes are fine.
…ted form Explicitly import and export Any::TypeId template instantiations for uses of llvm::Any in the LLVM codebase to support LLVM Windows shared library builds. This change is required to allow external code to use PassManager callbacks including LLVM's own tests for it. Remove the only use of llvm::Any for LoopNest that only existed in debug code and there was no code creating Any<LoopNest>
59e94fd
to
662a96b
Compare
@ymand, is your concern resolved? |
Sorry, it isn't -- a) we still need comments with at least minimal explanation and b) I'm not sure why the definition is now in TypeErasedDataflowAnalysis.cpp -- is this because we lack NoopAnalysis.cpp? If so, why is TypeErasedDataflowAnalysis.cpp then the right place? or, should we create a NoopAnalysis.cpp? |
What are the implications/requirements for users of DataflowAnalysis? Every instance specifices it's own lattice, which is converted to/from llvm::Any. Will every lattice definition require these new declarations? |
Yes there was no NoopAnalysis.cpp, if you fine with me creating one just to only contain an explicit instantiation i can move it there. |
If you think they will be used from external from the llvm library including by tests then yes. |
I see -- that's a bit of a problem. At the least, this requires very clear documentation -- but even better would be if there's a way to hide this in the implementation somehow, rather than pushing it on the user. Can you think of anything that would abstract this? I'd even propose a macro, but that's not enough because of demands on the namespacing. This seems unfortunately incompatible with any sort of modularity. :( Do you have reference to an Issue tracking this or, even better, an RFC to the clang (or llvm) community? I'd like to better understand the context to see if we (analysis owners) can think of a way to avoid this. I'll also see if we can rip llvm::Any out of our implementation to avoid this altogether. |
I discussed with other code owners -- we think our best option is to remove our use of llvm::Any entirely. So, I think you can go ahead with the changes here (no need for a new NoopLattice.cpp), but please add some comments explaining what's going on (even a pointer to some explanation elsewhere woudl be fine). Meantime, I'm going to work on a PR to remove use of Any. Thanks! |
idk if this was kinda the explanation you were after or not https://discourse.llvm.org/t/supporting-llvm-build-llvm-dylib-on-windows/58891 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the discussion/explanation and the comments!
Indeed, thank you! |
) This is part of the effort to support for enabling plugins on windows by adding better support for building llvm as a DLL. The export macros used here were added in llvm#96630 Since shared library symbols aren't deduplicated across multiple libraries on windows like Linux we have to manually explicitly import and export `Any::TypeId` template instantiations for the uses of `llvm::Any` in the LLVM codebase to support LLVM Windows shared library builds. This change ensures that external code, including LLVM's own tests, can use PassManager callbacks when LLVM is built as a DLL. I also removed the only use of llvm::Any for LoopNest that only existed in debug code and there also doesn't seem to be any code creating `Any<LoopNest>`
This is part of the effort to support for enabling plugins on windows by adding better support for building llvm as a DLL. The export macros used here were added in #96630
Since shared library symbols aren't deduplicated across multiple libraries on windows like Linux we have to manually explicitly import and export
Any::TypeId
template instantiations for the uses ofllvm::Any
in the LLVM codebase to support LLVM Windows shared library builds.This change ensures that external code, including LLVM's own tests, can use PassManager callbacks when LLVM is built as a DLL.
I also removed the only use of llvm::Any for LoopNest that only existed in debug code and there also doesn't seem to be any code creating
Any<LoopNest>