Some feedback & possible evolutions ideas by @jnastarot

[pthom]

Below, a copy of a message shared by @jnastarot in this issue

---

Hello,
Thank you so much for this awesome project! I truly appreciate the effort that has gone into developing Litgen.

I would like to share my experience and provide some suggestions

Litgen is Useful

Litgen has been much more useful to me compared to tools like SWIG and Rosetta's Binder. The main reason is its flexibility in configuring the processing flow, which allows for more tailored adjustments.
I tried SWIG and Binder, but the bindings they generate are difficult to fix manually

Thoughts

However, I feel that litgen is still in its early stages and could benefit from additional features and improvements.

• Replace regex configuration with callbacks
  Using callbacks instead of regular expressions would enable more robust decision-making. Callbacks allow developers to perform additional checks and apply specific fixes dynamically, which is not always feasible with regex-based configurations.

• Add more options
  Features like those proposed in Callback to customize the base classes used in generated bindings (merged) #27 are essential. Developers using ltgen often have diverse use cases, and more options would make the tool adaptable to their specific needs.

• Minimize header amalgamation
  Header amalgamation is not always desirable, especially in cases involving class derivation. Reducing this dependency or making it optional would enhance flexibility.

• My use case
  I am working on a pet project involving file format dissection. The project includes many files containing numerous classes organized under different namespaces. Header files have specific prefixes, and there are various definitions, such as struct and enum. Sometimes, structures are defined using typedef, and I believe this case should be handled natively by litgen Structures with typedef #7.

• Since my project has a large number of classes, files, and namespaces, I expect bindings to be separated into submodules. Currently, I achieve this by post-processing generated files, using markers like <generated_from> and . It would be fantastic if litgen could handle submodule separation out of the box.

• Performance
  I have noticed that litgen slows down significantly when processing many enums. For example, processing all my headers (~2.5 MB) takes around 40 minutes. I attempted to profile the issue but couldn’t quickly identify the bottleneck. This might be worth investigating further.

I believe litgen has the potential to become as popular and essential as tools like Pybind11 or Nanobind, as well as a valuable complement to other binding tools
Once again, thank you! Litgen has saved me a lot of time

[jnastarot]

I'm also thinking about including a custom __new__ to handle classes with std::enable_shared_from_this. (See https://nanobind.readthedocs.io/en/latest/classes.html#custom-new.)
Maybe it's a bit much, but I think a lot of people use std::shared_ptr and expect objects created in Python to be managed as shared_ptr so they can use them in C++.

I know srcml might not handle inheritance in C++ properly, and std::enable_shared_from_this might not be supported. So, I’m thinking about adding a callback for this feature, like it was done in #27.

[pthom]

This is quite an advanced use case, but interesting. Feel free to explore it and ping me if you have interesting results (with tests of course :-)