Break down ConstrainedLogitsProcessor
into library-specific logits processors
#1446
Labels
ConstrainedLogitsProcessor
into library-specific logits processors
#1446
We found working on #1444 that MLXLM has different conventions for the tensor shapes from the other libraries. The code was already very complex because we need conditionals to identify the type of array. On top of this we'll soon have methods in outlines-core to mask on-device that see optimized for torch, MLX and NumPy.
I thus recommend we implement as many logits processors as there are base libraries / underlying tensor libraries.
The text was updated successfully, but these errors were encountered: