Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break down ConstrainedLogitsProcessor into library-specific logits processors #1446

Open
rlouf opened this issue Feb 24, 2025 · 1 comment
Open

Comments

@rlouf
Copy link
Member

rlouf commented Feb 24, 2025

We found working on #1444 that MLXLM has different conventions for the tensor shapes from the other libraries. The code was already very complex because we need conditionals to identify the type of array. On top of this we'll soon have methods in outlines-core to mask on-device that see optimized for torch, MLX and NumPy.

I thus recommend we implement as many logits processors as there are base libraries / underlying tensor libraries.

@rlouf
Copy link
Member Author

rlouf commented Feb 24, 2025

Closing as duplicate of #1445

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant