Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create library-specific versions of logits processors #1445

Open
RobinPicard opened this issue Feb 24, 2025 · 0 comments
Open

Create library-specific versions of logits processors #1445

RobinPicard opened this issue Feb 24, 2025 · 0 comments

Comments

@RobinPicard
Copy link
Contributor

First suggested in #1444:

Not for this PR, but I think we'll have to break down the logits processor into library-specific ones. On top of this we'll soon have methods in outlines-core to mask on-device that see optimized for torch, MLX and NumPy. We can already open an issue for that.

The justification for this proposed change is that libraries providing local models (transformers, mlxlm, lamma.cpp...) use different object types and shapes to represent logits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant