You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
As of 0.9.1, there are several options in the help that are not recognized.
--seed
--min-p
--top-k
--top-p
--samplers
Additionally, some args are partially supported:
--prompt is unknown but -p works.
--file is unknown but -f works.
I assume the help is forked from llama.cpp or is somehow linked.
Motivation
Models like QwQ-32B have recommended samplers from Qwen so I would like to be able to use their recommended settings to stop looping at the end of a reply.
Possible Implementation
I noted #692 is about --repeat-penalty wasn't supported in the 0.8 stream but is fixed as of 0.9.1 so I assume these args need to be added to some list. I think that issue can be closed.
There may be more but these are the args that are not supported at all that I have found.
The text was updated successfully, but these errors were encountered:
yep similar. upstream sync is needed. not sure when it might happen.
however if someone submits a PR adding support for some of these directly it will be pulled in. i hear you on QwQ, would be good to have proper support
Prerequisites
Feature Description
As of 0.9.1, there are several options in the help that are not recognized.
Additionally, some args are partially supported:
--prompt
is unknown but-p
works.--file
is unknown but-f
works.I assume the help is forked from llama.cpp or is somehow linked.
Motivation
Models like QwQ-32B have recommended samplers from Qwen so I would like to be able to use their recommended settings to stop looping at the end of a reply.
Possible Implementation
I noted #692 is about --repeat-penalty wasn't supported in the 0.8 stream but is fixed as of 0.9.1 so I assume these args need to be added to some list. I think that issue can be closed.
There may be more but these are the args that are not supported at all that I have found.
The text was updated successfully, but these errors were encountered: