Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: token speed is low while using Jan Windows on GPU NVIDIA 4070ti #1002

Closed
Tracked by #1063
hiento09 opened this issue Dec 14, 2023 · 1 comment
Closed
Tracked by #1063

bug: token speed is low while using Jan Windows on GPU NVIDIA 4070ti #1002

hiento09 opened this issue Dec 14, 2023 · 1 comment
Assignees
@hiento09
Labels
type: bug Something isn't working
Milestone
Jan Windows MVP

Comments

@hiento09
Copy link
Collaborator

hiento09 commented Dec 14, 2023 •
edited
Loading

Describe the bug
token speed is low while using Jan Windows on GPU NVIDIA 4070ti, just around 6/s-7/s

Screenshots
image

Desktop (please complete the following information):

  • OS: Windows 11
  • RAM 64GB
  • GPU: RTX 4070ti
  • Jan App Version: 0.4.1
  • Model: Mistral Instruct 7B Q4
@hiento09 hiento09 added the type: bug Something isn't working label Dec 14, 2023
@hiento09 hiento09 added this to Menlo Dec 14, 2023
@freelerobot freelerobot added this to the Jan on Windows milestone Dec 14, 2023
@hiento09
Copy link
Collaborator Author

hiento09 commented Dec 14, 2023 •
edited
Loading

The root cause was mentioned in janhq/cortex.cpp#269

@freelerobot freelerobot moved this to Triaged (Backlog) in Menlo Dec 14, 2023
@freelerobot freelerobot moved this from Triaged (Backlog) to Todo in Menlo Dec 18, 2023
@louis-menlo louis-menlo assigned hiento09 and unassigned linhtran174 Dec 19, 2023
@hiento09 hiento09 moved this from Planned to In Progress in Menlo Dec 20, 2023
@tikikun tikikun removed their assignment Dec 21, 2023
@louis-menlo louis-menlo mentioned this issue Dec 21, 2023
Closed
79 tasks
@hiento09 hiento09 mentioned this issue Dec 21, 2023
Merged
@github-project-automation github-project-automation bot moved this from In Progress to Done in Menlo Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hiento09 hiento09

Labels
type: bug Something isn't working
Projects
Menlo
Archived in project
Milestone
Jan Windows MVP
Development

No branches or pull requests
5 participants
@linhtran174 @tikikun @freelerobot @dan-menlo @hiento09