Skip to content

Issues: open-compass/VLMEvalKit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

69 Open 264 Closed
69 Open 264 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

socre 0 for GPT4o_20241120 model on CCOCR_DocParsing_TablePhotoEng
#851 opened Mar 13, 2025 by nutsintheshell
About prompts in different models and datasets
#848 opened Mar 12, 2025 by kydxh
@FangXinyu-0913
5
OCRVQA、GQA数据集的prediction里跟有.句号，导致准确率为0
#842 opened Mar 11, 2025 by WalkerWorldPeace
@FangXinyu-0913
1
Inputs without JPEG conversion
#839 opened Mar 10, 2025 by kydxh
@FangXinyu-0913
2
InternVL2.5模型在USE_COT测试时精度均下降了？
#836 opened Mar 8, 2025 by zzk6626
@PhoenixZ810
2
MMbench1.1??
#833 opened Mar 6, 2025 by vinchary
@PhoenixZ810
1
请问关于TextVQA的评测现在是正常的吗？
#830 opened Mar 6, 2025 by kydxh
无法支持云端模型api？（比如硅基流动）
#829 opened Mar 6, 2025 by tsiens
@PhoenixZ810
5
run.py不支持传入本地模型
#828 opened Mar 5, 2025 by kkscilife
@PhoenixZ810
1
Failed to download MMBench_V11.tsv error
#823 opened Mar 3, 2025 by OfirArviv
@PhoenixZ810
2
Ola模型自测与OpenCompass榜单差距很大
#822 opened Mar 3, 2025 by bobo0810
@PhoenixZ810
4
是否支持采样评估部分数据集？
#820 opened Mar 1, 2025 by zhangyimi
@FangXinyu-0913
2
[BUG?] 使用同样的模型，在此 repo下跑推理与模型官方（例如 LLaVA）的推理 demo 结果不同
#817 opened Feb 28, 2025 by ZI-MA
Unable to reproduce Qwen2.5-VL-7B-Instruct
#813 opened Feb 27, 2025 by bollossom
@PhoenixZ810
1
关于使用 --config 的问题
#811 opened Feb 26, 2025 by yaogang2060
关于llava-cot的ChartQA的评估问题
#810 opened Feb 26, 2025 by nth2000
78B模型推理加速（78B Modeling Reasoning Acceleration）
#807 opened Feb 25, 2025 by zfr00
@PhoenixZ810
6
两次相同的测试结果，同一个评估模型，结果差异很大
#806 opened Feb 25, 2025 by slZheng077
@PhoenixZ810
9
会评测Baichuan-Omni-1.5吗？
#801 opened Feb 24, 2025 by bobo0810
How to run VITA1.0 on 4*80G GPUs?
#800 opened Feb 23, 2025 by Espere-1119-Song
@PhoenixZ810
14
自定义模型多卡推理问题，想一机多卡，每卡一个模型，并行推理一个测评
#799 opened Feb 22, 2025 by bjzhb666
@PhoenixZ810
6
Lmdeploy cannot be used to inference and eval
#798 opened Feb 22, 2025 by ligeng0197
@PhoenixZ810
1
qwen2.5VL-7B-Instruct 模型 MMMU(val)评测与官方不符
#797 opened Feb 21, 2025 by shawn0wang
@PhoenixZ810
10
使用qwen2.5_VL fine-tune后评测出现错误，是怎么回事
#791 opened Feb 20, 2025 by shawn0wang
1
Would VLLM be used to accelerate model inference?
#789 opened Feb 19, 2025 by ligeng0197
@PhoenixZ810
3
ProTip! Mix and match filters to narrow down what you’re looking for.