Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 9 23 #203

Merged
merged 91 commits into from
Sep 23, 2024
Merged

Upstream merge 24 9 23 #203

merged 91 commits into from
Sep 23, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Sep 23, 2024

Weekly sync from upstream

ElizaWszola and others added 30 commits September 16, 2024 09:47
@ElizaWszola @dsikka
[Kernel] Enable 8-bit weights in Fused Marlin MoE (vllm-project#8032)
a091e2d 
Co-authored-by: Dipika <[email protected]>
@lewtun
[Frontend] Expose revision arg in OpenAI server (vllm-project#8501)
837c196
@njhill
[BugFix] Fix clean shutdown issues (vllm-project#8492)
acd5511
@sasha0552
[Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (vllm-project#8506)
781e3b9
@ProExpertProg
[Kernel] AQ AZP 3/4: Asymmetric quantization kernels (vllm-project#7270)
5d73ae4
@youkaichao
[doc] update doc on testing and debugging (vllm-project#8514)
2759a43
@kevin314
[Bugfix] Bind api server port before starting engine (vllm-project#8491)
47f5e03
@simon-mo
[perf bench] set timeout to debug hanging (vllm-project#8516)
5478c4b
@simon-mo
[misc] small qol fixes for release process (vllm-project#8517)
5ce45eb
@joerunde
[Bugfix] Fix 3.12 builds on main (vllm-project#8510)
cca6164 
Signed-off-by: Joe Runde <[email protected]>
@simon-mo
[refactor] remove triton based sampler (vllm-project#8524)
546034b
@alex-jw-brooks
[Frontend] Improve Nullable kv Arg Parsing (vllm-project#8525)
1c1bb38 
Signed-off-by: Alex-Brooks <[email protected]>
@ywang96
[Misc][Bugfix] Disable guided decoding for mistral tokenizer (vllm-pr…
ee2bcea 
…oject#8521)
@youkaichao
[torch.compile] register allreduce operations as custom ops (vllm-pro…
99aa4ed 
…ject#8526)
@ruisearch42
[Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (v…
cbdb252 
…llm-project#8509)

Signed-off-by: Rui Qiao <[email protected]>
@Isotr0py
[Benchmark] Support sample from HF datasets and image input for bench…
1b6de83 
…mark_serving (vllm-project#8495)
@sroy745
[Encoder decoder] Add cuda graph support during decoding for encoder-…
1009e93 
…decoder models (vllm-project#7631)
@chenqianfzh
[Feature][kernel] tensor parallelism with bitsandbytes quantization (v…
9855b99 
…llm-project#8434)
@patrickvonplaten @DarkLight1337
[Model] Add mistral function calling format to all models loaded with…
a54ed80 
… "mistral" format (vllm-project#8515)

Co-authored-by: Cyrus Leung <[email protected]>
@njhill
[Misc] Don't dump contents of kvcache tensors on errors (vllm-project…
56c3de0 
…#8527)
@joerunde
[Bugfix] Fix TP > 1 for new granite (vllm-project#8544)
98f9713 
Signed-off-by: Joe Runde <[email protected]>
@youkaichao @Imss27
[doc] improve installation doc (vllm-project#8550)
fa0c114 
Co-authored-by: Andy Dai <[email protected]>
@alexeykondrat
[CI/Build] Excluding kernels/test_gguf.py from ROCm (vllm-project#8520)
09deb47
@tlrmchlsmth
[Kernel] Change interface to Mamba causal_conv1d_update for continuou…
8110e44 
…s batching (vllm-project#8012)
@dtrifiro
[CI/Build] fix Dockerfile.cpu on podman (vllm-project#8540)
95965d3
@Jeffwan
[Misc] Add argument to disable FastAPI docs (vllm-project#8554)
e351572
@DarkLight1337
[CI/Build] Avoid CUDA initialization (vllm-project#8534)
6ffa3f3
@aarnphm @DarkLight1337
[CI/Build] Update Ruff version (vllm-project#8469)
9d104b5 
Signed-off-by: Aaron Pham <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@alexm-redhat @njhill
@robertgshaw2-redhat @simon-mo
[Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (v…
7c7714d 
…llm-project#8157)

Co-authored-by: Nick Hill <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
Co-authored-by: Simon Mo <[email protected]>
@afeldman-nm
[Core] *Prompt* logprobs support in Multi-step (vllm-project#8199)
a8c1d16
rasmith and others added 22 commits September 21, 2024 09:20
@rasmith
[Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x…
ec4aaad 
… speedup MI300, minor improvement for MI250 (vllm-project#8646)
@divakar-amd
[dbrx] refactor dbrx experts to extend FusedMoe class (vllm-project#8518
9dc7c6c 
)
@tlrmchlsmth
[Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (v…
d66ac62 
…llm-project#8643)
@Isotr0py
[Bugfix] Refactor composite weight loading logic (vllm-project#8656)
13d88d4
@youkaichao
[ci][build] fix vllm-flash-attn (vllm-project#8699)
0e40ac9
@DarkLight1337
[Model] Refactor BLIP/BLIP-2 to support composite model loading (vllm…
06ed281 
…-project#8407)
@alex-jw-brooks
[Misc] Use NamedTuple in Multi-image example (vllm-project#8705)
8ca5051 
Signed-off-by: Alex-Brooks <[email protected]>
@ji-huazhong
[MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (vllm-projec…
ca2b628 
…t#8703)
@litianjian @DarkLight1337 @ywang96
[Model][VLM] Add LLaVA-Onevision model support (vllm-project#8486)
5b59532 
Co-authored-by: litianjian <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@LiuXiaoxuanPKU
[SpecDec][Misc] Cleanup, remove bonus token logic. (vllm-project#8701)
c6bd70d
@youkaichao
[build] enable existing pytorch (for GH200, aarch64, nightly) (vllm-p…
d4a2ac8 
…roject#8713)
@youkaichao
[misc] upgrade mistral-common (vllm-project#8715)
92ba7e7
@tlrmchlsmth
[Bugfix] Avoid some bogus messages RE CUTLASS's revision when building (
3dda7c2 
vllm-project#8702)
@ProExpertProg @zhouyuan
[Bugfix] Fix CPU CMake build (vllm-project#8723)
57a0702 
Co-authored-by: Yuan <[email protected]>
@yma11
[Bugfix] fix docker build for xpu (vllm-project#8652)
d23679e
@alex-jw-brooks
[Core][Frontend] Support Passing Multimodal Processor Kwargs (vllm-pr…
9b8c8ba 
…oject#8657)

Signed-off-by: Alex-Brooks <[email protected]>
@Isotr0py
[Hardware][CPU] Refactor CPU model runner (vllm-project#8729)
e551ca1
@bigPYJ1151
[Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model…
3e83c12 
…_runner (vllm-project#8733)
@liuyanyi
[Model] Support pp for qwen2-vl (vllm-project#8696)
a79e522
@janimo
[VLM] Fix paligemma, fuyu and persimmon with transformers 4.45 : use …
f2bd246 
…config.text_config.vocab_size (vllm-project#8707)
@gshtras
Merge remote-tracking branch 'upstream/main' into upstream_merge_24_9_23
87acddd
@gshtras
isort
7e2ac48
@gshtras gshtras force-pushed the upstream_merge_24_9_23 branch from 53e9c28 to 7e2ac48 Compare September 23, 2024 17:22
@gshtras gshtras requested a review from shajrawi September 23, 2024 18:48
@gshtras
Bias and more metadata in gradlib and tuned gemm (#202)
6e370fc 
* Adding bias to hipb_mm in gradlib. Expanding gradlib to tune based on bias and dtype captured from the actual capture run
@gshtras gshtras force-pushed the upstream_merge_24_9_23 branch from ba59c31 to 6e370fc Compare September 23, 2024 19:06
shajrawi
shajrawi approved these changes Sep 23, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this

@gshtras gshtras merged commit 57ea101 into main Sep 23, 2024
16 of 17 checks passed
@gshtras gshtras deleted the upstream_merge_24_9_23 branch September 23, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.