Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

promhttp: vendor the minimal, noasm version of klauspost/compress/zstd. #1767

Closed
wants to merge 1 commit into from

Conversation

bwplotka
Copy link
Member

@bwplotka bwplotka commented Mar 6, 2025

This proposes alternative to PR solving Kubernetes concerns around klauspost/compress dependency.

I propose vendoring a stripped zstd pkg (with required deps like "le", "fse" and "huff") that has only code needed for FastSpeed mode for encoding (writing). Removed tests, ASM, unsafe code (trivial - klauspost/compress allows noasm, nounsafe tags). I propose we do 1.20.x, 1.21.x with this. 1.20.x specifically to unblock Kubernetes ASAP.

NOTE: We still have klauspost/compress in go.mod because we use it in tests (for testing promhttp zstd encoding). All importers will NOT have it as indirect dependency.

Pros:

  • We don't break users, we can easily patch 1.20 too.
  • It's only tmp; there is a path forward -- zstd should be supported in std lib soon.
  • We already have tmp vendored code, so we have precedence.

Cons:

  • In case of security vulns. maintainability here will be more difficult.
  • Low test coverage for the vendor -- I removed all tests, we only have one test for zstd in promhttp.

cc @liggitt @kakkoyun

@bwplotka
Copy link
Member Author

bwplotka commented Mar 6, 2025

Tests are dead, because Dagger is still on this branch. If we need to backport this, we need to remove dagger too, will do later... Also some things are still unused and can be removed. I can get that to finish line if we want to pursue this.

@@ -26,7 +26,7 @@ import (
"testing"
"time"

"github.com/klauspost/compress/zstd"
orignalzstd "github.com/klauspost/compress/zstd"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we use the vendored package here as well? 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question! I mentioned in the description, but to reiterate:

  1. To reduce the vendored code by at least 50% (we only need encoding). I am pulling the original dep for test purposes only to decode for validation.
  2. Extra benefit is the validation against original, stable decoder so we know we did not mess something in our vendored code that makes the result encodable only for us :p

@bwplotka
Copy link
Member Author

bwplotka commented Mar 6, 2025

Seems like we are leaning towards #1765 (review) - closing this for now.

@bwplotka bwplotka closed this Mar 6, 2025
@kakkoyun kakkoyun deleted the zstdvendor1.20 branch March 7, 2025 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants