Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Host normalization when building IndexConnection #56

Merged
merged 5 commits into from
Aug 2, 2024

Conversation

austin-denoble
Copy link
Contributor

Problem

We had an issue that was logged around trying to call Client.Index() with a Host value including an http:// or https:// schema. The control plane does not attach these schemes, but when building an IndexConnection a user needs to provide a host and they could make this mistake. grpc-go is not expecting a target in NewClient that includes a scheme.

We could do a bit of normalization to handle the Host more thoroughly.

Solution

  • Add normalizeHost helper function in index_connection.go. This strips out http:// & https:// prefixes, and attaches the port :443 unless the user has provided one directly. We want to leave some flexibility for making RPCs in specific scenarios.
  • Add unit tests for normalizeHost.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Infrastructure change (CI configs, etc)
  • Non-code change (docs, etc)
  • None of the above: (explain here)

Test Plan

CI unit & integration tests should pass as normal.

To test manually, you can try and prepend schemes to the Host when calling client.Index()

indexConn, err := client.Index(pinecone.NewIndexConnParams{Host: fmt.Sprintf("https://%s", index.Host)})
if err != nil {
    panic(err)
}

stats, err := indexConn.DescribeIndexStats(context.Background())

@@ -1007,6 +1007,16 @@ func (idx *IndexConnection) delete(ctx context.Context, req *data.DeleteRequest)
return err
}

func (idx *IndexConnection) akCtx(ctx context.Context) context.Context {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this up to group with the other IndexConnection methods.

Copy link
Contributor

@haruska haruska left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bit of an oddity with stripping http:// from a host then defaulting to port 443 with TLS enabled on the connection.

{
name: "http:// scheme should be removed",
host: "http://this-is-my-host.io",
expectedHost: "this-is-my-host.io:443",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit odd... I would expect a http:// transport to default to port 80.

However, I also don't see a user specifying http plaintext without a port for an index.

@@ -3,8 +3,9 @@ package pinecone
import (
"context"
"crypto/tls"
"fmt"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this is meant to be deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was in my original PR because I removed the target := fmt.Sprintf("%s:443", in.host) which was the only usage in here previously.

Once rebasing on your changes though we have those fmt.Errorf calls with the input struct checking.

Comment on lines +1009 to +1019
name: "http:// scheme without a port should be removed",
host: "http://this-is-my-host.io",
expectedHost: "this-is-my-host.io:443",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference btwn this test and the 1st test? It looks like they both start and end with the same values, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nothing, accidentally duplicated it!

Comment on lines +1013 to +1023
name: "http:// scheme and port should be maintained",
host: "http://this-is-my-host.io:8080",
expectedHost: "http://this-is-my-host.io:8080",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice to add an iteration here that does https with a port, too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think I meant to add that case and kept one the same through copy pasta or something. 😅

Copy link
Contributor

@aulorbe aulorbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking, just some Qs. Great job!

@austin-denoble austin-denoble force-pushed the adenoble/add-http-validation branch from b1dca21 to 8533c61 Compare August 2, 2024 19:30
@austin-denoble austin-denoble merged commit a28aa6c into main Aug 2, 2024
3 checks passed
@austin-denoble austin-denoble deleted the adenoble/add-http-validation branch August 2, 2024 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants