Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add new Memory endpoints #889

Closed
valentimarco opened this issue Aug 21, 2024 · 11 comments
Closed

[Feature] Add new Memory endpoints #889

valentimarco opened this issue Aug 21, 2024 · 11 comments
Labels
endpoints Related to http / ws endpoints enhancement New feature or request good first issue Good for newcomers memory

Comments

@valentimarco
Copy link
Member

Is your feature request related to a problem? Please describe.
At the moment, is not possible via RESTAPI:

  1. [GET] Retrieve all the points from a single collection
  2. [GET] Retrieve all the points from all collections
  3. [PUT] Modify a single point from a single collection via qdrant point's ID
  4. [POST] Create a single point in a collection ()
  5. [POST] Create multiple points in a collection (Can be useful for custom points created outside, example knowleage graphs)
  6. [POST] Retrieve all points from a single collection by metadata
  7. [PUT] Modify conversation_history
  8. [POST] Create a custom collection

Describe the solution you'd like
For the 7. point where we need to discuss a little, Maybe is time to a little system of cache?

Additional context
I need to manipulate memory from outside and is difficult to connect my infrastructure with the cat

@valentimarco valentimarco added enhancement New feature or request good first issue Good for newcomers endpoints Related to http / ws endpoints memory labels Aug 21, 2024
@valentimarco valentimarco moved this to 📋 Backlog in Cat Project Kanban Aug 21, 2024
@pieroit
Copy link
Member

pieroit commented Aug 21, 2024

Is your feature request related to a problem? Please describe. At the moment, is not possible via RESTAPI:

  1. [GET] Retrieve all the points from a single collection

All the points? Usually APIs have the whole list of records, but paginated

  1. [GET] Retrieve all the points from all collections
  1. [PUT] Modify a single point from a single collection via qdrant point's ID

Yes this could be useful

  1. [POST] Create a single point in a collection ()

This endpoint is already present in the latest release, here

  1. [POST] Create multiple points in a collection (Can be useful for custom points created outside, example knowleage graphs)

You mean the same as 4, but batched?

  1. [POST] Retrieve all points from a single collection by metadata

Yes this is also useful

  1. [PUT] Modify conversation_history

Agree

  1. [POST] Create a custom collection

Also useful, not sure how complicated it coul be to maintain (es. what happens when embedder changes)

Describe the solution you'd like For the 7. point where we need to discuss a little, Maybe is time to a little system of cache?

Why a cache? It is just an assignation to StrayCat.working_memory.history

Additional context I need to manipulate memory from outside and is difficult to connect my infrastructure with the cat

Thanks for these suggestions, some are precious indeed :D
Some of them are feasible, some other looks like we are reimplementing Qdrant api (which is not useful, as you can talk directly to Qdrant)
Let's see which points to tackle first and open issues specific for them

@pieroit pieroit closed this as completed Aug 21, 2024
@pieroit pieroit reopened this Aug 21, 2024
@github-project-automation github-project-automation bot moved this from 📋 Backlog to ✅ Done in Cat Project Kanban Aug 21, 2024
@pieroit
Copy link
Member

pieroit commented Aug 21, 2024

(sorry closed for a mistake, reopened)

@valentimarco
Copy link
Member Author

Is your feature request related to a problem? Please describe. At the moment, is not possible via RESTAPI:

  1. [GET] Retrieve all the points from a single collection

All the points? Usually APIs have the whole list of records, but paginated

I forgot to write it xd

  1. [POST] Create multiple points in a collection (Can be useful for custom points created outside, example knowleage graphs)

You mean the same as 4, but batched?

Yes

@valentimarco valentimarco moved this from ✅ Done to 📋 Backlog in Cat Project Kanban Aug 21, 2024
@MihirKohli
Copy link
Contributor

Seems like this issue has been resolved

@pieroit
Copy link
Member

pieroit commented Aug 25, 2024

Seems like this issue has been resolved

Welcome, we need to separate these points in different issues so it is possible to contribute individually on them ;)

@MihirKohli
Copy link
Contributor

please do i am more than happy to contribute

@valentimarco
Copy link
Member Author

Seems like this issue has been resolved

Welcome, we need to separate these points in different issues so it is possible to contribute individually on them ;)

Or do It all in ones XD

@valentimarco
Copy link
Member Author

We can also use the power of markdown and do this already done stone to mark which is done

@dave90
Copy link
Contributor

dave90 commented Aug 25, 2024

#892 is for the first 2 items:

  • [GET] Retrieve all the points from a single collection
  • [GET] Retrieve all the points from all collections

@valentimarco
Copy link
Member Author

valentimarco commented Aug 26, 2024

Dev meeting 26/08:
Refactor recall: GET to POST, deprecate the old one and the body will have:

{
 "metadata": {},
"qdrantFilter": {} //qdrant filter is a dict, so you can write the custom filter. By defualt we should support equal for metadata
}

Add [POST] Retrieve all the points from a single collection by metadata
Remove 1.

@pieroit
Copy link
Member

pieroit commented Aug 26, 2024

@valentimarco @MihirKohli @dave90 @lucagobbi closing this issue as it has been separated in specific ones

Thanks for joining!

@pieroit pieroit closed this as completed Aug 26, 2024
@github-project-automation github-project-automation bot moved this from 📋 Backlog to ✅ Done in Cat Project Kanban Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
endpoints Related to http / ws endpoints enhancement New feature or request good first issue Good for newcomers memory
Projects
Status: Done
Development

No branches or pull requests

4 participants