-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No tools_call in message error on ChatVertexAI #5475
Comments
The error you're encountering, "No tools_call in message," suggests that the input message does not prompt any tool calls, likely because it doesn't match the expected patterns or the structured output does not include tool calls. To address this, ensure your input message is designed to trigger tool calls and that the schema passed to Here's a step-by-step approach to troubleshoot and potentially resolve the issue:
Here is a simplified example of how you might adjust your code: import { z } from "zod";
import { ChatOpenAI } from "@langchain/openai";
const calculatorSchema = z.object({
operation: z.enum(["add", "subtract", "multiply", "divide"]),
number1: z.number(),
number2: z.number(),
});
const modelWithStructuredOutput = chatOpenAIInstance.withStructuredOutput(calculatorSchema, {
name: "calculator",
includeRaw: true, // Optional: Include raw output for debugging
}); Ensure your input message and schema are correctly aligned with what ChatVertexAI is expected to produce. If the issue persists, providing more details about when the error occurs could help further diagnose the problem.
|
This seems odd. But looking into it. The system info dependencies list doesn't show any of the @langchain/google-* modules. Can you confirm which ones are installed and their versions? |
@afirstenberg sure ![]() |
Same problem here. When running in verbose mode it looks like the response when using withStructuredInput is an empty string.
|
same issue here |
Same issue here, same problem as well. Works fine without using
|
Natively support non-streaming mode.
Fixes for other tests.
While the name of the fix doesn't look like it should address it, #5571 should fix the issue. Thanks @talhaFayyazfolio for the test case, which made it much easier to verify. If you can, check out the patch to see if it looks good to you before merged. |
@afirstenberg thanks for this fix. It looks good to me . |
Thank you so much @afirstenberg! |
Fix is live in 0.0.17 of your |
Tested with the new live version in 0.0.17. If I do not attach a file input, the withOutputParser works great! However, if I attach a file, I am getting an error when I try to invoke the function. Example code below:
This looks to be the relevant error message:
From the sound of it, the issue seems to be with VertexAI / Gemini itself. However, in my mind it would seem a little odd if this is the case - This seems like it would be something relatively important. I was hoping we can confirm that this is expected behavior from the |
Okay actually it looks like this needs to be opened back up. I am getting the
The error message from the above comment is different from the error message in this one (which is expected, as csvs are plain texts). https://ai.google.dev/gemini-api/docs/prompting_with_media?lang=python |
Will check it out - I wonder if it's not forcing the tool call. What if you prompt more strongly? |
I'm investigating, but I wanted to point a few things out while I do:
|
Oh yes, sorry about that 😂 it was late last night. I mostly just wanted to point out that the csv is interpreted as text by it's file format definition. I tried with a regular text file as well and resulted in the same issue. I'm actually pretty surprised that vertex AI doesn't support it - perhaps it just interprets as text/plain instead? |
I can't duplicate the problem. It looks like Vertex AI is ok with "text/csv", so the documentation seems to be wrong (big shock). My test code: const zodChatMessage = z.object({
chatMessage: z.string().optional().describe("the chat message answer responding to the user's question"),
chatCitation: z.string().optional().describe('1-3 key sentences from the text segment justifying the extracted chat message'),
chatPage: z.number().optional().describe('the page number where this answer was extracted from')
});
const model = new ChatVertexAI({
temperature: 0.7,
model: 'gemini-1.5-pro',
}).withStructuredOutput(zodChatMessage);
const filePath = "src/tests/data/structuredOutput.csv";
const fileString = loadFile(filePath);
const inputContent: MessageContent = [{ type: 'text', text: 'What is the price of a shrimp cocktail?' }];
inputContent.push({ type: 'image_url', image_url: `data:text/csv;base64,${fileString}` });
const input = [new HumanMessage({ content: inputContent })];
const response = await model.invoke(input);
console.log(response); Gives a response:
When I run the test using the raw JSON it gives me: {
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": "The price of the Shrimp Cocktail is $13.50. \n"
},
{
"functionCall": {
"name": "extract",
"args": {
"chatMessage": "The price of the Shrimp Cocktail is $13.50."
}
}
}
]
},
"finishReason": "STOP",
"safetyRatings": [
{
"category": "HARM_CATEGORY_HATE_SPEECH",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.111627996,
"severity": "HARM_SEVERITY_NEGLIGIBLE",
"severityScore": 0.1362632
},
{
"category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.34620965,
"severity": "HARM_SEVERITY_NEGLIGIBLE",
"severityScore": 0.1468197
},
{
"category": "HARM_CATEGORY_HARASSMENT",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.24526574,
"severity": "HARM_SEVERITY_NEGLIGIBLE",
"severityScore": 0.08787644
},
{
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.17009404,
"severity": "HARM_SEVERITY_NEGLIGIBLE",
"severityScore": 0.07356305
}
]
}
],
"usageMetadata": {
"promptTokenCount": 421,
"candidatesTokenCount": 33,
"totalTokenCount": 454
}
} which seems correct. The only thing I can guess about why my prompt is working and yours isn't is that by the time Gemini gets the tokens, it doesn't see a "file" anywhere. It just sees more tokens. So a prompt that asks about a "file" doesn't make any sense to it unless there is something that explicitly says "here is a file" or something. The CSV file is a pretty straightforward CSV file that contains a menu item, description, and price. |
Okay I am using a different computer from the one last night, however it seems like the output is still quite inconsistent. When attaching a .csv, I will sometimes get the proper output structure, whereas other times it will not be correct (i.e. null vs. a number or optional), resulting in a error. For example:
It seems to like to put "null" as the response for optional inputs, however if you use zod's Lastly, it looks like you can still result in a |
I think the only way to avoid any possibility of "no tools_call" is to add a "fallback" tool and to force it to reply with a tool using the ANY function calling mode (which we haven't implemented support for yet - see #5072). Gemini ignoring the type seems odd, however. I haven't encountered that before. I'm not sure there is anything that LangChain.js can do differently aside from implementing #5072, but I'm open to suggestions. |
* docs: remove contextual search (langchain-ai#5378) * google-genai[minor]: Add support for video/audio inputs (langchain-ai#5368) * wip * added audio/video * genai update support media types * cr * chore: lint files * cr * all[patch]: Loosen core dependencies (langchain-ai#5367) * all[patch]: Loosen core dependencies * make community and lc depend on core rc * resoultions * community[patch]: Release 0.2.0-rc.0 (langchain-ai#5384) * partnet packages[patch]: Release all (langchain-ai#5388) * partnet packages[patch]: Release all * bump other google packages * docs: feedback link in banner (langchain-ai#5387) * anthropic[patch]: Run formatting (langchain-ai#5390) * docs[patch]: update api ref base url to contain v2 subdomain (langchain-ai#5392) * Update debugging and streaming guide content (langchain-ai#5395) * Fix link (langchain-ai#5396) * docs[minor]: Delete 'old' docs folder & contents (langchain-ai#5394) * Guide content fixes (langchain-ai#5400) * Update default headers azure & Add Token Provider Check (langchain-ai#5379) * Update Default Headers for Azure OpenAI Requests * nit * Update user agent string * Add Token Provider check * Update libs/langchain-openai/src/azure/llms.ts Co-authored-by: Yohan Lasorsa <[email protected]> * Update libs/langchain-openai/src/azure/embeddings.ts Co-authored-by: Yohan Lasorsa <[email protected]> * Update libs/langchain-openai/src/azure/chat_models.ts Co-authored-by: Yohan Lasorsa <[email protected]> * Format the code --------- Co-authored-by: Yohan Lasorsa <[email protected]> * openai[patch]: 0.0.31 (langchain-ai#5401) (langchain-ai#5402) * docs[patch]: LangGraph Docs remove unused import from code examples (langchain-ai#5391) Co-authored-by: Brace Sproul <[email protected]> * core[major]: Update streamEvent signature to return IterableReadableStream (langchain-ai#5360) * Update streamEvent signature to return IterableReadableStream * Fix lint * Update return type to be sync * docs[patch]: Move installation doc to how tos (langchain-ai#5399) * docs[patch]: Move instalation doc to how tos * update sidebar * cr * fix[runOnDataset]: accept custom criteria in config factory * docs[patch]: Add logprob docs, more updates (langchain-ai#5404) * Add logprob docs, more updates * More content updates * docs[patch]: Add onlyWsa prop to chat model tabs, other nits (langchain-ai#5405) * docs[patch]: Add onlyWsa prop to chat model tabs, other nits * fix link * chore: lint files * cr * docs[minor]: Show sidebar inside how to/tutorial (langchain-ai#5406) * docs[minor]: Show sidebar inside how to/tutorial * chore: lint files * cr * trailing slash * docs: fix monorepo typo (langchain-ai#5411) * docs[minor]: Match py how to index page (langchain-ai#5414) * docs[minor]: Match py how to index page * chore: lint files * anthropic[minor]: Add tool_choice arg (langchain-ai#5416) * anthropic[minor]: Add tool_choice arg * chore: lint files * release 0.1.19 * langchain[patch]: Add deprecation warnings to document loaders (langchain-ai#5419) * langchain[patch]: Add deprecation warnings to document loaders * chore: lint files * docs[patch]: Broken links * docs[patch]: Fix some broken links * docs[patch]: Fix broken links (langchain-ai#5420) * push * cr * make docusourus warn not throw on broken links * chore: lint files * Update text splitter docs (langchain-ai#5424) * docs[minor]: Hide prev/next buttons on how to and tutorials (langchain-ai#5425) * docs[minor]: Hide prev/next buttons on how to and tutorials * chore: lint files * chore: lint files * docs[minor]: Log actual prompt when using prompt hub (langchain-ai#5423) * docs[minor]: Log actual prompt when using prompt hub * nit * docs[minor]: version dropdown (langchain-ai#5422) * docs: version dropdown * Update docs/core_docs/docusaurus.config.js * Update docs/core_docs/docusaurus.config.js * docs[minor]: Add tools prompting docs (langchain-ai#5431) * docs[minor]: Add tools prompting docs * fix outputs * docs[patch]: Update API ref url in docs to v02 (langchain-ai#5432) * docs[patch]: Update retrieval and embeddings docs (langchain-ai#5429) * Update retrieval and embeddings docs * Remove silly rule * Replace prerequisite links component (langchain-ai#5430) * api_refs[minor]: Add version dropdown] (langchain-ai#5435) * core[minor]: Unified model params for LS (langchain-ai#5427) * core[minor]: Unified model params for LS * add to openai, anthropic and together * mistral * cohere fireworks ollama * fix tests * remove underscore prefix * fix metadata placement * format * fixes * chore: lint files * docs[patch]: Remove links to missing docs (langchain-ai#5438) * docs[patch]: Remove links to missing docs * fix title * core[patch]: Move async generator consumption code into local storage context (langchain-ai#5439) * Move async generator consumption code into local storage context * Typing fix, adds test, remove unnecessary void * lint n stuff --------- Co-authored-by: bracesproul <[email protected]> * core[patch]: Release 0.2.0 (next) (langchain-ai#5443) * environment_tests[minor],langchain[minor]: Upgrade core dep to 0.2.0 (langchain-ai#5444) * langchain[minor]: Upgrade core dep to 0.2.0 * update export tests to not use rc core version * langchain[patch]: Release 0.2.0 (next) (langchain-ai#5445) * community[minor]: Bump LC and core deps to 0.2.0 (langchain-ai#5446) * community[patch]: Release 0.2.0 (next) (langchain-ai#5447) * @langchain/google-genai [feature]: support Google genai API version and base URL (langchain-ai#5426) * feat: support Google genai API version and base URL * run lint n format --------- Co-authored-by: Brace Sproul <[email protected]> * all[minor]: Update partner packages core dep semver (langchain-ai#5448) * partner packages[patch]: Release all (langchain-ai#5449) * google-pkgs[patch]: Release Google packages (langchain-ai#5450) * google-pkgs[patch]: Release Google packages * chore: lint files * partner-packages[patch]: Update core dep versions to contain single space (langchain-ai#5452) * partner-packages[patch]: Update core dep versions to contain single space * cr * Fix styling (langchain-ai#5454) * docs[major]: Fix broken links and turn on throw on broken link (langchain-ai#5455) * docs[major]: Fix broken links and turn on throw on broken link * chore: lint files * Update QA use-case docs how tos (langchain-ai#5453) * Update other howto use-case docs (langchain-ai#5457) * docs[patch]: Update tools docs (langchain-ai#5458) * Update tools docs * Links * core[patch]: Fix bindTools return type (langchain-ai#5459) * Fix bindTools return type * Format * groq[patch]: Release 0.0.12 (langchain-ai#5461) * anthropic[patch]: Release 0.1.21 (langchain-ai#5462) * openai[patch]: Release 0.0.33 (langchain-ai#5463) * google-common[patch]: Release 0.0.16 (langchain-ai#5464) * mistralai[patch]: Remove default tool call value for translated messages (langchain-ai#5428) * mistral[patch]: Remove default tool call value for translated messages * Fix * Fix * Improve error handling * mistralai[patch]: Release 0.0.22 (langchain-ai#5465) * docs[patch]: Adds self query docs, LCEL cheatsheet (langchain-ai#5466) * Adds self query docs, LCEL cheatsheet * Bump deps * Add dep * Deno dep update * docs[patch]: Contributor guide, polish (langchain-ai#5468) * Contributor guide, polish * Typo * Fix build * docs[patch]: Adds callback docs (langchain-ai#5469) * Adds callback docs * Updates * Update banner * Update copy * Rename doc page (langchain-ai#5470) * docs[patch]: Update tutorials (langchain-ai#5473) * Update tutorials * Fix LangSmith section: * community[patch]: Neo4j metadata filtering (langchain-ai#5215) * Implement addGraphDocuments method for neo4j graph~ * Add integration tests for new method and make them pass. * Revert back to int skipped, because remote db * Fix Serializable extension * Make metadata optional * Linting * Format * Update extensions and props passing * Format * Add metadata filtering * Fix metadata test equality check. Format. * Skip int test. * Refactor IndexType from enum to plain type * Move filter arg to params * Format * google-common[minor],google-genai[minor]: Standardize LS params (langchain-ai#5486) * community[minor]: Adds support for Cohere Command-R via AWS Bedrock (langchain-ai#5336) * add-cohere-command-r-formatted-messages * add integration test patterns * fixed typo * add comments * fixed input system prompts check * fixed cohere modelName validation * fixed lint * Small typo fix * Add special message parsing for Cohere --------- Co-authored-by: jacoblee93 <[email protected]> * docs[patch]: Update output parser docs (langchain-ai#5408) * Update output parser docs * Typo * Remove bad link * Fix broken links * all[patch]: Ignore no unused vars eslint rule in test files (langchain-ai#5440) * all[patch]: Ignore no unused vars eslint rule in test files * chore: lint files * google-genai[patch]: Fix api options which caused requests to hang (langchain-ai#5490) * google-genai[patch]: Release 0.0.15 (langchain-ai#5491) * docs[patch]: update Azure documentation and deprecate @langchain/azure-openai (langchain-ai#5477) * docs: update Azure OpenAI chat documentation * docs: update azure chat openai * docs: update Azure OpenAI LLM docs * docs: update docs for Azure OpenAI embeddings * docs: update wording * docs: fix example path * docs: add deprecation notice * docs: update Microsoft integrations docs * chore: fix typo * docs: fix install * Format --------- Co-authored-by: jacoblee93 <[email protected]> * Fix docs infinite redirect, add redirect (langchain-ai#5492) * Replace ecosystem pages with real links (langchain-ai#5494) * community[minor]: updated Browserbase loader (langchain-ai#5412) * updated browserbase loader * ran format * Move to community * Format * Loosen peer dep --------- Co-authored-by: Mish Ushakov <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Add Google Routes Tool (langchain-ai#5329) * Initial routes feature * Update field masks, removed json mapping * Fixed JSON return and better fieldMask * working condition for all types of transport * Removed departure and arrival time due to timezone issues * Better prompt, better typescript, remove warning, fixed duplicated labels * Working routes tool * Add GoogleRoutesAPI interface and remove console.log statements * Update Google Routes Tool documentation * Updated example to include a example output * Helper functions for better readability * testing for google routes tool * Changed tool to StructuredTool, input is now an object * Updates tests for google routes tool * Better description * Removed some lines to make explanation simpler * Add TollInfo interface and update filteredRoutes function in google_routes.ts * Working version with other features * Fix handling of past departure and arrival times in GoogleRoutesAPI * Format * Skip tests by default --------- Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Spider Document Loader (langchain-ai#5415) * files * format * update yarn.lock * new version of spider-client * new spider-client version * spider constructor * correct doc * change ref link * commented out params in example * updated spider-client version * changed location of files * change location of files * changed file location * updated correct imports * Update spider.ts --------- Co-authored-by: Brace Sproul <[email protected]> Co-authored-by: Jacob Lee <[email protected]> * community[minor]: feat: Layerup Security integration (langchain-ai#4929) * feat: Layerup Security integration * nit: remove environment variable reference * docs: Layerup Security docs * fix: package.json, fix docs location * feat: bump layerup-security version, support untrusted_input * nit: LLM -> BaseLLM * Update lock * Build and lint fixes * fix: bump layerup-security SDK version to 1.5.9 * Update lockfile * feat: bump SDK to 1.5.10 to match OpenAI requirement * Bump version * fix: type errors * Bump dep * Rename test --------- Co-authored-by: jacoblee93 <[email protected]> * docs[minor]: Unstructured MD loader doc (langchain-ai#5489) * docs[minor]: Unstructured MD loader doc * chore: lint files * html loader * update firecrawl loader * fix firecrawl loader * cr * Add links to docs from how to index * cr * chore: lint files * community[patch]: VoyageAI embedding with input_type parameter (langchain-ai#5493) * added input_type * input_type set to "none" by default , then choose between document or query * Updated input_type for Voyage embeddings * Update voyage.ts * Update voyageai.mdx * Update voyageai.mdx * Update voyage.ts * Update voyageai.mdx ¯\_(ツ)_/¯ * Update voyage.int.test.ts With document to make sure it works * Format * Update voyageai.mdx is it the space ? (sorry...) * Format * Fix types --------- Co-authored-by: jacoblee93 <[email protected]> * Fix build artifacts (langchain-ai#5496) * Temporary build fix (langchain-ai#5499) * community[patch]: Release 0.2.1 (langchain-ai#5500) * scripts[minor]: Fix extra newlines issue, switch all packages to use workspace dep (langchain-ai#5498) * scripts[minor]: Fix extra newlines issue, switch all packages to use workspace dep * cr * docs[minor]: Add AI SDK docs (langchain-ai#5497) * docs[minor]: Add AI SDK docs * chore: lint files * ai streaming tools doc * chore: lint files * drop docs, move links to tutorials section * chore: lint files * Add neo4j metadata filtering docs (langchain-ai#5502) * langchain[patch]: Add entrypoints to deprecated omit import map (langchain-ai#5511) * langchain[patch]: Add entrypoints to deprecated omit import map * rm rf build artifacts * langchain[patch]: Release 0.2.1 (langchain-ai#5512) * google-genai[minor]: Add tool calling support (langchain-ai#5507) * google-genai[minor]: Add tool calling support * cr * chore: lint files * fixes * docs and build error fixes * dont set name in response * add ls links * cr * Multi modal docs * :cr * fixup multi modal tool calls * cr * google-genai[patch]: Release 0.0.16 (langchain-ai#5513) * community[minor]: Allow more forms of credentials for aws bedrock (langchain-ai#5514) * community[minor]: Allow more forms of credentials for aws bedrock * chore: lint files * docs: move feedback into paginator from content * core[patch]: Add support for messages in/messages out for RunnableWithMessageHistory (langchain-ai#5517) * Add support for messages in/messages out for RunnableWithMessageHistory * Add test * docs[patch]: Fix lint (langchain-ai#5526) * Fix lint * Fix * community[patch]: Release 0.2.2 (langchain-ai#5527) * chore: Update `@google-ai/generativelanguage` module to 2.5.0 (langchain-ai#5523) Signed-off-by: Oleg Ivaniv <[email protected]> * Bump package (langchain-ai#5529) * docs[minor]: Add generative UI docs (langchain-ai#5528) * docs[minor]: Add generative UI docs * chore: lint files * broken link * langchain[patch]: Readd dropped entrypoint (langchain-ai#5530) * Fix memory entrypoint * Update examples * Release 0.2.2 (langchain-ai#5531) * scripts[patch]: Add .js extension to build script (langchain-ai#5536) * scripts[patch]: Add .js extension to build script * cr * yarn.lock * commit it * docs[minor]: Multi modal docs (langchain-ai#5537) * docs[minor]: Multi modal docs * cr * scripts[major]: New build (langchain-ai#5538) * scripts[major]: New build * perfect build script and add to core! * chore: lint files * very nice much better * yarn format && yarn lint:fix * all build script updates * standard clean scrop[t] * add turbo to every repo * add back test:ci * remove concurrency limits in tests & ignore non testable packages * core[minor]: Stream events v2 (langchain-ai#5539) * Start stream events v2 * More progress * Fixes * Docs * Docs * Fix build * community[minor]: Add Upstash Embeddings Support (langchain-ai#5150) * add: metadata filtering for UpstashVectorStore * fmt * skip tests * add: upstash embeddings support * add: tests * fmt * Update upstash.mdx * Update upstash.mdx * use fake embeddings * fix: replace UpstashEmbeddings parameter with FakeEmbeddings class * Naming --------- Co-authored-by: Jacob Lee <[email protected]> * community[patch]: langchain-ai#3369 Streaming support for Replicate models (langchain-ai#5365) * langchain-community[patch]: langchain-ai#3369 Streaming support for Replicate models * lock * Add streaming test --------- Co-authored-by: jeasonnow <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Add Moonshot chat models integration (langchain-ai#5239) * Add Moonshot chat models * fix: MOONSHOT_API_KEY needs to be capitalized * feat: remove moonshotApiKey * fix: remove encodeApiKey * docs: complete parameter comment * Format * Lint --------- Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Integrate zep cloud components (langchain-ai#5542) * feat: Add Zep Cloud Langchain components + corresponding docs * chore: Run formatter * chore: Throw an error on unsupported vector store methods * Fix lint, update deprecated code --------- Co-authored-by: paulpaliychuk <[email protected]> * qdrant[minor]: Support maxMarginalRelevanceSearch() (langchain-ai#5467) * community[minor]: Add UpstashRatelimitHandler (langchain-ai#5474) * add upstash ratelimit callback * add ratelimit package and imports * improve error messages * fix tests * fix typo * Polish, format, lint --------- Co-authored-by: jacoblee93 <[email protected]> * community[minor]: add Neo4j Graph enhancedSchema option (langchain-ai#5413) * Implement addGraphDocuments method for neo4j graph~ * Add integration tests for new method and make them pass. * Revert back to int skipped, because remote db * Fix Serializable extension * Make metadata optional * Linting * Format * Update extensions and props passing * Format * Add enhanced schema option. * Format * Update test * Update test * Format --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: add ?| (arrayContains) filter on metadata to PGVector search (langchain-ai#5381) * add ?| (arrayContains) filter on metadata to PGVector search * fix: align to style of IN filter * Format * Update build artifacts * Format --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: Better errors for neo4j vector (langchain-ai#5501) * Better errors for neo4j vector * Lint --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: Default embeddingNodeProperty value (langchain-ai#5510) * Default embeddingNodeProperty value If this property is undefined when calling `Neo4jVectorStore.fromExistingGraph()`, the property is named `undefined`. * Format --------- Co-authored-by: jacoblee93 <[email protected]> * docs[patch]: update callbacks (langchain-ai#5515) * x * x * x * Format * Update --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: include metadata returned by Amazon bedrock knowledge base (langchain-ai#5535) * include metadata returned by Amazon bedrock knowledge base * Update amazon_knowledge_base.ts * Update dep --------- Co-authored-by: Jacob Lee <[email protected]> * preserve ordinality in postgres when checking existence of documents (langchain-ai#5533) * docs[minor]: LangGraph Migration Guide (langchain-ai#5487) * [Docs] LangGraph Migration Guide * fixup * link * Update * Update and polish * Add to how to index page --------- Co-authored-by: jacoblee93 <[email protected]> * qdrant[patch]: Release 0.0.5 (langchain-ai#5543) * community[patch]: Release 0.2.3 (langchain-ai#5544) * core[patch]: Release 0.2.1 (langchain-ai#5545) * pinecone[patch],weaviate[patch]: Bump weaviate and pinecone core deps (langchain-ai#5546) * Bump Pinecone core deps * Bump weaviate core dependency * pinecone[patch]: Release 0.0.7 (langchain-ai#5547) * weaviate[patch]: Release 0.0.4 (langchain-ai#5548) * openai[patch]: fix langchain-ai#5520 (langchain-ai#5521) * langchain-openai[patch]: fix langchain-ai#5520 * fix: don't rewrite config if they are already set in embedding and chat_model * Format --------- Co-authored-by: jeasonnow <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * openai[patch]: Release 0.0.34 (langchain-ai#5562) * core[patch]: Keep event stream for streamEvents v2 open until end of run (langchain-ai#5561) * Keep event stream for streamEvents v2 open until end of run * Style nit * Lint * Small fix * core[patch]: Release 0.2.2 (langchain-ai#5563) * docs[patch]: Missing how to index section for multi modal (langchain-ai#5575) * docs[patch]: Missing how to index section for multi modal * format * api_refs[patch]: Fix link to v0.1 api refs (langchain-ai#5576) * infra[minor]: Speed improvements for CI (langchain-ai#5580) * infra[minor]: Speed improvements for CI * cr * infra[patch]: Improve yarn format (langchain-ai#5578) * infra[patch]: Improve yarn format * chore: lint files * cr * Fixed Typo on Build Simple LLM application page (langchain-ai#5566) typo * google-common [patch], google-* [tests]: Fix streaming false (langchain-ai#5571) * Fix for issue langchain-ai#5475 - no tools_call in message. Natively support non-streaming mode. * Test for structuredOutput (see langchain-ai#5475). Fixes for other tests. * Fixes for tests. * Formatting. * fix: moonshot did not return usage cause error (langchain-ai#5551) * google-common[patch]: Zod to Gemini params conversion when the schema contains arrays of items (langchain-ai#5553) * Fix: [libs/langchain-google-common] already existing utils test * Fix: [lib/langchain-google-common] Add failing test for arrays containing additional properties * Fix: [lib/langchain-google-common] Fix failing test * Update int test * lint --------- Co-authored-by: jacoblee93 <[email protected]> * google-common[patch]: Release 0.0.17 (langchain-ai#5588) * google-*[patch]: Release 0.0.17 (langchain-ai#5589) * langchain[patch]: Support for sqljs (langchain-ai#5560) * Support for sqljs Added support for sqljs, the browser transpile of sqlite * Format --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: Upstash Vector Store Namespace Feature (langchain-ai#5557) * feat: Upstash Vector Namespace feature * add: Upstash Vector Namespace Tests * docs: Upstash Vector namespace * fmt * infra[patch]: Set fail-fast strategy to false (langchain-ai#5590) * langchain[patch],community[patch]: Loosen peer deps (langchain-ai#5583) * langchain[patch],community[patch]: Loosen peer deps * loosen more deps and cut community rc * cut rc for langchain and community again * drop unused google pkg from langchain * Release 0.2.3 (langchain-ai#5591) * community[patch]: Release 0.2.4 (langchain-ai#5592) * infra[major]: Commit yarn cache (langchain-ai#5593) * community[patch]: support stream for wenxin and zhipu chat (langchain-ai#5573) * langchain-community[patch]: support stream for wenxin and zhipu chat model * Format * refactor: remove stream util --------- Co-authored-by: jeasonnow <[email protected]> * community[patch]: add support for or in elastic vector query (langchain-ai#5568) * langchain[patch]: add support for or in elastic vector query Fixes langchain-ai#5558. const filter = [ { field: "genre", operator: "or", value: "Drama" }, { field: "genre", operator: "or", value: "Action" }, ]; const reteriver = vectorStore.asRetriever({ filter: filter ) or ScoreThresholdRetriever.fromVectorStore(vectorStore, { minSimilarityScore: 0.5, // Finds results with at least this similarity score maxK: 1, // The maximum K value to use. Use it based to your chunk size to make sure you don't run out of tokens kIncrement: 1, // How much to increase K by each time. It'll fetch N results, then N + kIncrement, then N + kIncrement * 2, etc. filter: filter, }); After this change the filter will be passed to the elastic vector similaritySearchVectorWithScore function. const relevantDocuments = await elasticVectorSearch.similaritySearchVectorWithScore( [/* your query vector */], 10, filter ); * Added type for should * Format * Lint --------- Co-authored-by: jacoblee93 <[email protected]> * core[patch]: Support LANGSMITH_TRACING env var (langchain-ai#5587) * [Core:Tracing] accept LANGSMITH_TRACING * Rework check --------- Co-authored-by: jacoblee93 <[email protected]> * core[patch]: Set global async local storage instance (langchain-ai#5601) * Set global async local storage instance * Rename * Rename * core[minor]: Make LLMs and chat models always stream when invoked within streamEvents (langchain-ai#5604) * Make LLMs and chat models always stream when invoked within streamEvents * Lint * Lint * core[patch]: Release 0.2.3 (langchain-ai#5606) * docs[patch]: Update README and other links (langchain-ai#5605) * Update README and other links * nit * Transparent handoff between @Traceable and LangChain * Add LangSmith to langchain-core internals * Fix CallbackManagerRunTree * Use singletons * Use singletons * Resolve circular dependency * Inherit more properties * Remove comment, now that we have a singleton * Use [email protected] * Fix base * Add package * Upstream changes from SDK * Update to [email protected] * Update base.ts * core[patch]: Release 0.2.4 (langchain-ai#5609) * docs: redirect integration links to 0.2 (langchain-ai#5608) * mistralai[minor]: Add llms entrypoint, update chat model integration (langchain-ai#5603) * mistralai[minor]: Add llms entrypoint * chore: lint files * cr * docs * docs nit * docs nit * update mistral-large models to mistral-large-latest * fix tools * add codestral chat model tests * chore: lint files * cr * mistralai[patch]: Release 0.0.23 (langchain-ai#5613) * langchain[patch],core[patch]: Fix agent executor stream event behavior (langchain-ai#5614) * Fix agent executor stream event behavior * Clean up streamEvents implementation, small remote runnable fix * Fix lint * core[minor],openai[patch]: Add usage metadata to `AIMessage`/`Chunk` (langchain-ai#5586) * core[minor],openai[patch]: Add usage metadata to base ai message * chore: lint files * Only set stream_options if the model is OpenAI * add test confirming last finish reason is stop * chore: lint files * docs * lint n format * cr * cr * Remove Azure specific check * Remove comment --------- Co-authored-by: jacoblee93 <[email protected]> * core[patch]: Release 0.2.5 (langchain-ai#5615) * Bump OpenAI deps (langchain-ai#5616) * openai[minor]: Release 0.1.0 (langchain-ai#5617) * community[patch],langchain[patch],groq[patch]: Bump deps (langchain-ai#5618) * Bump deps * Fix * Release 0.2.4 (langchain-ai#5619) * Release 0.2.4 * Update deps * community[minor]: DeepInfra embeddings integration #1 (langchain-ai#5382) * Init * fix(type errors) * feat(deepinfra embeddings) * fix(default model) * fix(deepinfra): axios is removed * ref(deepinfra): remove redundant cast * format(deepinfra) * doc(deepinfra) * doc(deepinfra) * Update deepinfra.mdx * Format --------- Co-authored-by: Jacob Lee <[email protected]> * Fix bug in Neo4j new enhancedSchema flag (langchain-ai#5622) * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings (langchain-ai#5476) * community: Add support for Tencent Hunyuan chat model and embeddings Tencent provides its Hunyuan chat model and embeddings through Tencent cloud. This PR adds support for the [chat model](https://cloud.tencent.com/document/product/1729/105701) as well as [embedding](https://cloud.tencent.com/document/product/1729/102832). * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * refactor Chat Model streaming implementation with AsyncGenerator * update stream example * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * make host configurable * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * support both nodejs and browser environment * update documents and examples * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * make separate chat model and embedding entrypoints for web vs. node * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * rename docs sidebar name * community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings * format code * Update build, lint, format * Type export --------- Co-authored-by: jacoblee93 <[email protected]> * Fix security header (langchain-ai#5625) * community[patch]: overriding similaritySearchWithScore in neo4j vector store to enable filtering (langchain-ai#5599) * overriding similaritySearchWithScore * Format * Fix build --------- Co-authored-by: jacoblee93 <[email protected]> * LangGraph fix typo (langchain-ai#5626) * Update import map (langchain-ai#5627) * Remove yarn cache (langchain-ai#5628) * community[patch]: Release 0.2.5 (langchain-ai#5629) * 👥 Update LangChain people data (langchain-ai#5630) Co-authored-by: github-actions <[email protected]> * Bump flaky better-sqlite3 dep (langchain-ai#5633) * openai[patch],anthropic[patch]: Populate usage_metadata on invoke, add docs (langchain-ai#5632) * Populate usage_metadata in more places * Add usage metadata for Anthropic * Remove unnecessary check * Update artifacts * Fix typo * Fix multimodal tool call docs * openai[patch]: Release 0.1.1 (langchain-ai#5634) * anthropic[minor]: Release 0.2.0 (langchain-ai#5635) * Switch to monthly shields (langchain-ai#5642) * docs: fix package path (langchain-ai#5653) * Update googlevertexai.mdx (langchain-ai#5638) Fix import paths * Fix JSDoc parameter description typo (langchain-ai#5644) * docs[patch]: Update Unstructured Docker Image Reference in Documentation (langchain-ai#5651) * Update unstructured.mdx * Update document_loader_html.ipynb * Update document_loader_markdown.ipynb * docs[patch]: Add keywords for common queries (langchain-ai#5655) * Add keywords for common queries * Format * standard-tests[major], openai[minor]: Init package & add standard tests to openai (langchain-ai#5612) * standard-tests[major]: Init package * fix build, cleanup * base class and unit tests * format n lint * updates * Added standard unit tests to oai and scripts, made getLsParams public * yarn install * always include api key * set env var instead of constructor args * added integration tests * add int tests to openai * add wso tests * Add to root & turbo, added gh action * usage metadata tests for streaming and invoke * Added call options arg to integration tests * tmp run standard test action in pr ci * remove from pr ci * test moar things, add readme * chore: lint files * token usage standard tests non streaming * docs[patch]: Sitemap fixes (langchain-ai#5658) * Sitemap fixes * Update * anthropic[minor]: Add standard chat model tests to anthropic (langchain-ai#5659) * anthropic[minor]: Add standard chat model tests to anthropic * format * docs[patch]: Update quickstart tutorial (langchain-ai#5662) * Update quickstart tutorial * Broken link * mongodb[minor]: add, implement delete method (langchain-ai#5559) * feat: atlas search vector store delete method Method delete was missing in MongoDBAtlasVectorSearch class and it is needed if incremental indexing strategy is used * refactor: use built-in function for array chunking --------- Co-authored-by: Guille <[email protected]> * mongodb[patch]: Release 0.0.4 (langchain-ai#5664) * community[minor]: upgraded @mlc/web-llm dependency and updated it's ChatModel (langchain-ai#5637) * chore(community/webllm): upgraded @mlc/web-llm dependency and updated it's ChatModel * Format --------- Co-authored-by: jacoblee93 <[email protected]> * langchain[minor]: add EnsembleRetriever (langchain-ai#5556) * langchain[patch]: add support to merge retrievers * Format * Parallelize, lint, format, small fixes * Add entrypoint * Fix import * Add docs and fix build artifacts --------- Co-authored-by: jeasonnow <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Add support for bedrock guardrails and trace (langchain-ai#5631) * add support for bedrock guardrails and trace * updated bedrock tests for guardrails * resolved guardrail and trace format errors, also resolved type errors for fetchFn * updated chatbedrock test with guardrails * Format and revert call option changes * Lint --------- Co-authored-by: jacoblee93 <[email protected]> * Release 0.2.5 (langchain-ai#5665) * docs[patch]: Redirects for LangSmith and LangGraph (langchain-ai#5667) * LangGraph docs link redirect * LangSmith redirect * community[patch]: Release 0.2.6 (langchain-ai#5668) * partners[minor]: Add standard chat model tests to partner packages (langchain-ai#5660) * partners[minor]: Add standard chat model tests to partner packages * google genai * yarn * groq * groq nit and mistral * add to azure in chat openai * chore: lint files * drop azure openai pkg * add generic constructor args to standard tests pkg * implement cloudflare standard tests * implement cohere standard tests * google genai package standard tests * groq * allow for custom function call ids, fix mistral * azure tests * chore: lint files * update standard tests gh action to run all pkgs * chore: lint files * revert workflow file rename * fix workflow job naming issue * add anthropic, fix api keys * cache deps? * fix build * update standard tests * cr * fix * remove dep on job which doesnt exist * cr * cr * core[patch]: Fix formatting mustache image templates (langchain-ai#5666) * core[patch]: Fix formatting mustache image templates * unfocus test * fix tests * community[minor]: Add standard tests to community chat models (langchain-ai#5669) * partners[minor]: Add standard chat model tests to partner packages * google genai * yarn * groq * groq nit and mistral * add to azure in chat openai * chore: lint files * drop azure openai pkg * add generic constructor args to standard tests pkg * implement cloudflare standard tests * implement cohere standard tests * google genai package standard tests * groq * allow for custom function call ids, fix mistral * azure tests * chore: lint files * update standard tests gh action to run all pkgs * chore: lint files * revert workflow file rename * fix workflow job naming issue * add anthropic, fix api keys * cache deps? * fix build * update standard tests * cr * fix * remove dep on job which doesnt exist * cr * cr * community[minor]: Add standard tests to community chat models * integration tests * chore: lint files * add bedrock test * core[release]: Release 0.2.6 (langchain-ai#5670) * docs[patch]: Adds heading keywords for search (langchain-ai#5678) * Adds heading keywords for search * Format * Change neo4j verify connectivity method (langchain-ai#5679) * partners[patch]: Release partner & community packages (langchain-ai#5688) * openai[patch]: Release 0.1.2 (langchain-ai#5689) * community[patch]: Deprecate Google PaLM integrations (langchain-ai#5690) * community[patch]: Deprecate Google PaLM integrations * chore: lint files * docs[patch]: Adds PDF ingestion and QA tutorial (langchain-ai#5692) * Adds PDF ingestion and QA tutorial * Typo * typo * Bump dependency range for @langchain/textsplitters (langchain-ai#5693) * textsplitters[patch]: Release 0.0.3 (langchain-ai#5695) * docs[patch]: Add deprecation warnings to Google PaLM docs (langchain-ai#5696) * community[patch]: Loosen conflicting peer dep (langchain-ai#5694) * Loosen conflicting peer dep * Bound better * Deps * Bump * community[patch]: Release 0.2.8 (langchain-ai#5697) * docs: remove duplicated heading section in rag.ipynb (langchain-ai#5674) Co-authored-by: Brace Sproul <[email protected]> * community[patch]: anthropic add tool call support new tools api (langchain-ai#5640) * 5639 community/bedrock anthropic add tool call support new tools api * Re-use most of parsing logic from @langchain/anthropic directly; export it there * Fix build errors * Updated docs, BedrockChat partial support for Tool Calling, Anthropic models * Small updates * Format * Revert Anthropic * Refactor to remove dep * Delegate streaming tool calls to invoke for now * Fix * Bump core version --------- Co-authored-by: jacoblee93 <[email protected]> * community[minor]: Fixed ChatWebLLM reload function and updated model name in example (langchain-ai#5671) * [fix] Fix parameter order of reload and update model name in example * Update example --------- Co-authored-by: jacoblee93 <[email protected]> * community[patch]: Release 0.2.9 (langchain-ai#5698) * docs[patch]: Add crosslinks, add LangGraph and LangSmith sections (langchain-ai#5702) * Add crosslinks, add LangGraph and LangSmith sections * Fix anchors * docs: add azure dynamic sessions documentation * Fix build and make it CJS compatible * Update error message * Add sessions to platform docs * Formatting * Add env var * Add package to examples * Fix agent example * Add readme * Fix bad merge * Add missing package * Fix formatting * Fix langchain/core dependency --------- Signed-off-by: Oleg Ivaniv <[email protected]> Co-authored-by: Erick Friis <[email protected]> Co-authored-by: Brace Sproul <[email protected]> Co-authored-by: Bagatur <[email protected]> Co-authored-by: Jacob Lee <[email protected]> Co-authored-by: Sarangan Rajamanickam <[email protected]> Co-authored-by: theItalianDev <[email protected]> Co-authored-by: Tat Dat Duong <[email protected]> Co-authored-by: yoogle <[email protected]> Co-authored-by: sugarforever <[email protected]> Co-authored-by: Anej Gorkič <[email protected]> Co-authored-by: m-hamaro <[email protected]> Co-authored-by: Mish Ushakov <[email protected]> Co-authored-by: Mish Ushakov <[email protected]> Co-authored-by: Luis Otavio Martins <[email protected]> Co-authored-by: WilliamEspegren <[email protected]> Co-authored-by: Jamsheed Mistri <[email protected]> Co-authored-by: Nicolas <[email protected]> Co-authored-by: Tomaz Bratanic <[email protected]> Co-authored-by: oleg <[email protected]> Co-authored-by: Fahreddin Özcan <[email protected]> Co-authored-by: santree <[email protected]> Co-authored-by: jeasonnow <[email protected]> Co-authored-by: Crazy Urus <[email protected]> Co-authored-by: paulpaliychuk <[email protected]> Co-authored-by: Anush <[email protected]> Co-authored-by: Cahid Arda Öz <[email protected]> Co-authored-by: George Herbert <[email protected]> Co-authored-by: Adam Cowley <[email protected]> Co-authored-by: Eugene Yurtsev <[email protected]> Co-authored-by: Mohamed Belhadj <[email protected]> Co-authored-by: Michael Watts <[email protected]> Co-authored-by: William FH <[email protected]> Co-authored-by: LukeCali-949 <[email protected]> Co-authored-by: Allen Firstenberg <[email protected]> Co-authored-by: Miguel Fernández <[email protected]> Co-authored-by: Darren Govoni <[email protected]> Co-authored-by: Danish J <[email protected]> Co-authored-by: Oguz Vuruskaner <[email protected]> Co-authored-by: TeCHiScy <[email protected]> Co-authored-by: AumeshMisra <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Jelf <[email protected]> Co-authored-by: Andrey <[email protected]> Co-authored-by: Oceansdeep7 <[email protected]> Co-authored-by: Sarfudheen JAINULABUDHEEN <[email protected]> Co-authored-by: Guillermo C. Martínez <[email protected]> Co-authored-by: Guille <[email protected]> Co-authored-by: Sharath <[email protected]> Co-authored-by: Quinn <[email protected]> Co-authored-by: Evan <[email protected]> Co-authored-by: LordMsz <[email protected]> Co-authored-by: Kai Wang <[email protected]>
Checked other resources
Example Code
Error Message and Stack Trace (if applicable)
Description
I am trying to integrate ChatVertexAI with withStructuredOutput.
This works fine without withStructuredOutput.
System Info
The text was updated successfully, but these errors were encountered: