Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partners[minor]: Add standard chat model tests to partner packages #5660

Merged
merged 30 commits into from
Jun 4, 2024
Merged
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
4820bf9
partners[minor]: Add standard chat model tests to partner packages
bracesproul Jun 4, 2024
400b04b
google genai
bracesproul Jun 4, 2024
efb7f3f
yarn
bracesproul Jun 4, 2024
1510d62
groq
bracesproul Jun 4, 2024
74d2fe9
groq nit and mistral
bracesproul Jun 4, 2024
f694aad
add to azure in chat openai
bracesproul Jun 4, 2024
f9b4d85
chore: lint files
bracesproul Jun 4, 2024
5a8767e
drop azure openai pkg
bracesproul Jun 4, 2024
bdb4d55
add generic constructor args to standard tests pkg
bracesproul Jun 4, 2024
de2259c
implement cloudflare standard tests
bracesproul Jun 4, 2024
8991340
implement cohere standard tests
bracesproul Jun 4, 2024
183816c
google genai package standard tests
bracesproul Jun 4, 2024
b96008e
groq
bracesproul Jun 4, 2024
3f4a3b3
allow for custom function call ids, fix mistral
bracesproul Jun 4, 2024
ded3663
azure tests
bracesproul Jun 4, 2024
18c748e
chore: lint files
bracesproul Jun 4, 2024
20d16d2
update standard tests gh action to run all pkgs
bracesproul Jun 4, 2024
cc4f4e7
chore: lint files
bracesproul Jun 4, 2024
63de31e
revert workflow file rename
bracesproul Jun 4, 2024
c323fab
fix workflow job naming issue
bracesproul Jun 4, 2024
50971e2
add anthropic, fix api keys
bracesproul Jun 4, 2024
01573ab
cache deps?
bracesproul Jun 4, 2024
f978653
fix build
bracesproul Jun 4, 2024
ca50201
Merge branch 'main' into brace/standard-tests-partners
bracesproul Jun 4, 2024
74da454
update standard tests
bracesproul Jun 4, 2024
ee55ad9
cr
bracesproul Jun 4, 2024
31aa8f8
fix
bracesproul Jun 4, 2024
d607d2b
remove dep on job which doesnt exist
bracesproul Jun 4, 2024
4c0789b
cr
bracesproul Jun 4, 2024
e66d30e
cr
bracesproul Jun 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: lint files
bracesproul committed Jun 4, 2024
commit f9b4d85ae86691620d52211b8a7cd6b1ed947834
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { AzureChatOpenAI, ChatOpenAICallOptions } from "../chat_models.js";

class AzureChatOpenAIStandardIntegrationTests extends ChatModelIntegrationTests<
ChatOpenAICallOptions,
ChatOpenAICallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { AzureChatOpenAI, ChatOpenAICallOptions } from "../chat_models.js";

class AzureChatOpenAIStandardUnitTests extends ChatModelUnitTests<
ChatOpenAICallOptions,
ChatOpenAICallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -2,10 +2,13 @@
import { test, expect } from "@jest/globals";
import { ChatModelIntegrationTests } from "@langchain/standard-tests";
import { AIMessageChunk } from "@langchain/core/messages";
import { ChatCloudflareWorkersAI, ChatCloudflareWorkersAICallOptions } from "../chat_models.js";
import {
ChatCloudflareWorkersAI,
ChatCloudflareWorkersAICallOptions,
} from "../chat_models.js";

class ChatCloudflareWorkersAIStandardIntegrationTests extends ChatModelIntegrationTests<
ChatCloudflareWorkersAICallOptions,
ChatCloudflareWorkersAICallOptions,
AIMessageChunk
> {
constructor() {
11 changes: 8 additions & 3 deletions libs/langchain-cloudflare/src/tests/chat_models.standard.test.ts
Original file line number Diff line number Diff line change
@@ -2,10 +2,13 @@
import { test, expect } from "@jest/globals";
import { ChatModelUnitTests } from "@langchain/standard-tests";
import { AIMessageChunk } from "@langchain/core/messages";
import { ChatCloudflareWorkersAI, ChatCloudflareWorkersAICallOptions } from "../chat_models.js";
import {
ChatCloudflareWorkersAI,
ChatCloudflareWorkersAICallOptions,
} from "../chat_models.js";

class ChatCloudflareWorkersAIStandardUnitTests extends ChatModelUnitTests<
ChatCloudflareWorkersAICallOptions,
ChatCloudflareWorkersAICallOptions,
AIMessageChunk
> {
constructor() {
@@ -18,7 +21,9 @@ ChatCloudflareWorkersAICallOptions,
}

testChatModelInitApiKey() {
console.warn("Skipping testChatModelInitApiKey for ChatCloudflareWorkersAI. Multiple API keys are required.");
console.warn(
"Skipping testChatModelInitApiKey for ChatCloudflareWorkersAI. Multiple API keys are required."
);
}
}

Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatCohere, CohereChatCallOptions } from "../chat_models.js";

class ChatCohereStandardIntegrationTests extends ChatModelIntegrationTests<
CohereChatCallOptions,
CohereChatCallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatCohere, CohereChatCallOptions } from "../chat_models.js";

class ChatCohereStandardUnitTests extends ChatModelUnitTests<
CohereChatCallOptions,
CohereChatCallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -2,10 +2,13 @@
import { test, expect } from "@jest/globals";
import { ChatModelIntegrationTests } from "@langchain/standard-tests";
import { AIMessageChunk } from "@langchain/core/messages";
import { ChatGoogleGenerativeAI, GoogleGenerativeAIChatCallOptions } from "../chat_models.js";
import {
ChatGoogleGenerativeAI,
GoogleGenerativeAIChatCallOptions,
} from "../chat_models.js";

class ChatGoogleGenerativeAIStandardIntegrationTests extends ChatModelIntegrationTests<
GoogleGenerativeAIChatCallOptions,
GoogleGenerativeAIChatCallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -2,10 +2,13 @@
import { test, expect } from "@jest/globals";
import { ChatModelUnitTests } from "@langchain/standard-tests";
import { AIMessageChunk } from "@langchain/core/messages";
import { ChatGoogleGenerativeAI, GoogleGenerativeAIChatCallOptions } from "../chat_models.js";
import {
ChatGoogleGenerativeAI,
GoogleGenerativeAIChatCallOptions,
} from "../chat_models.js";

class ChatGoogleGenerativeAIStandardUnitTests extends ChatModelUnitTests<
GoogleGenerativeAIChatCallOptions,
GoogleGenerativeAIChatCallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatGroq, ChatGroqCallOptions } from "../chat_models.js";

class ChatGroqStandardIntegrationTests extends ChatModelIntegrationTests<
ChatGroqCallOptions,
ChatGroqCallOptions,
AIMessageChunk
> {
constructor() {
2 changes: 1 addition & 1 deletion libs/langchain-groq/src/tests/chat_models.standard.test.ts
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatGroq, ChatGroqCallOptions } from "../chat_models.js";

class ChatGroqStandardUnitTests extends ChatModelUnitTests<
ChatGroqCallOptions,
ChatGroqCallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatMistralAI, MistralAICallOptions } from "../chat_models.js";

class ChatMistralAIStandardIntegrationTests extends ChatModelIntegrationTests<
MistralAICallOptions,
MistralAICallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { AIMessageChunk } from "@langchain/core/messages";
import { ChatMistralAI, MistralAICallOptions } from "../chat_models.js";

class ChatMistralAIStandardUnitTests extends ChatModelUnitTests<
MistralAICallOptions,
MistralAICallOptions,
AIMessageChunk
> {
constructor() {
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ import { AzureChatOpenAI } from "../../azure/chat_models.js";
import { ChatOpenAICallOptions } from "../../chat_models.js";

class AzureChatOpenAIStandardIntegrationTests extends ChatModelIntegrationTests<
ChatOpenAICallOptions,
ChatOpenAICallOptions,
AIMessageChunk
> {
constructor() {
@@ -15,7 +15,7 @@ ChatOpenAICallOptions,
chatModelHasToolCalling: true,
chatModelHasStructuredOutput: true,
constructorArgs: {
model: "gpt-3.5-turbo"
model: "gpt-3.5-turbo",
},
});
}
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ import { AzureChatOpenAI } from "../../azure/chat_models.js";
import { ChatOpenAICallOptions } from "../../chat_models.js";

class AzureChatOpenAIStandardUnitTests extends ChatModelUnitTests<
ChatOpenAICallOptions,
ChatOpenAICallOptions,
AIMessageChunk
> {
constructor() {
@@ -19,7 +19,9 @@ ChatOpenAICallOptions,
}

testChatModelInitApiKey() {
console.warn("AzureChatOpenAI does not require a single API key. Skipping...");
console.warn(
"AzureChatOpenAI does not require a single API key. Skipping..."
);
}
}


Unchanged files with check annotations Beta

/**
* Install a internal template to a given `root` directory.
*/
export async function installTemplate({ appName, root }: any) {

Check warning on line 10 in libs/create-langchain-integration/helpers/templates.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
/**
* Copy the template files to the target directory.
*/
* Update the package.json scripts.
*/
const packageJsonFile = path.join(root, "package.json");
const packageJson: any = JSON.parse(

Check warning on line 28 in libs/create-langchain-integration/helpers/templates.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
await fs.readFile(packageJsonFile, "utf8")
);
packageJson.name = appName;
} from "@langchain/core/language_models/chat_models";
import { BaseMessageChunk } from "@langchain/core/messages";
export type RecordStringAny = Record<string, any>;

Check warning on line 7 in libs/langchain-standard-tests/src/base.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
export type BaseChatModelConstructor<
CallOptions extends BaseChatModelCallOptions = BaseChatModelCallOptions,
OutputMessageType extends BaseMessageChunk = BaseMessageChunk
> = new (...args: any[]) => BaseChatModel<CallOptions, OutputMessageType>;

Check warning on line 12 in libs/langchain-standard-tests/src/base.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
export type BaseChatModelsTestsFields<
CallOptions extends BaseChatModelCallOptions = BaseChatModelCallOptions,
try {
await this.testInvoke();
} catch (e: any) {

Check warning on line 360 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testInvoke failed", e);
}
try {
await this.testStream();
} catch (e: any) {

Check warning on line 367 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testStream failed", e);
}
try {
await this.testBatch();
} catch (e: any) {

Check warning on line 374 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testBatch failed", e);
}
try {
await this.testConversation();
} catch (e: any) {

Check warning on line 381 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testConversation failed", e);
}
try {
await this.testUsageMetadata();
} catch (e: any) {

Check warning on line 388 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testUsageMetadata failed", e);
}
try {
await this.testUsageMetadataStreaming();
} catch (e: any) {

Check warning on line 395 in libs/langchain-standard-tests/src/integration_tests/chat_models.ts

GitHub Actions / Check linting

Unexpected any. Specify a different type
allTestsPassed = false;
console.error("testUsageMetadataStreaming failed", e);
}