Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add LlamaIndex TS support to JS SDK #1178

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

abhishekpatil4
Copy link
Contributor

@abhishekpatil4 abhishekpatil4 commented Jan 10, 2025

  • Added support for LlamaIndex TS framework in JS SDK
  • Created test cases for framework validation

Important

Add LlamaIndex framework support to JS SDK with new class and test cases.

  • Framework Support:
    • Add LlamaIndexToolSet class in llamaindex.ts for LlamaIndex framework support.
    • Import and export LlamaIndexToolSet in index.ts.
  • Testing:
    • Add test cases in llamaindex.spec.ts to validate LlamaIndex framework functionality.
  • Dependencies:
    • Add llamaindex dependency in package.json.

This description was created by Ellipsis for d602522. It will automatically update as commits are pushed.

Copy link

vercel bot commented Jan 10, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
composio ❌ Failed (Inspect) Jan 10, 2025 7:19pm

Comment on lines +51 to +54
parameters: {
type: "object",
properties: schema.parameters.properties || {},
required: schema.parameters.required || [],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The getTools function in llamaindex.ts does not correctly handle the case where schema.parameters is undefined. This can lead to a runtime error when trying to access properties or required.

Comment on lines +66 to +70
owner: "composioHQ",
repo: "composio",
},
entityId: "default",
connectedAccountId: "9442cab3-d54f-4903-976c-ee67ef506c9b",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test Should create custom action to star a repository in llamaindex.spec.ts uses hardcoded values for connectedAccountId and the repository details. This makes the test less robust and prone to failures if these values change.

Comment on lines +36 to +56
await llamaindexToolSet.createAction({
actionName: "starRepositoryCustomAction",
toolName: "github",
description: "This action stars a repository",
inputParams: z.object({
owner: z.string(),
repo: z.string(),
}),
callback: async (
inputParams,
_authCredentials,
executeRequest
): Promise<ActionExecuteResponse> => {
const res = await executeRequest({
endpoint: `/user/starred/${inputParams.owner}/${inputParams.repo}`,
method: "PUT",
parameters: [],
});
return res;
},
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The createAction call in the test Should create custom action to star a repository lacks proper error handling. If the action creation fails, the test will not fail and the subsequent getTools and executeAction calls might behave unexpectedly.

Copy link

github-actions bot commented Jan 10, 2025

This comment was generated by github-actions[bot]!

JS SDK Coverage Report

📊 Coverage report for JS SDK can be found at the following URL:
https://pub-92e668239ab84bfd80ee07d61e9d2f40.r2.dev/coverage-12715859283/coverage/index.html

📁 Test report folder can be found at the following URL:
https://pub-92e668239ab84bfd80ee07d61e9d2f40.r2.dev/html-report-12715859283/html-report/report.html

Copy link

🚀 Code review completed!

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to b7a8481 in 58 seconds

More details
  • Looked at 192 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. js/src/frameworks/llamaindex.spec.ts:16
  • Draft comment:
    The test name 'getools' is not descriptive. Consider renaming it to something more meaningful, like 'should return tools array'.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The test name 'getools' is not descriptive and seems to be a typo. It should be more descriptive to reflect the test's purpose.
2. js/src/frameworks/llamaindex.spec.ts:22
  • Draft comment:
    The test case 'check if tools are coming' is redundant with 'check if getTools, actions are coming'. Consider removing one to avoid duplication.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The test case 'check if tools are coming' and 'check if getTools, actions are coming' are redundant as they perform the same check. One of them should be removed to avoid duplication.
3. js/src/frameworks/llamaindex.spec.ts:63
  • Draft comment:
    Typo in variable name 'actionOuput'. It should be 'actionOutput'.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The variable name 'actionOuput' is misspelled. It should be 'actionOutput'.
4. js/src/frameworks/llamaindex.ts:36
  • Draft comment:
    Avoid using 'any' type for 'schema'. Use a more specific type to ensure type safety.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of 'any' type in TypeScript should be avoided as it defeats the purpose of type safety. A more specific type should be used for 'schema'.

Workflow ID: wflow_1ImRrec93dw0lj3s


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@abhishekpatil4 abhishekpatil4 changed the title Feat/llamaindex-ts Fix: removed some methods Jan 10, 2025
import { ActionExecuteResponse } from "../sdk/models/actions";
import { LlamaIndexToolSet } from "./llamaindex";

describe("Apps class tests", () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test suite description Apps class tests is misleading as it's testing the LlamaIndexToolSet class. Consider renaming it to LlamaIndexToolSet tests for better clarity.

});
}

private _wrapTool(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _wrapTool method accepts schema: any as a parameter type. Consider creating a proper interface for better type safety:

interface ToolSchema {
  name: string;
  description: string;
  parameters: {
    properties?: Record<string, unknown>;
    required?: string[];
  };
}

entityId: Optional<string> = null
): FunctionTool<Record<string, unknown>, JSONValue | Promise<JSONValue>> {
return FunctionTool.from(
async (params: Record<string, unknown>) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding error handling for the JSON parsing operation:

try {
  return JSON.parse(JSON.stringify(result)) as JSONValue;
} catch (error) {
  throw new Error(`Failed to parse tool result: ${error.message}`);
}

This would provide better error messages and make debugging easier.

@shreysingla11
Copy link
Collaborator

Code Review Summary

Overall Assessment

The implementation of LlamaIndex TypeScript framework support is well-structured and follows the established patterns in the codebase. The code is generally clean and functional, with good test coverage.

Positive Aspects

  • Good integration with the base ComposioToolSet class
  • Proper implementation of LlamaIndex's FunctionTool interface
  • Clean removal of unused OpenAI-specific code
  • Good test coverage for basic functionality

Suggestions for Improvement

  1. Type Safety

    • Replace any type in _wrapTool with a proper interface
    • Add proper error handling for JSON operations
  2. Testing

    • Rename test suite to accurately reflect what's being tested
    • Consider adding error case tests
    • Add tests for edge cases with empty or invalid filters
  3. Documentation

    • Add JSDoc comments for public methods
    • Consider adding examples in comments for common use cases

Code Quality Rating: 8/10

The code is well-implemented and follows good practices, with minor improvements suggested for type safety and documentation.

@plxity
Copy link
Contributor

plxity commented Jan 10, 2025

@abhishekpatil4 Couple of actions are failing, please fix them

Comment on lines +57 to +61
parameters: {
type: "object",
properties: schema.parameters.properties || {},
required: schema.parameters.required || [],
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The getTools function in llamaindex.ts does not correctly handle the case when schema.parameters is null or undefined, which could lead to a runtime error.

Comment on lines +16 to +21
it("getools", async () => {
const tools = await llamaindexToolSet.getTools({
apps: ["github"],
});
expect(tools).toBeInstanceOf(Array);
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test getools in llamaindex.spec.ts doesn't assert anything meaningful. It only checks the type of the return, not the content.

Comment on lines +35 to +74
it("Should create custom action to star a repository", async () => {
await llamaindexToolSet.createAction({
actionName: "starRepositoryCustomAction",
toolName: "github",
description: "This action stars a repository",
inputParams: z.object({
owner: z.string(),
repo: z.string(),
}),
callback: async (
inputParams,
_authCredentials,
executeRequest
): Promise<ActionExecuteResponse> => {
const res = await executeRequest({
endpoint: `/user/starred/${inputParams.owner}/${inputParams.repo}`,
method: "PUT",
parameters: [],
});
return res;
},
});

const tools = await llamaindexToolSet.getTools({
actions: ["starRepositoryCustomAction"],
});

await expect(tools.length).toBe(1);
const actionOuput = await llamaindexToolSet.executeAction({
action: "starRepositoryCustomAction",
params: {
owner: "composioHQ",
repo: "composio",
},
entityId: "default",
connectedAccountId: "9442cab3-d54f-4903-976c-ee67ef506c9b",
});

expect(actionOuput).toHaveProperty("successful", true);
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test suite in llamaindex.spec.ts is missing a test to verify that the createAction method adds a new action correctly. The current test only checks if the number of tools increased but not the properties of the new tool/action.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 769477f in 59 seconds

More details
  • Looked at 26 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_bml1QyLngGW09OCX


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

type ToolSchema = {
name: string;
description: string;
parameters: Record<string, unknown>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameters property in ToolSchema is defined as Record<string, unknown>, but it is accessed as if it has properties and required fields in _wrapTool. Consider defining parameters with a more specific type that includes properties and required fields to avoid potential runtime errors.

Comment on lines +57 to +61
parameters: {
type: "object",
properties: schema.parameters.properties || {},
required: schema.parameters.required || [],
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The getTools function in llamaindex.ts does not correctly handle the case where parameters is undefined in the tool schema. This could lead to a runtime error when trying to access schema.parameters.properties or schema.parameters.required.

Comment on lines +16 to +21
it("getools", async () => {
const tools = await llamaindexToolSet.getTools({
apps: ["github"],
});
expect(tools).toBeInstanceOf(Array);
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test getools in llamaindex.spec.ts doesn't assert anything meaningful. It only checks if the returned value is an array, but not its contents. This test should verify the expected tools are returned.

Comment on lines +22 to +34
it("check if tools are coming", async () => {
const tools = await llamaindexToolSet.getTools({
actions: ["GITHUB_GITHUB_API_ROOT"],
});
expect(tools.length).toBe(1);
});
it("check if getTools, actions are coming", async () => {
const tools = await llamaindexToolSet.getTools({
actions: ["GITHUB_GITHUB_API_ROOT"],
});

expect(tools.length).toBe(1);
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant tests in llamaindex.spec.ts. The tests "check if tools are coming" and "check if getTools, actions are coming" are effectively identical. Consolidate these into a single test.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 57535ff in 15 seconds

More details
  • Looked at 19 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. js/src/frameworks/llamaindex.ts:13
  • Draft comment:
    The 'parameters' property in 'ToolSchema' is defined as 'Record<string, unknown>', but is accessed as if it has 'properties' and 'required' fields in '_wrapTool'. Ensure 'parameters' is correctly typed to avoid runtime errors.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_rovD7CEFRmRi9Sw4


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@abhishekpatil4 abhishekpatil4 changed the title Fix: removed some methods feat: add LlamaIndex TS support to JS SDK Jan 10, 2025
@@ -40,6 +40,7 @@
"@langchain/openai": "^0.2.5",
"ai": "^3.2.22",
"langchain": "^0.2.11",
"llamaindex": "^0.8.31",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plus signs in the added line are redundant and should be removed.

📝 Committable Code Suggestion

‼️ Ensure you review the code suggestion before committing it to the branch. Make sure it replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
"llamaindex": "^0.8.31",
"llamaindex": "^0.8.31",

Comment on lines +1 to +75
import { beforeAll, describe, expect, it } from "@jest/globals";
import { z } from "zod";
import { getTestConfig } from "../../config/getTestConfig";
import { ActionExecuteResponse } from "../sdk/models/actions";
import { LlamaIndexToolSet } from "./llamaindex";

describe("Apps class tests", () => {
let llamaindexToolSet: LlamaIndexToolSet;
beforeAll(() => {
llamaindexToolSet = new LlamaIndexToolSet({
apiKey: getTestConfig().COMPOSIO_API_KEY,
baseUrl: getTestConfig().BACKEND_HERMES_URL,
});
});

it("getools", async () => {
const tools = await llamaindexToolSet.getTools({
apps: ["github"],
});
expect(tools).toBeInstanceOf(Array);
});
it("check if tools are coming", async () => {
const tools = await llamaindexToolSet.getTools({
actions: ["GITHUB_GITHUB_API_ROOT"],
});
expect(tools.length).toBe(1);
});
it("check if getTools, actions are coming", async () => {
const tools = await llamaindexToolSet.getTools({
actions: ["GITHUB_GITHUB_API_ROOT"],
});

expect(tools.length).toBe(1);
});
it("Should create custom action to star a repository", async () => {
await llamaindexToolSet.createAction({
actionName: "starRepositoryCustomAction",
toolName: "github",
description: "This action stars a repository",
inputParams: z.object({
owner: z.string(),
repo: z.string(),
}),
callback: async (
inputParams,
_authCredentials,
executeRequest
): Promise<ActionExecuteResponse> => {
const res = await executeRequest({
endpoint: `/user/starred/${inputParams.owner}/${inputParams.repo}`,
method: "PUT",
parameters: [],
});
return res;
},
});

const tools = await llamaindexToolSet.getTools({
actions: ["starRepositoryCustomAction"],
});

await expect(tools.length).toBe(1);
const actionOuput = await llamaindexToolSet.executeAction({
action: "starRepositoryCustomAction",
params: {
owner: "composioHQ",
repo: "composio",
},
entityId: "default",
connectedAccountId: "9442cab3-d54f-4903-976c-ee67ef506c9b",
});

expect(actionOuput).toHaveProperty("successful", true);
});
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant plus signs should be removed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on d602522 in 8 seconds

More details
  • Looked at 12 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. js/package.json:43
  • Draft comment:
    'llamaindex' is listed in both 'dependencies' and 'peerDependencies'. It should be in one, not both, to avoid redundancy and potential version conflicts.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_o8WpdIgjNCaS3Ek6


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Comment on lines +1 to +91
import { FunctionTool, type JSONValue } from "llamaindex";
import { z } from "zod";
import { ComposioToolSet as BaseComposioToolSet } from "../sdk/base.toolset";
import { COMPOSIO_BASE_URL } from "../sdk/client/core/OpenAPI";
import { TELEMETRY_LOGGER } from "../sdk/utils/telemetry";
import { TELEMETRY_EVENTS } from "../sdk/utils/telemetry/events";
import { ZToolSchemaFilter } from "../types/base_toolset";
import { Optional, Sequence } from "../types/util";

type ToolSchema = {
name: string;
description: string;
parameters: Record<string, unknown>;
};

export class LlamaIndexToolSet extends BaseComposioToolSet {
/**
* Composio toolset for LlamaIndex framework.
*
*/
static FRAMEWORK_NAME = "llamaindex";
static DEFAULT_ENTITY_ID = "default";
fileName: string = "js/src/frameworks/llamaindex.ts";

constructor(
config: {
apiKey?: Optional<string>;
baseUrl?: Optional<string>;
entityId?: string;
runtime?: string;
} = {}
) {
super({
apiKey: config.apiKey || null,
baseUrl: config.baseUrl || COMPOSIO_BASE_URL,
runtime: config?.runtime || null,
entityId: config.entityId || LlamaIndexToolSet.DEFAULT_ENTITY_ID,
});
}

private _wrapTool(
schema: ToolSchema,
entityId: Optional<string> = null
): FunctionTool<Record<string, unknown>, JSONValue | Promise<JSONValue>> {
return FunctionTool.from(
async (params: Record<string, unknown>) => {
const result = await this.executeAction({
action: schema.name,
params,
entityId: entityId || this.entityId,
});
return JSON.parse(JSON.stringify(result)) as JSONValue;
},
{
name: schema.name,
description: schema.description,
parameters: {
type: "object",
properties: schema.parameters.properties || {},
required: schema.parameters.required || [],
},
}
);
}

async getTools(
filters: z.infer<typeof ZToolSchemaFilter> = {},
entityId: Optional<string> = null
): Promise<
Sequence<
FunctionTool<Record<string, unknown>, JSONValue | Promise<JSONValue>>
>
> {
TELEMETRY_LOGGER.manualTelemetry(TELEMETRY_EVENTS.SDK_METHOD_INVOKED, {
method: "getTools",
file: this.fileName,
params: { filters, entityId },
});

const tools = await this.getToolsSchema(filters, entityId);
return tools.map((tool) => {
const wrappedTool = this._wrapTool(tool, entityId || this.entityId);
Object.assign(wrappedTool, {
name: tool.name,
description: tool.description,
parameters: tool.parameters,
});
return wrappedTool;
});
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant plus signs should be removed.

@himanshu-dixit
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants