Skip to content

Latest commit

Β 

History

History
7000 lines (4859 loc) Β· 84.1 KB

reference.md

File metadata and controls

7000 lines (4859 loc) Β· 84.1 KB

Reference

Prompts

client.prompts.log({ ...params }) -> Humanloop.CreatePromptLogResponse

πŸ“ Description

Log to a Prompt.

You can use query parameters version_id, or environment, to target an existing version of the Prompt. Otherwise, the default deployed version will be chosen.

Instead of targeting an existing version explicitly, you can instead pass in Prompt details in the request body. In this case, we will check if the details correspond to an existing version of the Prompt. If they do not, we will create a new version. This is helpful in the case where you are storing or deriving your Prompt details in code.

πŸ”Œ Usage

await client.prompts.log({
    path: "persona",
    prompt: {
        model: "gpt-4",
        template: [
            {
                role: "system",
                content: "You are {{person}}. Answer questions as this person. Do not break character.",
            },
        ],
    },
    messages: [
        {
            role: "user",
            content: "What really happened at Roswell?",
        },
    ],
    inputs: {
        person: "Trump",
    },
    createdAt: "2024-07-19T00:29:35.178992",
    providerLatency: 6.5931549072265625,
    outputMessage: {
        content:
            "Well, you know, there is so much secrecy involved in government, folks, it's unbelievable. They don't want to tell you everything. They don't tell me everything! But about Roswell, it's a very popular question. I know, I just know, that something very, very peculiar happened there. Was it a weather balloon? Maybe. Was it something extraterrestrial? Could be. I'd love to go down and open up all the classified documents, believe me, I would. But they don't let that happen. The Deep State, folks, the Deep State. They're unbelievable. They want to keep everything a secret. But whatever the truth is, I can tell you this: it's something big, very very big. Tremendous, in fact.",
        role: "assistant",
    },
    promptTokens: 100,
    outputTokens: 220,
    promptCost: 0.00001,
    outputCost: 0.0002,
    finishReason: "stop",
});

βš™οΈ Parameters

request: Humanloop.PromptLogRequest

requestOptions: Prompts.RequestOptions

client.prompts.updateLog(id, logId, { ...params }) -> Humanloop.LogResponse

πŸ“ Description

Update a Log.

Update the details of a Log with the given ID.

πŸ”Œ Usage

await client.prompts.updateLog("id", "log_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

logId: string β€” Unique identifier for the Log.

request: Humanloop.PromptLogUpdateRequest

requestOptions: Prompts.RequestOptions

client.prompts.callStream({ ...params }) -> core.Stream

πŸ“ Description

Call a Prompt.

Calling a Prompt calls the model provider before logging the request, responses and metadata to Humanloop.

You can use query parameters version_id, or environment, to target an existing version of the Prompt. Otherwise the default deployed version will be chosen.

Instead of targeting an existing version explicitly, you can instead pass in Prompt details in the request body. In this case, we will check if the details correspond to an existing version of the Prompt. If they do not, we will create a new version. This is helpful in the case where you are storing or deriving your Prompt details in code.

πŸ”Œ Usage

await client.prompts.callStream({
    versionId: "string",
    environment: "string",
    path: "string",
    id: "string",
    messages: [
        {
            content: "string",
            name: "string",
            toolCallId: "string",
            role: "user",
            toolCalls: [
                {
                    id: "string",
                    type: "function",
                    function: {
                        name: "string",
                        arguments: undefined,
                    },
                },
            ],
        },
    ],
    toolChoice: "none",
    prompt: {
        model: "string",
        endpoint: undefined,
        template: undefined,
        provider: undefined,
        maxTokens: undefined,
        temperature: undefined,
        topP: undefined,
        stop: undefined,
        presencePenalty: undefined,
        frequencyPenalty: undefined,
        other: undefined,
        seed: undefined,
        responseFormat: undefined,
        tools: undefined,
        linkedTools: undefined,
        attributes: undefined,
    },
    inputs: {
        string: {
            key: "value",
        },
    },
    source: "string",
    metadata: {
        string: {
            key: "value",
        },
    },
    startTime: "2024-01-15T09:30:00Z",
    endTime: "2024-01-15T09:30:00Z",
    sourceDatapointId: "string",
    traceParentId: "string",
    user: "string",
    promptsCallStreamRequestEnvironment: "string",
    save: true,
    logId: "string",
    providerApiKeys: {
        openai: "string",
        ai21: "string",
        mock: "string",
        anthropic: "string",
        bedrock: "string",
        cohere: "string",
        openaiAzure: "string",
        openaiAzureEndpoint: "string",
    },
    numSamples: 1,
    returnInputs: true,
    logprobs: 1,
    suffix: "string",
});

βš™οΈ Parameters

request: Humanloop.PromptsCallStreamRequest

requestOptions: Prompts.RequestOptions

client.prompts.call({ ...params }) -> Humanloop.PromptCallResponse

πŸ“ Description

Call a Prompt.

Calling a Prompt calls the model provider before logging the request, responses and metadata to Humanloop.

You can use query parameters version_id, or environment, to target an existing version of the Prompt. Otherwise the default deployed version will be chosen.

Instead of targeting an existing version explicitly, you can instead pass in Prompt details in the request body. In this case, we will check if the details correspond to an existing version of the Prompt. If they do not, we will create a new version. This is helpful in the case where you are storing or deriving your Prompt details in code.

πŸ”Œ Usage

await client.prompts.call({
    path: "persona",
    prompt: {
        model: "gpt-4",
        template: [
            {
                role: "system",
                content: "You are stockbot. Return latest prices.",
            },
        ],
        tools: [
            {
                name: "get_stock_price",
                description: "Get current stock price",
                parameters: {
                    type: "object",
                    properties: {
                        ticker_symbol: {
                            type: "string",
                            name: "Ticker Symbol",
                            description: "Ticker symbol of the stock",
                        },
                    },
                    required: [],
                },
            },
        ],
    },
    messages: [
        {
            role: "user",
            content: "latest apple",
        },
    ],
});

βš™οΈ Parameters

request: Humanloop.PromptsCallRequest

requestOptions: Prompts.RequestOptions

client.prompts.list({ ...params }) -> core.Page

πŸ“ Description

Get a list of all Prompts.

πŸ”Œ Usage

await client.prompts.list({
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListPromptsGetRequest

requestOptions: Prompts.RequestOptions

client.prompts.upsert({ ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Create a Prompt or update it with a new version if it already exists.

Prompts are identified by the ID or their path. The parameters (i.e. the prompt template, temperature, model etc.) determine the versions of the Prompt.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

πŸ”Œ Usage

await client.prompts.upsert({
    path: "Personal Projects/Coding Assistant",
    model: "gpt-4o",
    endpoint: "chat",
    template: [
        {
            content: "You are a helpful coding assistant specialising in {{language}}",
            role: "system",
        },
    ],
    provider: "openai",
    maxTokens: -1,
    temperature: 0.7,
    commitMessage: "Initial commit",
});

βš™οΈ Parameters

request: Humanloop.PromptRequest

requestOptions: Prompts.RequestOptions

client.prompts.get(id, { ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Retrieve the Prompt with the given ID.

By default, the deployed version of the Prompt is returned. Use the query parameters version_id or environment to target a specific version of the Prompt.

πŸ”Œ Usage

await client.prompts.get("pr_30gco7dx6JDq4200GVOHa");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

request: Humanloop.GetPromptsIdGetRequest

requestOptions: Prompts.RequestOptions

client.prompts.delete(id) -> void

πŸ“ Description

Delete the Prompt with the given ID.

πŸ”Œ Usage

await client.prompts.delete("pr_30gco7dx6JDq4200GVOHa");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

requestOptions: Prompts.RequestOptions

client.prompts.move(id, { ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Move the Prompt to a different path or change the name.

πŸ”Œ Usage

await client.prompts.move("pr_30gco7dx6JDq4200GVOHa", {
    path: "new directory/new name",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

request: Humanloop.UpdatePromptRequest

requestOptions: Prompts.RequestOptions

client.prompts.listVersions(id, { ...params }) -> Humanloop.ListPrompts

πŸ“ Description

Get a list of all the versions of a Prompt.

πŸ”Œ Usage

await client.prompts.listVersions("pr_30gco7dx6JDq4200GVOHa", {
    status: "committed",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

request: Humanloop.ListVersionsPromptsIdVersionsGetRequest

requestOptions: Prompts.RequestOptions

client.prompts.commit(id, versionId, { ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Commit a version of the Prompt with a commit message.

If the version is already committed, an exception will be raised.

πŸ”Œ Usage

await client.prompts.commit("pr_30gco7dx6JDq4200GVOHa", "prv_F34aba5f3asp0", {
    commitMessage: "Reiterated point about not discussing sentience",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

versionId: string β€” Unique identifier for the specific version of the Prompt.

request: Humanloop.CommitRequest

requestOptions: Prompts.RequestOptions

client.prompts.deletePromptVersion(id, versionId) -> void

πŸ“ Description

Delete a version of the Prompt.

πŸ”Œ Usage

await client.prompts.deletePromptVersion("id", "version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

versionId: string β€” Unique identifier for the specific version of the Prompt.

requestOptions: Prompts.RequestOptions

client.prompts.setDeployment(id, environmentId, { ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Deploy Prompt to an Environment.

Set the deployed version for the specified Environment. This Prompt will be used for calls made to the Prompt in this Environment.

πŸ”Œ Usage

await client.prompts.setDeployment("id", "environment_id", {
    versionId: "version_id",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

environmentId: string β€” Unique identifier for the Environment to deploy the Version to.

request: Humanloop.SetDeploymentPromptsIdEnvironmentsEnvironmentIdPostRequest

requestOptions: Prompts.RequestOptions

client.prompts.removeDeployment(id, environmentId) -> void

πŸ“ Description

Remove deployed Prompt from the Environment.

Remove the deployed version for the specified Environment. This Prompt will no longer be used for calls made to the Prompt in this Environment.

πŸ”Œ Usage

await client.prompts.removeDeployment("id", "environment_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

environmentId: string β€” Unique identifier for the Environment to remove the deployment from.

requestOptions: Prompts.RequestOptions

client.prompts.listEnvironments(id) -> Humanloop.FileEnvironmentResponse[]

πŸ“ Description

List all Environments and their deployed versions for the Prompt.

πŸ”Œ Usage

await client.prompts.listEnvironments("pr_30gco7dx6JDq4200GVOHa");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

requestOptions: Prompts.RequestOptions

client.prompts.updateMonitoring(id, { ...params }) -> Humanloop.PromptResponse

πŸ“ Description

Activate and deactivate Evaluators for monitoring the Prompt.

An activated Evaluator will automatically be run on all new Logs within the Prompt for monitoring purposes.

πŸ”Œ Usage

await client.prompts.updateMonitoring("pr_30gco7dx6JDq4200GVOHa", {
    activate: [
        {
            evaluatorVersionId: "evv_1abc4308abd",
        },
    ],
});

βš™οΈ Parameters

id: string

request: Humanloop.EvaluatorActivationDeactivationRequest

requestOptions: Prompts.RequestOptions

Tools

client.tools.log({ ...params }) -> Humanloop.CreateToolLogResponse

πŸ“ Description

Log to a Tool.

You can use query parameters version_id, or environment, to target an existing version of the Tool. Otherwise the default deployed version will be chosen.

Instead of targeting an existing version explicitly, you can instead pass in Tool details in the request body. In this case, we will check if the details correspond to an existing version of the Tool, if not we will create a new version. This is helpful in the case where you are storing or deriving your Tool details in code.

πŸ”Œ Usage

await client.tools.log({
    path: "math-tool",
    tool: {
        function: {
            name: "multiply",
            description: "Multiply two numbers",
            parameters: {
                type: "object",
                properties: {
                    a: {
                        type: "number",
                    },
                    b: {
                        type: "number",
                    },
                },
                required: ["a", "b"],
            },
        },
    },
    inputs: {
        a: 5,
        b: 7,
    },
    output: "35",
});

βš™οΈ Parameters

request: Humanloop.ToolLogRequest

requestOptions: Tools.RequestOptions

client.tools.update(id, logId, { ...params }) -> Humanloop.LogResponse

πŸ“ Description

Update a Log.

Update the details of a Log with the given ID.

πŸ”Œ Usage

await client.tools.update("id", "log_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

logId: string β€” Unique identifier for the Log.

request: Humanloop.ToolLogUpdateRequest

requestOptions: Tools.RequestOptions

client.tools.list({ ...params }) -> core.Page

πŸ“ Description

Get a list of all Tools.

πŸ”Œ Usage

await client.tools.list({
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListToolsGetRequest

requestOptions: Tools.RequestOptions

client.tools.upsert({ ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Create a Tool or update it with a new version if it already exists.

Tools are identified by the ID or their path. The name, description and parameters determine the versions of the Tool.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

πŸ”Œ Usage

await client.tools.upsert({
    path: "math-tool",
    function: {
        name: "multiply",
        description: "Multiply two numbers",
        parameters: {
            type: "object",
            properties: {
                a: {
                    type: "number",
                },
                b: {
                    type: "number",
                },
            },
            required: ["a", "b"],
        },
    },
    commitMessage: "Initial commit",
});

βš™οΈ Parameters

request: Humanloop.ToolRequest

requestOptions: Tools.RequestOptions

client.tools.get(id, { ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Retrieve the Tool with the given ID.

By default, the deployed version of the Tool is returned. Use the query parameters version_id or environment to target a specific version of the Tool.

πŸ”Œ Usage

await client.tools.get("tl_789ghi");

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

request: Humanloop.GetToolsIdGetRequest

requestOptions: Tools.RequestOptions

client.tools.delete(id) -> void

πŸ“ Description

Delete the Tool with the given ID.

πŸ”Œ Usage

await client.tools.delete("tl_789ghi");

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

requestOptions: Tools.RequestOptions

client.tools.move(id, { ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Move the Tool to a different path or change the name.

πŸ”Œ Usage

await client.tools.move("tl_789ghi", {
    path: "new directory/new name",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

request: Humanloop.UpdateToolRequest

requestOptions: Tools.RequestOptions

client.tools.listVersions(id, { ...params }) -> Humanloop.ListTools

πŸ“ Description

Get a list of all the versions of a Tool.

πŸ”Œ Usage

await client.tools.listVersions("tl_789ghi", {
    status: "committed",
});

βš™οΈ Parameters

id: string β€” Unique identifier for the Tool.

request: Humanloop.ListVersionsToolsIdVersionsGetRequest

requestOptions: Tools.RequestOptions

client.tools.commit(id, versionId, { ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Commit a version of the Tool with a commit message.

If the version is already committed, an exception will be raised.

πŸ”Œ Usage

await client.tools.commit("tl_789ghi", "tv_012jkl", {
    commitMessage: "Initial commit",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

versionId: string β€” Unique identifier for the specific version of the Tool.

request: Humanloop.CommitRequest

requestOptions: Tools.RequestOptions

client.tools.deleteToolVersion(id, versionId) -> void

πŸ“ Description

Delete a version of the Tool.

πŸ”Œ Usage

await client.tools.deleteToolVersion("id", "version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

versionId: string β€” Unique identifier for the specific version of the Tool.

requestOptions: Tools.RequestOptions

client.tools.setDeployment(id, environmentId, { ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Deploy Tool to an Environment.

Set the deployed version for the specified Environment. This Prompt will be used for calls made to the Tool in this Environment.

πŸ”Œ Usage

await client.tools.setDeployment("tl_789ghi", "staging", {
    versionId: "tv_012jkl",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

environmentId: string β€” Unique identifier for the Environment to deploy the Version to.

request: Humanloop.SetDeploymentToolsIdEnvironmentsEnvironmentIdPostRequest

requestOptions: Tools.RequestOptions

client.tools.removeDeployment(id, environmentId) -> void

πŸ“ Description

Remove deployed Tool from the Environment.

Remove the deployed version for the specified Environment. This Tool will no longer be used for calls made to the Tool in this Environment.

πŸ”Œ Usage

await client.tools.removeDeployment("tl_789ghi", "staging");

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

environmentId: string β€” Unique identifier for the Environment to remove the deployment from.

requestOptions: Tools.RequestOptions

client.tools.listEnvironments(id) -> Humanloop.FileEnvironmentResponse[]

πŸ“ Description

List all Environments and their deployed versions for the Tool.

πŸ”Œ Usage

await client.tools.listEnvironments("tl_789ghi");

βš™οΈ Parameters

id: string β€” Unique identifier for Tool.

requestOptions: Tools.RequestOptions

client.tools.updateMonitoring(id, { ...params }) -> Humanloop.ToolResponse

πŸ“ Description

Activate and deactivate Evaluators for monitoring the Tool.

An activated Evaluator will automatically be run on all new Logs within the Tool for monitoring purposes.

πŸ”Œ Usage

await client.tools.updateMonitoring("tl_789ghi", {
    activate: [
        {
            evaluatorVersionId: "evv_1abc4308abd",
        },
    ],
});

βš™οΈ Parameters

id: string

request: Humanloop.EvaluatorActivationDeactivationRequest

requestOptions: Tools.RequestOptions

Datasets

client.datasets.list({ ...params }) -> core.Page

πŸ“ Description

List all Datasets.

πŸ”Œ Usage

await client.datasets.list({
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListDatasetsGetRequest

requestOptions: Datasets.RequestOptions

client.datasets.upsert({ ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Create a Dataset or update it with a new version if it already exists.

Datasets are identified by the ID or their path. The datapoints determine the versions of the Dataset.

By default, the new Dataset version will be set to the list of Datapoints provided in the request. You can also create a new version by adding or removing Datapoints from an existing version by specifying action as add or remove respectively. In this case, you may specify the version_id or environment query parameters to identify the existing version to base the new version on. If neither is provided, the default deployed version will be used.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

Humanloop also deduplicates Datapoints. If you try to add a Datapoint that already exists, it will be ignored. If you intentionally want to add a duplicate Datapoint, you can add a unique identifier to the Datapoint's inputs such as {_dedupe_id: <unique ID>}.

πŸ”Œ Usage

await client.datasets.upsert({
    path: "test-questions",
    datapoints: [
        {
            inputs: {
                question: "What is the capital of France?",
            },
            target: {
                answer: "Paris",
            },
        },
        {
            inputs: {
                question: "Who wrote Hamlet?",
            },
            target: {
                answer: "William Shakespeare",
            },
        },
    ],
    action: "set",
    commitMessage: "Add two new questions and answers",
});

βš™οΈ Parameters

request: Humanloop.DatasetRequest

requestOptions: Datasets.RequestOptions

client.datasets.get(id, { ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Retrieve the Dataset with the given ID.

Unless include_datapoints is set to true, the response will not include the Datapoints. Use the List Datapoints endpoint (GET /{id}/datapoints) to efficiently retrieve Datapoints for a large Dataset.

By default, the deployed version of the Dataset is returned. Use the query parameters version_id or environment to target a specific version of the Dataset.

πŸ”Œ Usage

await client.datasets.get("ds_b0baF1ca7652", {
    versionId: "dsv_6L78pqrdFi2xa",
    includeDatapoints: true,
});

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

request: Humanloop.GetDatasetsIdGetRequest

requestOptions: Datasets.RequestOptions

client.datasets.delete(id) -> void

πŸ“ Description

Delete the Dataset with the given ID.

πŸ”Œ Usage

await client.datasets.delete("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

requestOptions: Datasets.RequestOptions

client.datasets.move(id, { ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Move the Dataset to a different path or change the name.

πŸ”Œ Usage

await client.datasets.move("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

request: Humanloop.UpdateDatasetRequest

requestOptions: Datasets.RequestOptions

client.datasets.listDatapoints(id, { ...params }) -> core.Page

πŸ“ Description

List all Datapoints for the Dataset with the given ID.

πŸ”Œ Usage

await client.datasets.listDatapoints("ds_b0baF1ca7652", {
    size: 1,
});

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

request: Humanloop.ListDatapointsDatasetsIdDatapointsGetRequest

requestOptions: Datasets.RequestOptions

client.datasets.listVersions(id, { ...params }) -> Humanloop.ListDatasets

πŸ“ Description

Get a list of the versions for a Dataset.

πŸ”Œ Usage

await client.datasets.listVersions("ds_b0baF1ca7652", {
    status: "committed",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

request: Humanloop.ListVersionsDatasetsIdVersionsGetRequest

requestOptions: Datasets.RequestOptions

client.datasets.commit(id, versionId, { ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Commit a version of the Dataset with a commit message.

If the version is already committed, an exception will be raised.

πŸ”Œ Usage

await client.datasets.commit("ds_b0baF1ca7652", "dsv_6L78pqrdFi2xa", {
    commitMessage: "initial commit",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

versionId: string β€” Unique identifier for the specific version of the Dataset.

request: Humanloop.CommitRequest

requestOptions: Datasets.RequestOptions

client.datasets.deleteDatasetVersion(id, versionId) -> void

πŸ“ Description

Delete a version of the Dataset.

πŸ”Œ Usage

await client.datasets.deleteDatasetVersion("id", "version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

versionId: string β€” Unique identifier for the specific version of the Dataset.

requestOptions: Datasets.RequestOptions

client.datasets.uploadCsv(file, id, { ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Add Datapoints from a CSV file to a Dataset.

This will create a new committed version of the Dataset with the Datapoints from the CSV file.

If either version_id or environment is provided, the new version will be based on the specified version, with the Datapoints from the CSV file added to the existing Datapoints in the version. If neither version_id nor environment is provided, the new version will be based on the version of the Dataset that is deployed to the default Environment.

πŸ”Œ Usage

await client.datasets.uploadCsv(fs.createReadStream("/path/to/your/file"), "id", {
    commitMessage: "commit_message",
});

βš™οΈ Parameters

file: File | fs.ReadStream | Blob

id: string

request: Humanloop.BodyUploadCsvDatasetsIdDatapointsCsvPost

requestOptions: Datasets.RequestOptions

client.datasets.setDeployment(id, environmentId, { ...params }) -> Humanloop.DatasetResponse

πŸ“ Description

Deploy Dataset to Environment.

Set the deployed version for the specified Environment.

πŸ”Œ Usage

await client.datasets.setDeployment("ds_b0baF1ca7652", "staging", {
    versionId: "dsv_6L78pqrdFi2xa",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

environmentId: string β€” Unique identifier for the Environment to deploy the Version to.

request: Humanloop.SetDeploymentDatasetsIdEnvironmentsEnvironmentIdPostRequest

requestOptions: Datasets.RequestOptions

client.datasets.removeDeployment(id, environmentId) -> void

πŸ“ Description

Remove deployed Dataset from Environment.

Remove the deployed version for the specified Environment.

πŸ”Œ Usage

await client.datasets.removeDeployment("ds_b0baF1ca7652", "staging");

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

environmentId: string β€” Unique identifier for the Environment to remove the deployment from.

requestOptions: Datasets.RequestOptions

client.datasets.listEnvironments(id) -> Humanloop.FileEnvironmentResponse[]

πŸ“ Description

List all Environments and their deployed versions for the Dataset.

πŸ”Œ Usage

await client.datasets.listEnvironments("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Dataset.

requestOptions: Datasets.RequestOptions

Evaluators

client.evaluators.log({ ...params }) -> Humanloop.CreateEvaluatorLogResponse

πŸ“ Description

Submit Evaluator judgment for an existing Log.

Creates a new Log. The evaluated Log will be set as the parent of the created Log.

πŸ”Œ Usage

await client.evaluators.log({
    parentId: "parent_id",
});

βš™οΈ Parameters

request: Humanloop.CreateEvaluatorLogRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.list({ ...params }) -> core.Page

πŸ“ Description

Get a list of all Evaluators.

πŸ”Œ Usage

await client.evaluators.list({
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListEvaluatorsGetRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.upsert({ ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Create an Evaluator or update it with a new version if it already exists.

Evaluators are identified by the ID or their path. The spec provided determines the version of the Evaluator.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

πŸ”Œ Usage

await client.evaluators.upsert({
    path: "Shared Evaluators/Accuracy Evaluator",
    spec: {
        argumentsType: "target_required",
        returnType: "number",
        evaluatorType: "python",
        code: "def evaluate(answer, target):\n    return 0.5",
    },
    commitMessage: "Initial commit",
});

βš™οΈ Parameters

request: Humanloop.EvaluatorRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.get(id, { ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Retrieve the Evaluator with the given ID.

By default, the deployed version of the Evaluator is returned. Use the query parameters version_id or environment to target a specific version of the Evaluator.

πŸ”Œ Usage

await client.evaluators.get("ev_890bcd");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

request: Humanloop.GetEvaluatorsIdGetRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.delete(id) -> void

πŸ“ Description

Delete the Evaluator with the given ID.

πŸ”Œ Usage

await client.evaluators.delete("ev_890bcd");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

requestOptions: Evaluators.RequestOptions

client.evaluators.move(id, { ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Move the Evaluator to a different path or change the name.

πŸ”Œ Usage

await client.evaluators.move("ev_890bcd", {
    path: "new directory/new name",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

request: Humanloop.UpdateEvaluatorRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.listVersions(id, { ...params }) -> Humanloop.ListEvaluators

πŸ“ Description

Get a list of all the versions of an Evaluator.

πŸ”Œ Usage

await client.evaluators.listVersions("ev_890bcd");

βš™οΈ Parameters

id: string β€” Unique identifier for the Evaluator.

request: Humanloop.ListVersionsEvaluatorsIdVersionsGetRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.commit(id, versionId, { ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Commit a version of the Evaluator with a commit message.

If the version is already committed, an exception will be raised.

πŸ”Œ Usage

await client.evaluators.commit("ev_890bcd", "evv_012def", {
    commitMessage: "Initial commit",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Prompt.

versionId: string β€” Unique identifier for the specific version of the Evaluator.

request: Humanloop.CommitRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.deleteEvaluatorVersion(id, versionId) -> void

πŸ“ Description

Delete a version of the Evaluator.

πŸ”Œ Usage

await client.evaluators.deleteEvaluatorVersion("id", "version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

versionId: string β€” Unique identifier for the specific version of the Evaluator.

requestOptions: Evaluators.RequestOptions

client.evaluators.setDeployment(id, environmentId, { ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Deploy Evaluator to an Environment.

Set the deployed version for the specified Environment. This Evaluator will be used for calls made to the Evaluator in this Environment.

πŸ”Œ Usage

await client.evaluators.setDeployment("ev_890bcd", "staging", {
    versionId: "evv_012def",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

environmentId: string β€” Unique identifier for the Environment to deploy the Version to.

request: Humanloop.SetDeploymentEvaluatorsIdEnvironmentsEnvironmentIdPostRequest

requestOptions: Evaluators.RequestOptions

client.evaluators.removeDeployment(id, environmentId) -> void

πŸ“ Description

Remove deployed Evaluator from the Environment.

Remove the deployed version for the specified Environment. This Evaluator will no longer be used for calls made to the Evaluator in this Environment.

πŸ”Œ Usage

await client.evaluators.removeDeployment("ev_890bcd", "staging");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

environmentId: string β€” Unique identifier for the Environment to remove the deployment from.

requestOptions: Evaluators.RequestOptions

client.evaluators.listEnvironments(id) -> Humanloop.FileEnvironmentResponse[]

πŸ“ Description

List all Environments and their deployed versions for the Evaluator.

πŸ”Œ Usage

await client.evaluators.listEnvironments("ev_890bcd");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluator.

requestOptions: Evaluators.RequestOptions

client.evaluators.updateMonitoring(id, { ...params }) -> Humanloop.EvaluatorResponse

πŸ“ Description

Activate and deactivate Evaluators for monitoring the Evaluator.

An activated Evaluator will automatically be run on all new Logs within the Evaluator for monitoring purposes.

πŸ”Œ Usage

await client.evaluators.updateMonitoring("id", {});

βš™οΈ Parameters

id: string

request: Humanloop.EvaluatorActivationDeactivationRequest

requestOptions: Evaluators.RequestOptions

Flows

client.flows.log({ ...params }) -> Humanloop.CreateFlowLogResponse

πŸ“ Description

Log to a Flow.

You can use query parameters version_id, or environment, to target an existing version of the Flow. Otherwise, the default deployed version will be chosen.

πŸ”Œ Usage

await client.flows.log({
    logId: "medqa_experiment_0001",
    id: "fl_6o701g4jmcanPVHxdqD0O",
    flow: {
        attributes: {
            prompt: {
                template: "You are a helpful assistant helping with medical anamnesis",
                model: "gpt-4o",
                temperature: 0.8,
            },
            tool: {
                name: "retrieval_tool_v3",
                description: "Retrieval tool for MedQA.",
                source_code: "def retrieval_tool(question: str) -> str:\n    pass\n",
            },
        },
    },
    inputs: {
        question:
            "Patient with a history of diabetes and hypertension presents with chest pain and shortness of breath.",
    },
    output: "The patient is likely experiencing a myocardial infarction. Immediate medical attention is required.",
    traceStatus: "incomplete",
    startTime: "2024-07-08T22:40:35",
    endTime: "2024-07-08T22:40:39",
});

βš™οΈ Parameters

request: Humanloop.FlowLogRequest

requestOptions: Flows.RequestOptions

client.flows.get(id, { ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Retrieve the Flow with the given ID.

By default, the deployed version of the Flow is returned. Use the query parameters version_id or environment to target a specific version of the Flow.

πŸ”Œ Usage

await client.flows.get("fl_6o701g4jmcanPVHxdqD0O");

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

request: Humanloop.GetFlowsIdGetRequest

requestOptions: Flows.RequestOptions

client.flows.delete(id) -> void

πŸ“ Description

Delete the Flow with the given ID.

πŸ”Œ Usage

await client.flows.delete("fl_6o701g4jmcanPVHxdqD0O");

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

requestOptions: Flows.RequestOptions

client.flows.move(id, { ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Move the Flow to a different path or change the name.

πŸ”Œ Usage

await client.flows.move("fl_6o701g4jmcanPVHxdqD0O", {
    path: "new directory/new name",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

request: Humanloop.UpdateFlowRequest

requestOptions: Flows.RequestOptions

client.flows.list({ ...params }) -> core.Page

πŸ“ Description

Get a list of Flows.

πŸ”Œ Usage

await client.flows.list({
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListFlowsGetRequest

requestOptions: Flows.RequestOptions

client.flows.upsert({ ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Create or update a Flow.

Flows can also be identified by the ID or their path.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

πŸ”Œ Usage

await client.flows.upsert({
    path: "Personal Projects/MedQA Flow",
    attributes: {
        prompt: {
            template: "You are a helpful medical assistant helping with medical anamnesis. Answer {{question}}",
            model: "gpt-4o",
            temperature: 0.8,
        },
        tool: {
            name: "retrieval_tool_v3",
            description: "Retrieval tool for MedQA.",
            source_code: "def retrieval_tool(question: str) -> str:\n    pass\n",
        },
        commit_message: "Initial commit",
    },
});

βš™οΈ Parameters

request: Humanloop.FlowRequest

requestOptions: Flows.RequestOptions

client.flows.updateLog(logId, { ...params }) -> Humanloop.FlowLogResponse

πŸ“ Description

Update the status, inputs, output of a Flow Log.

Marking a Flow Log as complete will trigger any monitoring Evaluators to run. Inputs and output (or error) must be provided in order to mark it as complete.

The end_time log attribute will be set to match the time the log is marked as complete.

πŸ”Œ Usage

await client.flows.updateLog("medqa_experiment_0001", {
    inputs: {
        question:
            "Patient with a history of diabetes and normal tension presents with chest pain and shortness of breath.",
    },
    output: "The patient is likely experiencing a myocardial infarction. Immediate medical attention is required.",
    traceStatus: "complete",
});

βš™οΈ Parameters

logId: string β€” Unique identifier of the Flow Log.

request: Humanloop.UpdateTraceRequest

requestOptions: Flows.RequestOptions

client.flows.listVersions(id, { ...params }) -> Humanloop.ListFlows

πŸ“ Description

Get a list of all the versions of a Flow.

πŸ”Œ Usage

await client.flows.listVersions("fl_6o701g4jmcanPVHxdqD0O", {
    status: "committed",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

request: Humanloop.ListVersionsFlowsIdVersionsGetRequest

requestOptions: Flows.RequestOptions

client.flows.commit(id, versionId, { ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Commit a version of the Flow with a commit message.

If the version is already committed, an exception will be raised.

πŸ”Œ Usage

await client.flows.commit("fl_6o701g4jmcanPVHxdqD0O", "flv_6o701g4jmcanPVHxdqD0O", {
    commitMessage: "RAG lookup tool bug fixing",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

versionId: string β€” Unique identifier for the specific version of the Flow.

request: Humanloop.CommitRequest

requestOptions: Flows.RequestOptions

client.flows.deleteFlowVersion(id, versionId) -> void

πŸ“ Description

Delete a version of the Flow.

πŸ”Œ Usage

await client.flows.deleteFlowVersion("id", "version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

versionId: string β€” Unique identifier for the specific version of the Flow.

requestOptions: Flows.RequestOptions

client.flows.setDeployment(id, environmentId, { ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Deploy Flow to an Environment.

Set the deployed version for the specified Environment. This Flow will be used for calls made to the Flow in this Environment.

πŸ”Œ Usage

await client.flows.setDeployment("fl_6o701g4jmcanPVHxdqD0O", "staging", {
    versionId: "flv_6o701g4jmcanPVHxdqD0O",
});

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

environmentId: string β€” Unique identifier for the Environment to deploy the Version to.

request: Humanloop.SetDeploymentFlowsIdEnvironmentsEnvironmentIdPostRequest

requestOptions: Flows.RequestOptions

client.flows.removeDeployment(id, environmentId) -> void

πŸ“ Description

Remove deployed Flow from the Environment.

Remove the deployed version for the specified Environment. This Flow will no longer be used for calls made to the Flow in this Environment.

πŸ”Œ Usage

await client.flows.removeDeployment("fl_6o701g4jmcanPVHxdqD0O", "staging");

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

environmentId: string β€” Unique identifier for the Environment to remove the deployment from.

requestOptions: Flows.RequestOptions

client.flows.listEnvironments(id) -> Humanloop.FileEnvironmentResponse[]

πŸ“ Description

List all Environments and their deployed versions for the Flow.

πŸ”Œ Usage

await client.flows.listEnvironments("fl_6o701g4jmcanPVHxdqD0O");

βš™οΈ Parameters

id: string β€” Unique identifier for Flow.

requestOptions: Flows.RequestOptions

client.flows.updateMonitoring(id, { ...params }) -> Humanloop.FlowResponse

πŸ“ Description

Activate and deactivate Evaluators for monitoring the Flow.

An activated Evaluator will automatically be run on all new "completed" Logs within the Flow for monitoring purposes.

πŸ”Œ Usage

await client.flows.updateMonitoring("fl_6o701g4jmcanPVHxdqD0O", {
    activate: [
        {
            evaluatorVersionId: "evv_1abc4308abd",
        },
    ],
});

βš™οΈ Parameters

id: string

request: Humanloop.EvaluatorActivationDeactivationRequest

requestOptions: Flows.RequestOptions

Directories

client.directories.list() -> Humanloop.DirectoryResponse[]

πŸ“ Description

Retrieve a list of all Directories.

πŸ”Œ Usage

await client.directories.list();

βš™οΈ Parameters

requestOptions: Directories.RequestOptions

client.directories.create({ ...params }) -> Humanloop.DirectoryResponse

πŸ“ Description

Creates a Directory.

πŸ”Œ Usage

await client.directories.create();

βš™οΈ Parameters

request: Humanloop.CreateDirectoryRequest

requestOptions: Directories.RequestOptions

client.directories.get(id) -> Humanloop.DirectoryWithParentsAndChildrenResponse

πŸ“ Description

Fetches a directory by ID.

πŸ”Œ Usage

await client.directories.get("id");

βš™οΈ Parameters

id: string β€” String ID of directory. Starts with dir_.

requestOptions: Directories.RequestOptions

client.directories.delete(id) -> void

πŸ“ Description

Delete the Directory with the given ID.

The Directory must be empty (i.e. contain no Directories or Files).

πŸ”Œ Usage

await client.directories.delete("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Directory. Starts with dir_.

requestOptions: Directories.RequestOptions

client.directories.update(id, { ...params }) -> Humanloop.DirectoryResponse

πŸ“ Description

Update the Directory with the given ID.

πŸ”Œ Usage

await client.directories.update("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Directory. Starts with dir_.

request: Humanloop.UpdateDirectoryRequest

requestOptions: Directories.RequestOptions

Files

client.files.list({ ...params }) -> Humanloop.PaginatedDataUnionPromptResponseToolResponseDatasetResponseEvaluatorResponseFlowResponse

πŸ“ Description

Get a paginated list of files.

πŸ”Œ Usage

await client.files.list();

βš™οΈ Parameters

request: Humanloop.FilesListRequest

requestOptions: Files.RequestOptions

Evaluations

client.evaluations.list({ ...params }) -> core.Page

πŸ“ Description

Retrieve a list of Evaluations for the specified File.

πŸ”Œ Usage

await client.evaluations.list({
    fileId: "pr_30gco7dx6JDq4200GVOHa",
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListEvaluationsGetRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.create({ ...params }) -> Humanloop.EvaluationResponse

πŸ“ Description

Create an Evaluation.

Create a new Evaluation by specifying the File to evaluate, and a name for the Evaluation. You can then add Runs to this Evaluation using the POST /evaluations/{id}/runs endpoint.

πŸ”Œ Usage

await client.evaluations.create({
    evaluators: [
        {
            versionId: "version_id",
        },
    ],
});

βš™οΈ Parameters

request: Humanloop.CreateEvaluationRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.addEvaluators(id, { ...params }) -> Humanloop.EvaluationResponse

πŸ“ Description

Add Evaluators to an Evaluation.

The Evaluators will be run on the Logs generated for the Evaluation.

πŸ”Œ Usage

await client.evaluations.addEvaluators("id", {
    evaluators: [
        {
            versionId: "version_id",
        },
    ],
});

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

request: Humanloop.AddEvaluatorsRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.removeEvaluator(id, evaluatorVersionId) -> Humanloop.EvaluationResponse

πŸ“ Description

Remove an Evaluator from an Evaluation.

The Evaluator will no longer be run on the Logs in the Evaluation.

πŸ”Œ Usage

await client.evaluations.removeEvaluator("id", "evaluator_version_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

evaluatorVersionId: string β€” Unique identifier for Evaluator Version.

requestOptions: Evaluations.RequestOptions

client.evaluations.get(id) -> Humanloop.EvaluationResponse

πŸ“ Description

Get an Evaluation.

This includes the Evaluators associated with the Evaluation and metadata about the Evaluation, such as its name.

To get the Runs associated with the Evaluation, use the GET /evaluations/{id}/runs endpoint. To retrieve stats for the Evaluation, use the GET /evaluations/{id}/stats endpoint.

πŸ”Œ Usage

await client.evaluations.get("ev_567yza");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

requestOptions: Evaluations.RequestOptions

client.evaluations.delete(id) -> void

πŸ“ Description

Delete an Evaluation.

The Runs and Evaluators in the Evaluation will not be deleted.

πŸ”Œ Usage

await client.evaluations.delete("ev_567yza");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

requestOptions: Evaluations.RequestOptions

client.evaluations.listRunsForEvaluation(id) -> Humanloop.EvaluationRunsResponse

πŸ“ Description

List all Runs for an Evaluation.

πŸ”Œ Usage

await client.evaluations.listRunsForEvaluation("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

requestOptions: Evaluations.RequestOptions

client.evaluations.createRun(id, { ...params }) -> Humanloop.EvaluationRunResponse

πŸ“ Description

Create an Evaluation Run.

Optionally specify the Dataset and version to be evaluated.

Humanloop will automatically start generating Logs and running Evaluators where orchestrated=true. If you are generating Logs yourself, you can set orchestrated=false and then generate and submit the required Logs via the API.

If dataset and version are provided, you can set use_existing_logs=True to reuse existing Logs, avoiding generating new Logs unnecessarily. Logs that are associated with the specified Version and have source_datapoint_id referencing a datapoint in the specified Dataset will be associated with the Run.

To keep updated on the progress of the Run, you can poll the Run using the GET /evaluations/{id}/runs endpoint and check its status.

πŸ”Œ Usage

await client.evaluations.createRun("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

request: Humanloop.CreateRunRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.addExistingRun(id, runId) -> unknown

πŸ“ Description

Add an existing Run to the specified Evaluation.

This is useful if you want to compare the Runs in this Evaluation with an existing Run that exists within another Evaluation.

πŸ”Œ Usage

await client.evaluations.addExistingRun("id", "run_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

runId: string β€” Unique identifier for Run.

requestOptions: Evaluations.RequestOptions

client.evaluations.removeRun(id, runId) -> void

πŸ“ Description

Remove a Run from an Evaluation.

The Logs and Versions used in the Run will not be deleted. If this Run is used in any other Evaluations, it will still be available in those Evaluations.

πŸ”Œ Usage

await client.evaluations.removeRun("id", "run_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

runId: string β€” Unique identifier for Run.

requestOptions: Evaluations.RequestOptions

client.evaluations.updateEvaluationRun(id, runId, { ...params }) -> Humanloop.EvaluationRunResponse

πŸ“ Description

Update an Evaluation Run.

Specify control=true to use this Run as the control Run for the Evaluation. You can cancel a running/pending Run, or mark a Run that uses external or human Evaluators as completed.

πŸ”Œ Usage

await client.evaluations.updateEvaluationRun("id", "run_id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

runId: string β€” Unique identifier for Run.

request: Humanloop.UpdateEvaluationRunRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.addLogsToRun(id, runId, { ...params }) -> Humanloop.EvaluationRunResponse

πŸ“ Description

Add the specified Logs to a Run.

πŸ”Œ Usage

await client.evaluations.addLogsToRun("id", "run_id", {
    logIds: ["log_ids"],
});

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

runId: string β€” Unique identifier for Run.

request: Humanloop.AddLogsToRunRequest

requestOptions: Evaluations.RequestOptions

client.evaluations.getStats(id) -> Humanloop.EvaluationStats

πŸ“ Description

Get Evaluation Stats.

Retrieve aggregate stats for the specified Evaluation. This includes the number of generated Logs for each Run and the corresponding Evaluator statistics (such as the mean and percentiles).

πŸ”Œ Usage

await client.evaluations.getStats("id");

βš™οΈ Parameters

id: string β€” Unique identifier for Evaluation.

requestOptions: Evaluations.RequestOptions

client.evaluations.getLogs(id, { ...params }) -> Humanloop.PaginatedDataEvaluationLogResponse

πŸ“ Description

Get the Logs associated to a specific Evaluation.

This returns the Logs associated to all Runs within with the Evaluation.

πŸ”Œ Usage

await client.evaluations.getLogs("id");

βš™οΈ Parameters

id: string β€” String ID of evaluation. Starts with ev_ or evr_.

request: Humanloop.GetLogsEvaluationsIdLogsGetRequest

requestOptions: Evaluations.RequestOptions

Logs

client.logs.list({ ...params }) -> core.Page

πŸ“ Description

List all Logs for the given filter criteria.

πŸ”Œ Usage

await client.logs.list({
    fileId: "file_123abc",
    size: 1,
});

βš™οΈ Parameters

request: Humanloop.ListLogsGetRequest

requestOptions: Logs.RequestOptions

client.logs.delete({ ...params }) -> void

πŸ“ Description

Delete Logs with the given IDs.

πŸ”Œ Usage

await client.logs.delete({
    id: "string",
});

βš™οΈ Parameters

request: Humanloop.LogsDeleteRequest

requestOptions: Logs.RequestOptions

client.logs.get(id) -> Humanloop.LogResponse

πŸ“ Description

Retrieve the Log with the given ID.

πŸ”Œ Usage

await client.logs.get("prv_Wu6zx1lAWJRqOyL8nWuZk");

βš™οΈ Parameters

id: string β€” Unique identifier for Log.

requestOptions: Logs.RequestOptions