Generate Chat Completion

Description

Given a prompt and other inputs, generates a chat completion response.

Project compatibility

Windows | Cross-platform

Configuration

• Connection ID - The connection established in Integration Service. Access the drop-down menu to choose, add, or manage connections.

• Messages - The messages you want to use for the chat completion(s) generation. Provide the model with dialogue or 'chat' context by entering a string and then return. There must be an odd number of strings input for the activity to work. The first input will be the 'User', the second the 'Model', and so on. This field supports String type input.
• Model ID - The large language model (LLM) to utilize for the completion request. Select one of the available options: Bison-002, Bison-001.

Manage Properties

Use the Manage Properties wizard to configure or use any of the object's standard or custom fields. You can select fields to add them to the activity canvas. The added standard or custom fields are available in the Properties panel (in Studio Desktop) or under Show additional options (in Studio Web).

Additional options

• Context - Contextual information in the prompt guides the model's response and helps limit the scope of its answer based on the provided context. This field supports String type input.
• Max output tokens - The maximum number of tokens allowed for the prompt and generated answer. Fewer tokens are less expensive. Most models support a maximum of 1024 tokens. Default value is 1024.
• Temperature - A number between 0 and 1. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. Default value is 0.8.
• Top p - A number between 0 and 1. The lower the number, the lesser the randomness of the result. Defaults to 0.8.
• Top k - A number between 1 and 40. The higher the number, the higher the diversity of generated text. Defaults to 40.
• Examples - Structured message pairs, comprising an input message like "Who do you work for?" and its corresponding ideal response "I work for Ned", are essential examples for training the model to respond effectively in diverse conversations. This field supports String type input.

Output

• Top generated text - The top generated chat. Automatically generated output variable.
• Chat completion - Automatically generated output variable.