Generate Chat Completion

UiPath.MicrosoftAzureOpenAI.IntegrationService.Activities.GenerateChatCompletion

Description

Given a user prompt, an optional instruction, and other optional inputs such as the Model ID, etc., generates a chat completion response. This functionality mimics the popular ChatGPT capabilities. This activity enables users to interact with GPT-3.5 and GPT-4 models and simulate a dialogue experience with the model via context and user roles.

Project Compatibility

Windows | Cross-platform

Configuration

• Connection - The connection established in Integration Service. Access the drop-down to choose, add or manage connections. This field supports only strings or String variables.
• Prompt - The prompt you want to use for the generation of the chat completion. You can read more on prompts here.

Advanced

Options

• Instruction - Instructions on how you want the AI to generate the chat completion response. This is the equivalent of the ‘system role’ discussed in the OpenAI docs. The ‘Assistant’ role is not currently available in this activity. For more information, please see this page.
• Model Name - The large language model (LLM) to use for the chat completion. You can choose between gpt-35-turbo and gpt-4. You can read more on the differences between these models here. By default, gpt-4 is selected.
  Note: This activity enables users to select any GPT-3.5 or GPT-4 model that has been deployed into your Azure instance. If you don’t see a model in the dropdown, please reach out to your Azure Administrator and ensure that they have deployed the model to production.
• Max Tokens - The maximum number of tokens allowed for the prompt and generated answer. Fewer tokens are less expensive. Most models support a maximum of 4096 tokens, however, some models support only 2048. Default value is 1920. You can read more information on Tokens here.
• N - The number of completion choices you want the AI to return. The minimum value of this field is 1. Default value is 1.
• Temperature - A number between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. Default value is 1.
• Top P - A number between 0 and 1. The lower the number, the fewer tokens are considered. Defaults to 1.
• Stream - Specifies whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available, with the stream terminated by a data:[DONE] message. Defaults to false.
• Presence Penalty - A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Defaults to 0.
• Frequency Penalty - Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Defaults to 0.

Output

• Response - The full GenerateChatCompletion object that includes:
  • Choices - or responses from the model. You can set the total number of responses that you want back from the model using the ‘N’ field in the additional options of the activity which defaults to 1. This will return as an array with the first object starting at 0. Ex. Referencing a single object in an array within a ‘For Each’ control: Response.Choices(0).Text

    For information on other parts of the Chat Completion response object, please see this page.

• Top Generated Text - The top generated chat completion string. If you are interested in only returning a single response (ie. N=1, which is the default), this will be the output you most likely will want to use. It is a ‘Text’ data type which is easy to assign a variable to and use directly in other activities/text inputs.