Generate Text Completion

UiPath.MicrosoftAzureOpenAI.IntegrationService.Activities.GenerateTextCompletion

Description

Given a prompt and other inputs such as the Model ID, etc., generates a text completion response

Project Compatibility

Windows | Cross-platform

Configuration

• ConnectionId - The connection established in Integration Service. Access the drop-down to choose, add or manage connections. This field supports only strings or String variables.
• Prompt - The prompt you want to use for the generation of the text completion.

Advanced

Options

• Model - The model name or ID utilized for the generation of the text completions.
  Note: This activity enables users to select any GPT-3 model that has been deployed into your Azure instance. If you don’t see a model in the dropdown, please reach out to your Azure Administrator and ensure that they have deployed the model to production.
• Max Tokens - The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model’s context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096). You can read more on tokens here.
• N - The number of completion choices you want the AI to return. The minimum value of this field is 1. Default value is 1.
• Temperature - The value of the sampling temperature to use. Higher values means the model takes more risks. Try 0.9 for more creative responses or completions, and 0 (also called argmax sampling) for ones with a well-defined or more exact answer. The general recommendation is to alter, from the default value, this or the Nucleus Sample Value, but not both values. Defaults to 1.
• Top P - A number between 0 and 1. The lower the number, the fewer tokens are considered. Defaults to 1.
• Stream - Specifies whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available, with the stream terminated by a data:[DONE] message. Defaults to false.
• Log Probs - Include the log probabilities on the logprob's most likely tokens, as well as the chosen tokens. For example, if logprobs is 5, the API returns a list of the 5 most likely tokens. The API always returns the logprob of the sampled token, so there may be up to logprobs+1 elements in the response. The maximum value for logprobs is 5. Defaults to null.
• Echo - In addition to completion, if this field is set to true, echoes back the prompt. Default value is false.
• Stop - Up to 4 sequences where the API stops generating further tokens. The returned text does not contain the stop sequence. Defaults to null.
• Presence Penalty - A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Defaults to 0.
• Frequency Penalty - Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Defaults to 0.
• Best Of - Generates best_of completions server-side and returns the best (the one with the highest log probability per token). Results cannot be streamed. When used with N, best_of controls the number of candidate completions and N specifies how many to return - best_of must be greater than N. Defaults to 1.

Output

• Response - The full GenerateTextCompletion object that includes:
  • Choices - or responses from the model. You can set the total number of responses that you want back from the model using the ‘N’ field in the additional options of the activity which defaults to 1. This will return as an array with the first object starting at 0. Ex. Referencing a single object in an array within a ‘For Each’ control: Response.Choices(0).Text.

    For information on other parts of the Text Completion response object, please see this page.

• Top Generated Text - The top generated text completion string. If you are interested in only returning a single response (ie. N=1, which is the default), this will be the output you most likely will want to use. It is a ‘Text’ data type which is easy to assign a variable to and use directly in other activities/text inputs.