Generate Text Completion

Description

Given a prompt and other inputs such as the Model ID, etc., generates a text completion response.

Project compatibility

Windows | Cross-platform

Configuration

• Connection ID - The connection established in Integration Service. Access the dropdown menu to choose, add, or manage connections.

• Model - The large language model (LLM) to use for the text completions.
  Note: This activity enables users to select any GPT-3 model that has been deployed into your Azure instance. If you don’t see a model in the dropdown, please reach out to your Azure Administrator and ensure that they have deployed the model to production.
• Prompt - The prompt you want to use for the generation of the text completion. This field supports String type input.

Manage Properties

Use the Manage Properties wizard to configure or use any of the object's standard or custom fields. You can select fields to add them to the activity canvas. The added standard or custom fields are available in the Properties panel (in Studio Desktop) or under Show additional properties (in Studio Web).

Additional properties

• API version - The API version to use for this operation. this follows the YYYY-MM-DD format. This field supports String type input.
• Max Tokens - The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model’s context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096). Read more information about tokens.
• N - The number of completion choices you want the AI to return. The minimum value of this field is 1. Default value is 1.
• Temperature - The value of the sampling temperature to use. Higher values means the model takes more risks. Try 0.9 for more creative responses or completions, and 0 (also called argmax sampling) for ones with a well-defined or more exact answer. The general recommendation is to alter, from the default value, this or the Nucleus Sample Value, but not both values. Defaults to 1.
• Top P - A number between 0 and 1. The lower the number, the fewer tokens are considered. Defaults to 1.
• Stream - Specifies whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available, with the stream terminated by a data:[DONE] message. Defaults to false.
• Log Probs - Include the log probabilities on the logprob's most likely tokens, as well as the chosen tokens. For example, if logprobs is 5, the API returns a list of the 5 most likely tokens. The API always returns the logprob of the sampled token, so there may be up to logprobs+1 elements in the response. The maximum value for logprobs is 5. Defaults to null.
• Echo - In addition to completion, if this field is set to true, echoes back the prompt. Default value is false.
• Stop - Up to 4 sequences where the API stops generating further tokens. The returned text does not contain the stop sequence. Defaults to null.
• Presence Penalty - A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Defaults to 0.
• Frequency Penalty - Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Defaults to 0.
• Best Of - Generates best_of completions server-side and returns the best (the one with the highest log probability per token). Results cannot be streamed. When used with N, best_of controls the number of candidate completions and N specifies how many to return - best_of must be greater than N. Defaults to 1.

Output

• Top Generated Text - The top generated text completion string. If you are interested in only returning a single response (ie. N=1, which is the default), this will be the output you most likely will want to use. It is a ‘Text’ data type which is easy to assign a variable to and use directly in other activities/text inputs.
• Generate Text Completion - The full TextCompletion object that includes:
  • Choices or responses from the model. You can set the total number of responses that you want back from the model using the N field in the additional options of the activity which defaults to 1. This will return as an array with the first object starting at 0. Ex. Referencing a single object in an array within a For Each control: Response.Choices(0).Text. Read more about the Text Completion response object.