Generate Text Completion

Description

Given a prompt and other inputs such as the Model ID, etc., generate a text completion response.

Project compatibility

Windows | Cross-platform

Configuration

• Connection ID - The connection established in Integration Service. Access the drop-down menu to choose, add, or manage connections.

• Input - The input is the prompt to generate completions. The method tokenizes the input internally. It is recommended not to leave any trailing spaces. This field supports String type input.
• Model ID - The ID of the model to be used for text generation. Select a model from the available dropdown list.

Manage Properties

Use the Manage Properties wizard to configure or use any of the object's standard or custom fields. You can select fields to add them to the activity canvas. The added standard or custom fields are available in the Properties panel (in Studio Desktop) or under Show additional options (in Studio Web).

Additional options

• Repetition penalty - The value which represents the penalty for penalizing tokens that have already been generated or belong to the context. This field supports Double type input.
• Temperature - The value used to module the next token probabilities. The range is 0.00 to 1.00. A value set to 0.00 would make it deterministic. This field supports Double type input.
• Max new tokens - The maximum number of new tokens to be generated. Default value is 150. This field supports Int64 type input.
• Min new tokens - The minimum number of new tokens to be generated. Default value is 50. This field supports Int64 type input.
• Stop sequences - A stop sequence is a string of one or more characters. If you specify stop sequences, the model will automatically stop generating output after one of the stop sequences that you specify appears in the generated output. This field supports String[] type input.
• Decoding method - The process a model uses to choose the tokens in the generated output, like sample or greedy. This field supports String type input.

Output

• Top generated text - The generated text with the highest confidence score. Automatically generated output variable. You can use it as it is or save it with a different name.
• Generate Text Completion - Automatically generated output variable. You can use it as it is or save it with a different name.