AI Trust Layer

Enable custom AI gateways in AI Trust Layer for connecting own AI models through Integration Service on OpenShift.

Enabling custom AI gateways

The Bring-your-own AI gateway feature enables custom connector templates in the AI Trust Layer, allowing you to connect your own AI models through Integration Service's Connector Builder. You must enable this feature during installation by editing the input.json configuration file.

1. Open input.json and add or edit the following configuration under the platform component:

   {
     "platform": {
       "advanced_configuration": {
         "portal": {
           "llm_configurations": {
             "enableByoAiGateway": true
           }
         }
       }
     }
   }
   

2. Add or edit the following configuration under the llmgateway component:

   {
     "llmgateway": {
       "advanced_configuration": {
         "enableByoAiGateway": true
       }
     }
   }
   

3. Save input.json and proceed with the installation.

Once enabled, the Create custom connector option in the AI Trust Layer becomes available, allowing you to configure custom LLM connections using Connector Builder templates.

For details, refer to:

• Configuring LLMs
• How-to guide: Connecting Mistral through AI Trust Layer