クエリ サービング エンドポイント

説明

Databricks Agents securely connect your data with any AI model to create accurate, domain-specific applications. Through the Mosaic AI Gateway, agents that use many different frameworks can be deployed and assigned serving endpoints (Mosaic AI Model Serving).

This activity enables the use of Databricks agents as participants in an automated process orchestrated by Maestro.

Creating the Databricks Agent

Generic LLM based agents, information extraction, and other types of agents within .Databricks can be used with the Databricks Agent connector as long as they are assigned and exposed via Serving endpoints. In most cases, integration with Maestro requires that the Agent render its output in a structured JSON schema. Information Extraction agents in Databricks are a good example. However, any agent can be prompted to respond in a well defined, simple schema using examples.

アクティビティの使用方法

To use this activity in a Maestro agentic process, follow these steps:

1. Add a service task element to the canvas and open the task's Properties panel.
2. Name the service task Databricks Hello World.
3. In the Implementation section, from the Action dropdown list, select Start and wait for external agent.
4. Select the Databricks Agent connector.
5. Select an existing connection or create a new one. For more information, see Databricks Agent authentication.

6. From Activity, select Query serving endpoint.

7. From Serving Endpoint, select an agent previously created in Databricks.
8. Under Additional Options, in the Message Content field, enter "What can you do?". Make sure to include the quotes in the prompt.
   

9. Connect the start event to the service task, and the service task to an end event node on the canvas.

10. Select Debug to run this process. After a successful run, review the global variables and look for the {:} response from the source: Databricks Hello World. Take note of the structure of the reply.

    For example, this is the agent's response to the prompt "What can you do?":

    {
      "id": "bf185700-c100-41be-9d4b-6a8aee2d8444",
      "databricks_output": {
        "databricks_request_id": "bf185700-c100-41be-9d4b-6a8aee2d8444"
      },
      "messages": [
        {
          "role": "assistant",
          "id": "run--38ced1fa-f810-49c2-87fc-e831e5ffb1d0-0",
          "content": "I can provide information and answer questions to the best of my ability. I can also execute Python code in a stateless sandboxed environment using the provided function. If you have a specific question or task in mind, feel free to ask and I'll do my best to assist you."
        }
      ]
    }

The agent’s output must be assigned to a process variable so it can influence the progress of the Maestro process, for example to make a decision based on a boolean evaluation, or to use the answer from a classification task.

1. In Design mode, select the agent from the design canvas.

2. In the Properties panel, select Add new and name the variable agent_reponse.

3. For Value, select Databricks Hello World > Response > Message array > Message content (string).

Tip: In practice, specify the structured output of your choice from the agent, and then evaluate the output within Maestro using the Expression editor to extract the specific part of the output you need in the type necessary for your process flow.

Example of handling agent output in Maestro using the Expression editor:

If the prompt was:

"What is the capital of France?" answer in a JSON only on the form of {"capital":"Normandy") only JSON output

The response is result.reponse.messages[0].content (type string):

{"capital":"Paris"}

If we want to convert it to JSON, create a new variable of type JSON, for example answer_in_JSON and use the Expression editor:

js:JSON.parse(result.response.messages[0].content)

The result of the js expression is (type JSON):

{
  "capital": "Paris"
}

Troubleshooting and Tuning

Beyond establishing connectivity, you should test prompts both in the Databricks workspace as well as from Maestro. This ensures you achieve the desired output that can best be consumed by Maestro, assigned to variables, and passed on to other actors in the process.

We recommend that detailed prompts remain within the system prompts of the agent within Databricks. The user prompt which is provided by Maestro to the agent at runtime should be brief and to the point. Its role is primarily to indicate the relevant variables needed by the agent to perform a specific tasks and generate an expected consistent output.

Output that is aimed at humans, for example the reasons for an escalation, can be easily passed as natural text for the human. Output that is expected for an API/robot action must be strictly composed. Here is an example user prompt that yields a specific output from an agent. Use the C# Expression editor within Studio to add variables as needed.

"What is the quantity on inventory of Order ID " + vars.orderId_1 + "respond only with a JSON object with the quantity in the key Order_Quantity. No explanations, only JSON"

The agent will reply with:

{"Order_Quantity":"100"}

Pay special attention to types in your request to the agent and in the actual response. Even if the response looks like type JSON, it may actually be of type string.