Agent Chat

Overview

Agent Chat allows builders to spin up a conversational interface on top of on top of a single agent, retrieval-augmented mode, or LLM, with little admin involvement.

Key Features

  • Single-agent conversational UI: OOB interface dedicated to one agent / Augmented LLM / RA model.
  • Self-service: Builders can easily create and share the interface with their team.
  • Governance and Control: Configure which generative AI assistant is exposed to the end-user, ensuring control over usage.
  • Response Transparency: View detailed sources, activities, and downloads for each agent response to understand how an answer was generated.

Note

Agent Chat is delivered through the Agent Hub plugin as a Visual Webapp.

Getting Access

To use Agent Chat, a Dataiku instance administrator must first install the Agent Hub plugin from the plugin store. The administrator also needs to set up the associated code environment.

Once installed, Agent Chat becomes available as a Visual Webapp that can be created within any Dataiku project.

Configuration

Requirements

To successfully set up and use an Agent Chat, the following prerequisites are necessary:

  • Dataiku Version: Dataiku 14.5 or later.

  • Agent Hub plugin: v1.3 or above. The latest plugin versions are always the best choice to leverage the plugin’s full capabilities.

  • User Profile:

    • To create and configure Agent Chat: you must have at least Write access to the project containing the Agent Chat webapp.

    • To use Agent Chat: users need access to the webapp and a DSS license.

  • Connections: A connection to at least one LLM.

Initial Setup

Agent Chat is created as a visual webapp within a Dataiku project.

  1. From your project, navigate to the Code menu and select Webapps.

  2. Click + New Webapp and choose Visual webapp.

  3. Select the Agent Chat template from the list.

Backend Settings

Once the webapp has been created, you can select the backend settings from the Edit tab.

  • Auto-start backend: Check this box to ensure the Agent Chat is running automatically.

  • Number of processes: 0

  • Container: Choose between using backend to execute or containers.

  • Data storage : Choose between a local or external database to store Agent Chat data.

Webapp Settings

Once the backend settings have been selected, all of the webapp settings are directly configured in the webapp (click on View).

Core Settings

Agent Chat has one mandatory LLM connection required to function.

  • Conversational Agent or LLM (Mandatory): Select the Agent or LLM with whom users will interact.

  • Optional Instructions: You can optionally add a system prompt that will be used by the Agent/LLM.

  • Title generation LLM: Pick the LLM used to generate titles.

  • Document guardrails: Screen documents title and body in-chat against a customizable list of prohibited keywords or regular expressions.

  • Include DSS caller ticket in the agent context: Toggle to include the DSS caller ticket in the context sent to the agent, to execute tools with end-user identity.

  • Permanently delete messages: Toggle so that when users delete messages in the conversation, they are also deleted from the database.

  • Advanced settings: Options to include network and client metadata for gateway reporting.

Charts Generation

Agent Chat can directly generate charts from agent responses that use tools returning records.

  • Charts Generation Mode:

    • Select None - Disable charts generation if this functionality is not needed. Else, charts will be generated using SQL artefacts of agents.

    • Select On Demand - Let end users generate charts, lets users decide when to generate charts and what type.

    • Select Auto - Generate charts automatically, automatically generates charts whenever the agent’s response includes SQL artefacts. The text completion model automatically chooses the type of graph based on the user query.

  • LLM: Choose an LLM to power chart generation.

Upload in Conversation

When the option Enable Document Upload is enabled by the admin, users can upload documents (txt, pdf, docx, png, jpg, jpeg, pptx, md) in chat to enrich conversation context. This requires setting up a managed folder where uploaded files will be stored.

There are two extraction modes admins choose from:

  1. Using pages as screenshots

Pages of PDF, PPTX and DOCX files are screenshotted and passed as is to the multimodal LLM (the selected LLM needs to support multimodal inputs). In this mode, a maximum number of images that can be uploaded per conversation must be defined. If this maximum is reached, images (png, jpg, jpeg) are passed on to the multimodal LLM but only text is extracted from documents (PDF, PPTX and DOCX files).

  1. Extracting text

In this mode, text is extracted from PDF, PPTX, DOCX files. Inline images in these documents can be processed with:

  • OCR:

    • This identifies the characters in the inline image and converts it to text

    • Best for documents that don’t hold significant visual information (i.e. receipts)

  • VLM:

    • Can “understand” visual elements, output is a textual description of the image. When queried, the LLM only uses the textual description generated, not the actual screenshot as in the first mode.

    • Best for photos, complex diagrams, charts, or screenshots of user interfaces where context/visual understanding matters.

    • It’s important to note that each and every image (including icons, logos, etc.) within the uploaded document are processed by the VLM, which can be very resource-intensive.

    • This requires DSS 14.3+

  • Not processed: This may result in information loss

User Experience

In the User Experience tab, the admin can configure conversation starters, which are suggested prompts in the homepagethat users can click on to start the conversation.

Look and feel customization

Customize the interface look match your company’s brand guidelines or preferences. This applies to all users of the webapp. * Select a managed folder where assets will be stored. * Choose a default main color, home page title and logos for the interface.

Setting up integration with Traces Explorer

On top of in-app traces of tool/agent calls, the comprehensive trace can be viewed using the integration with Trace Explorer. To use this integration, you must:

  1. Ensure that the Traces Explorer plugin is installed on your Dataiku instance.

  2. Create a Trace Explorer webapp in a project and give read access to relevant user groups.

  3. In Administration > Settings > LLM Mesh, set the default Trace Explorer webapp to the one you just created.

The button to navigate to Trace Explorer will then appear in See details > Activities.

Building tools that integrate with Agent Chat

Refer to Building tools that integrate with Agent Hub and Agent Chat.

Using Agent Chat

Conversations

Users can start a conversation from the homepage or click on Start new conversation.

The user query is directly passed to the agent/LLM defined in the admin settings, with the conversation history.

All conversations are saved in the left panel, where you can rename, delete, or revisit them.

Understanding Responses

Agent Chat allows you to inspect how an agent generated its response. Click the See details button below any response to open a panel with three tabs:

  • Sources: References to the documents, datasets, or other knowledge sources used by the agent.

  • Activities: A log of which agents and tools were called and what actions were performed.

  • Downloads: Download any files generated by the agent.

Monitoring and Quality checks

Extracting Agent Chat’s data

Once Agent Chat is set up and in production, you can extract Agent Chat’s data to monitor usage and evaluate the agent exposed in it.

To do that, you can use a recipe that extracts Agent Chat’s raw database.

In the flow, click on Add item > Recipe > Generative AI > Agent Hub > Export Agent Chat Data. In the recipe, you can choose the project, the Agent Chat webapp instance, and the tables you wants to export. These are mapped out to the output datasets. Given the potentially important size of artifacts, you can choose to keep them compressed. A scenario can be run regularly to sync the data.

Recipe settings depends on where Agent Chat stores its data.

Built-in storage (SQLite)

  • The recipe must run locally (i.e. no containerized execution) - Agent Chat keeps its database in the webapp’s workload folder, on the DSS node’s filesystem.

  • The recipe must run with the same identity as the Agent Chat’s backend — the “run as” user in the webapp’s settings. DSS grants access to the workload folder to that identity only; any other user is denied, even an administrator.

Note

When the backend identity changes with a new user, the filesystem access to the webapp workload folder is extended to that user (both the previous and the new backend identity can access it).

Remote SQL database

  • The end user running the recipe needs read access to that connection and its tables.

Monitoring

Once the database is extracted, a dashboard can then be easily created atop these datasets to monitor Agent Chat activity more deeply.

Quality checks

The extracted datasets include conversations and traces, can be used to perform custom quality checks of the agents exposed through Agent Chat.

Security and Permissions

When setting up Agent Chat, you have control over who has access to the webapp and what they can do with them. This is managed through a combination of DSS project permissions, DSS user groups and webapp settings.

Webapp permissions

First you want to define who can access the Agent Chat webapp. There are multiple ways to do this:

  • Give end-users or their user groups read access to the project hosting the Agent Chat application.

  • or Share the Agent Chat webapp within a workspace that the end-users have access to.

  • or Add the Agent Chat webapp to the list of authorized objects in the Project security settings and grant Read dashboards permission to end-users to the project hosting the webapp.

End-users that have write access to the project hosting the webapp will have access to the webapp’s settings.

Given there’s only one agent/LLM that can be connected to Agent Chat, controlling access to the webapp is the main way to control who can use the agent (i.e. users who have access to the webapp will be able to use the agent/LLM).

Document-level security

Document-Level Security enables granular access control over documents within a knowledge bank. It ensures that when a user performs a search or query, the results only include documents that user is authorized to view.

User security tokens are passed on to the agent in Agent Chat. These tokens aren’t used for authentication but rather filtering of the knowledge bank.

The caller security tokens include dss_group, dss_login and dss_email.

For instance, here are the tokens passed on for a user named Alex (login: alex), who belongs to readers and editors groups:

{
"callerSecurityTokens": [
   "readers",
   "editors",
   "dss_group:readers",
   "dss_group:editors",
   "dss_user_login:alex",
   "dss_user_emailaddress:[email protected]"
],
"dku_user_email": "[email protected]",
"dku_user_login": "alex",
"dku_user_display_name": "Alex"
}

In this case, Alex will only see documents that are accessible to the readers group or the editors group.

Role-based access control

Rather than calling tools with the backend identity, you can choose to use end-user identity for tool calls.

This is useful for row-level security on datasets and can be used when building custom tools making API calls in DSS.

This has to be configured at the tool level, and works natively with the SQL Query Tool and the Dataset Lookup Tool.

In the tool configuration, under the “Security” section, select “Access datasets as” and choose “End-user caller”.

This is done by obtaining the API ticket from the user’s browser headers, which is passed by the agent through the agent as dkuCallerTicket.

Note

Make sure that agents pass on the context. This requires ticking Forward context in the Query another agent or an LLM tool.