Knowledge Bank Search tool

This tool searches for relevant documents in a Knowledge Bank.

Importantly, this tool is a search/retrieval tool. It does not perform “Retrieval-Augmented Generation”: it does not “generate an answer” but simply returns matching documents. Generating the answer is the responsibility of the calling agent.

Core configuration

You configure the Knowledge Bank to use for the tool. For more information on how to build Knowledge Banks, see documentation about Knowledge bank.

Retrieval settings

The tool supports a variety of search options, some depending on the underlying vector store of the Knowledge Bank. The Knowledge Bank Search tool has the same retrieval options as the builtin RAG. See /generative-ai/knowledge/advanced-search for details about the retrieval settings.

Sources

In order for the Chat UIs or your own application to properly display documents, the tool returns rich source items, that can include a title, text, URL (for building a link to the document, for example in your Sharepoint site) and thumbnail URL (for displaying an image next to the result).

All of these are configured by optionally selecting which meta stored in the Knowledge Bank holds the information. Stored meta are configured in the embedding recipes.

Filtering

You can apply filters to the Knowledge Bank search tool in several ways

Static filtering

Define a static filter to permanently restrict the tool to a subset of the documents in the Knowledge Bank. This is useful for creating specialized tools that should only ever search over a specific portion of your data.

Dynamic filtering

Enable dynamic filtering to allow the calling application to provide filters for each query. Filters are passed in the context using the callerFilters key. You can specify any number of filters in callerFilters. callerFilters takes a filter property.

The filter clause can be constructed programmatically with the DSSSimpleFilter helper class. Additionally, if called from an agent, you can specify a toolRef parameter to only apply the filter to relevant knowledge bank search tools.

{
   "input": {
      "searchQuery": "Enter search query here"
   },
   "context": {
      "callerFilters": [
          {
             "filter": {
                 "operator": "AND",
                 "clauses": [
                    { "column": "Category", "operator": "EQUALS", "value": "Economy"},
                    { "column": "Year", "operator": "GREATER_THAN", "value": 2012},
                    { "column": "Month", "operator": "IN_ANY_OF", "value": ["January", "February", "March"]}
                ]
             },
             "toolRef": "optional tool ref"
          }
      ]
   }
}

Agent-inferred filtering

Enable agent-inferred filtering to empower the agent to generate filters automatically based on the user’s prompt.

When this is enabled, you must specify which columns the agent is allowed to filter on. You can also provide descriptions for these columns to help guide the agent in generating relevant filters.

Document-Level Security

Document-Level Security enables granular access control over documents within a knowledge bank. It ensures that when a user performs a search or query, the results only include documents that user is authorized to view.

To enable document-level security:

• A security token column must selected in the Embedding recipe settings,

• In the Knowledge Bank Search Tool, select ‘Enforce document-level security’,

• Provide the end-user security tokens at query time

Passing security tokens to the tool is done via the “context” parameter, in a key called “callerSecurityTokens”. Dataiku Chat UIs do this automatically along with Agent Hub and Agent Connect. Importantly, you must make sure to pass tokens of the “final” end-user, not the technical user simply calling the agent or tool.