LLM Mesh API

The LLM Mesh API lets applications use completion and embedding models, agents, knowledge banks, and other generative AI features. See the LLM Mesh section of the Developer Guide.

OpenAI-compatible API

The OpenAI-compatible API lets applications use standard OpenAI API clients to send Chat Completions and Responses requests to models available through the LLM Mesh, even when those models are not OpenAI models or do not use an OpenAI connection.

See the OpenAI-compatible API section of the Developer Guide.

Persisted conversations

Regular completion queries require the client to provide the conversation history with every request and to maintain the state of the context. Persisted conversations instead keep the active conversation history and context in DSS. The client only sends the input for the next turn. Persisted conversations support blocking or streamed responses, alternate conversation threads, tool outputs, and human-in-the-loop tool validation.

Conversations are scoped to a project. Applications can create, list, retrieve, update, archive, and delete them through the public API.

See the persisted conversations section of the Developer Guide for Python examples covering the conversation lifecycle, streaming, threads / branching, and tool interactions.

Note

Persisted conversations use project-level permissions. Users with Read project content can list and retrieve every conversation in the project. The API client can use end_user_id to filter conversation lists, but it is not an access-control boundary. When a conversation is created without an end_user_id, it defaults to the authenticated Dataiku user. Client applications serving multiple end users must enforce authorization before accessing a conversation.

Conversation storage

By default, DSS stores persisted conversations in a local SQLite database. This option requires no additional configuration.

Another option is to use a PostgreSQL connection. The connection’s identity must be able to create tables and indexes, and to select, insert, update, and delete rows in a dedicated schema.

To use that connection, as an administrator, in Administration > Settings > LLM Mesh > Persisted conversations, set Storage to PostgreSQL, select the PostgreSQL connection, and save the settings.

Changing the storage option does not migrate conversations that are already stored in the previous backend. These conversations remain in the previous store and are unavailable while another store is selected.