LLM Council

Note

This capability is provided by the “Agentic LLM Council” plugin, which you need to install. Please see Installing plugins.

An LLM Council agent queries multiple language models or agents in parallel to generate, critique, and refine a response. Using structured disagreement and synthesis provides a more balanced result than a single model alone.

To configure an LLM-council, you need to create an agent of type “Custom agent llm-council-provider”. You can then assign any models from the LLM Mesh to two roles: Members and a Chairman.

The workflow proceeds as follows:

• Answering: Members generate independent answers.

• Review: Members critique peer answers.

• Arbitration: The Chairman synthesizes the final response from all answers and reviews.

This agent can typically be used in the LLM Evaluate or Agent Evaluate recipes to apply council-based consensus to evaluations.