Sword Health Introduces MindEval: A New Framework To Measure LLM Clinical Competence

Sword Health announced the launch of MindEval, a novel framework designed in collaboration with licensed Clinical Psychologists to evaluate large language models (LLMs) in realistic, multi-turn mental health conversations. MindEval is a fully automated, model-agnostic framework that evaluates therapy sessions dynamically. The framework relies on the interaction between specific components to simulate a full therapeutic session.

The framework consists of three primary agents:

The Patient LLM (PLM): This model is prompted with a highly detailed profile and backstory to simulate a patient. It mimics a real person engaging in a multi-turn conversation, maintaining consistency in personality and . . .