Open Internet by MindsNet

Improving Confidence Evaluation for Local LLMs

The author encountered difficulties in building an effective confidence evaluator for local LLMs, specifically with the 'Grounded Self-Assessment' signal. The evaluator's performance was hurt by injecting retrieved context, leading to incorrect self-assessment by the model.

Computing & Technology, Computer Science, Machine Learning