Open Internet by MindsNet

LLM Limitations in Simulating Human Preferences

Current Large Language Models (LLMs) struggle to accurately simulate human preferences, matching human majority only 53% of the time in 78 choice tasks. Even with detailed personas and chain-of-thought reasoning, LLMs fail to capture actual lived experiences. This limitation hinders the replacement of real human feedback with synthetic users in product testing and evaluation.

Computing & Technology, Computer Science, Machine Learning