Cui’s insights focus on the limitations of large language models, emphasising that while they can generate human-like responses, their understanding remains fundamentally different from human cognition. She draws on the Chinese Room thought experiment to illustrate how AI systems manipulate symbols without true comprehension:
"A classic example is Searle's 'Chinese Room' thought experiment: if a person who does not know Chinese learns the rule of 'changing a string of symbols to another one when seeing one,' they can answer like a 'person who knows Chinese.' This process is completely based on symbol manipulation, but does not involve understanding the meaning of language."
She further explains how large models generate responses without truly grasping the meaning behind them:
"The big model relies on the prediction of text tokens and generates answers by processing symbols and rules, but does not really understand the reality pointed to by these symbols. For example, 'drinking red wine on Valentine's Day' is just a probabilistic language structure for it, rather than a comprehensive experience associated with taste, action, culture, and common sense. Therefore, even if the big model behaves as if it understands, it cannot be said that it 'really understands.'
Understanding is not just about processing symbols, but about understanding the meaning and reference of these symbols in the real world. For example, the word 'red wine' may be associated with color, smell, scene, social atmosphere, and the impact it brings, such as 'you can't drive after drinking.' This is a semantic understanding based on perception, experience, and common sense.
Even if the big model can describe 'knockover of the red wine glass, the glass shattered, and the red wine flowing down the edge of the table' in language, it does not know what 'breaking' means, let alone what the physical process of 'red wine flowing down' is. GPT, although it has ‘read’ the Internet, has never drunk a sip of red wine, smashed a glass, or experienced anything in person. Its ‘understanding’ is more based on the probabilistic structure of language rather than the causal model based on experience or physical common sense."
Cui also reflects on whether AI could ever develop a truly human-like understanding of the world:
"Even if AI in the future has a complex world model, it is very likely that it will still not be able to truly understand the world like humans. This is because they do not have instincts, intuition, emotions and pain, which are indispensable for understanding the world. As for machines, they may only be infinitely close in simulation, but they cannot be equal."
Cui’s full contributions can be found in the original article, which includes perspectives from five other interviewees. Read the full piece (in Chinese) below.