Internal monologue: artificial intelligence has been taught to think (was it possible to do that?)

A new study shows that providing artificial intelligence systems with an "inner monologue" makes their performance significantly better. Essentially, artificial intelligence has been taught to think before responding to queries, much like how humans think about what to say next before speaking. This differs from how popular AI language models like ChatGPT behave. The latter do not "think" about what they are writing and do not consider different possibilities for subsequent steps in the conversation.

A new method called Quiet-STaR instructs the AI system to parallelly generate multiple internal arguments before responding to a query. When the AI responds to prompts, it generates many options and outputs the best answer. In the end, artificial intelligence learns by discarding options that turn out to be incorrect. Essentially, the learning method gives AI models the ability to predict future conversations and learn from current ones.

Researchers from Stanford University and Notbad AI applied the Quiet-STaR algorithm to Mistral 7B, a large open-source language model, and published the results on arXiv. The Quiet-STaR trained version of Mistral 7B scored 47.2% on an argumentation test, up from 36.3% before any training. The model still failed a math school test, scoring 10.9%. However, this is nearly double the initial version's score of 5.9%.

Models like ChatGPT and Gemini do not correlate data with common sense or context, so they do not actually understand their own responses, just generating words. Previous attempts to improve the "thinking" ability of language models have been very specialized and could not be applied to various AI models.

The self-learning STaR algorithm that the researchers used as the basis for their work is an example of such learning, but it is limited by these constraints. The scientists who developed Quiet-STaR named the method so because the work of STaR was done in the background. This could work with different models, regardless of the training data. Now they want to explore how similar methods can reduce the gap between neural network-based artificial intelligence systems and human reasoning abilities.

Source: Live Sciense