MYTHOLOGY OF THE ARTIFICIAL INTELLIGENCE FIELD

In their daily activities, people are usually guided not by the results of their own rational analysis but by the results obtained by someone else; regularly used rules are transformed into "automatic" reactive actions. This is quite natural since rational analysis requires not only mental effort but also a significant amount of time for collecting information and the analysis itself—and this is not often acceptable for many reasons.

The result of these circumstances is a not entirely obvious aspect of the activity of homo sapiens: rational activity is based on the belief that the rules mentioned above that he must use are correct, useful, and therefore worth using.

Naturally, such confidence is sometimes justified and sometimes not; this is true not only for everyday activities, political life, and the religious sphere but also for the scientific and technical sphere.

An interesting situation arises when certain ideas, the correctness of which a person is sure, begin to contradict each other. Sometimes, this leads to a correction of what the person is sure of, but often, the contradictions are simply ignored - the person simply believes that both mutually exclusive statements are correct or that the existing contradiction is unimportant.

The described situation occurs in the field of AI in the form of quite persistent myths; below are examples of such myths:

MYTH: Language is a natural way of storing knowledge, which is the basis of rational reasoning and, at the same time, indicates the presence of intelligence.

Reality obviously contradicts this opinion: firstly, animals that do not have a language are capable of solving quite complex cognitive problems. Mice find a way out of a maze, birds figure out that they can throw pebbles into a bottle to make water accessible, bees find their way home, having moved kilometers away from home, octopuses learn to unscrew bottle caps, etc. Secondly, from time to time, almost every one of us gets into a situation when, despite serious mental efforts, we cannot explain something in words - and this obviously cannot happen if knowledge is stored in verbal form.

In fact, language is only a tool for exchanging information stored in a non-verbal form. Humans differ from animals in that they have the added ability to use language to exchange information, but the primary way of storing/representing knowledge about the world in humans and animals is the same and is non-verbal at its core. At the same time, the capabilities of natural language are obviously unsuitable for transmitting (let alone storing) many types of information/knowledge, so along with natural language, maps, programming languages, mathematical notation, drawings, notes, photographs, and videos are used to transmit information. If natural language were the primary way for humans to represent knowledge, there would be no need to provide missing persons, criminals, or pets with photographs.

However, the presence of the described contradictions does not prevent many specialists (if not most) in the field of AI from being the bearers and propagandists of this myth.

MYTH: LLM is based on statistics.

In forming an answer to a question, the LLM neural network of the system (like almost all the used variants of neural networks) does not use statistical data or statistical calculations. They are not used at the stage of training the LLM system, that is, at the stage of forming the numerical parameters of the neural network by an external training tool - neither gradient descent nor its alternatives are statistical in any way, as are other methods of finding the extremum of a function of many variables. In the " training " process, the same data are repeatedly used, but this does not indicate the presence of statistical processing in an explicit or implicit form. The approximation function, which is essentially a neural network, does not use statistical methods either at the formation stage or the use stage. All this is not a secret, but in most minds, faith in the myth has won over rationality - the myth is alive and well, preventing the correct assessment of the limits of the capabilities of this LLM technology.

MYTH: Modern LLM systems based on neural networks are capable of reasoning.

In reality, the LLM system generates a fragment of text similar to what can be found in a combination of the texts used to create ("train") the LLM and the text entered by the user. If a reasonable answer can be found - without any reasoning - the LLM generates something reasonable. The result is obvious nonsense as soon as the answer to the question in some ready-made form was not present in the "training" texts.

Testing of LLM capabilities is carried out in the overwhelming majority of cases on tasks that do not require reasoning, and the acceptability of the result is a false confirmation of LLM's use of some reasoning system.

In more complex testing situations, the LLM system's inability to generate a correct result is compensated for by user hints. The essence of this process is that the user, using his or her reasoning ability, first detects the incorrectness of the result generated by the LLM system and then informs the LLM system of the reasoning result —which then sometimes leads to the generation of the correct result and sometimes not.

Naturally, in a situation where the user lacks sufficient knowledge, he is unable to detect an incorrect result of the LLM system or to correct it by adding his own reasoning—a situation of misinformation arises as a consequence of LLM systems' inability to reason.

There are quite numerous examples of anecdotal answers that demonstrate a complete lack of both understanding of the question and reasoning in the process of generating an answer by the LLM system, but they are not a hindrance to the bearers of the myth.

One reason, however, in this case is the significant number of those who are trying to build a business using LLM and accordingly advertise the technology, hiding the boundaries of LLM systems' applicability.