AGI: KNOWLEDGE MINING vs. KNOWLEDGE TRANSFER

SOCIETY ROLE AND REINFORCEMENT LEARNING WEAKNESS

There are only two ways to acquire new knowledge:

• find new knowledge autonomously as a result of knowledge mining

• get new knowledge from outside ready for use as a result of knowledge transfer

Knowledge transfer is relatively easy to implement, and the mutual exchange of knowledge is, in most cases, mutually beneficial. Therefore, it is not surprising that a person, being a social being, receives a significant part of knowledge in the knowledge transfer process.

Technologically, knowledge transfer involves only two abilities: the ability to interpret information received from outside, using previously accumulated knowledge, and the ability to remember the result of interpretation, replenishing the collected knowledge.

A kind of "border option" of the knowledge transfer is a widely discussed reinforcement learning technology, which is distinguished by minimal requirements for message interpretation addressed to the consumer (trained system) since the message is reduced to a single boolean value (correct/incorrect).

Many AI experts believe that to achieve human-level artificial intelligence systems, that is, to create AGI, it is enough to implement knowledge transfer. In this chapter, we will try to figure out whether this is actually the case.

The first circumstance, which is not apparent to everyone, is that to start a knowledge transfer, the two parties involved in the process must have a specific common stock of knowledge, which will allow one party to express the transferred knowledge, and the other party to interpret what is received. The initial (starting) common pool of knowledge can be initially given (innate) or must be formed by both communicating systems independently of each other. In the knowledge exchange process, such a shared knowledge pool expands, thereby expanding the possibilities for expressing/interpreting information.

Using the reinforcement learning approach, it is generally impossible to form referred above an initial pool of knowledge common to both parties. The reason is straightforward: reinforcement learning is based on encouraging specific "correct" actions/results; if the trainee does not demonstrate them for some reason, there is nothing to reinforce. The limit of possibilities in the training of animals is determined precisely by this: it is possible to stimulate in the reinforcement learning only what animals demonstrate of their own free will. In practical applications of reinforcement learning, this is compensated by the fact that the set of possible actions of the system being trained is not only finite (and very small) but also predetermined. In addition, the learning system explicitly or implicitly "knows" that it must try all the possibilities available. These additional conditions provide the theoretically unlimited potential for knowledge transfer. Thus, a certain amount of predefined knowledge is also required when implementing knowledge transfer using reinforcement learning. The extremely high labor intensity of training based on reinforcement learning (or, what is the same, extremely low efficiency) complicates the practical use of this technology if it is necessary to transfer large amounts of complexly structured knowledge.

The second circumstance is that the total amount of knowledge available to humanity is obviously growing, and it is equally evident that it is impossible to increase it by sharing existing knowledge (as opposed to equalizing the knowledge level among individuals). The total amount of knowledge of mankind is growing since it is replenished, and new portions of knowledge are discovered by individuals. And if it is declared to create a human-level AI system, this ability must be implemented.

As a result, we conclude that an intelligent system must be able to extract knowledge on its own if we are talking about a human-level system. If this ability is available, the implementation of transfer learning becomes quite simple. Both sides of the information exchange process operate in the same environment; each accumulates knowledge about the world around them, forming a subset of knowledge common to both sides required for knowledge transfer. At the initial stage of knowledge exchange, a correspondence is established between identical concepts, formed individually by both parties, which creates a common vocabulary of terms that is a base for communication language.

Of course, knowledge exchange is possible when both parties are interested in it.

What is required for an intelligent system to learn autonomously?

First, we need an initial universal knowledge extraction algorithm. It can be simple (more complex ones can be found later using this one), but it must be reliable.

Secondly, a subsystem for stimulating self-learning is needed, including a novelty detector and the inclusion of the level of novelty in the system for forming the overall level of satisfaction with the situation (the combination gives an analog of the instinct of curiosity).

With the outward similarity of stimulating self-learning to incentives in reinforcement learning, the essence is radically different: it is not the correctness/incorrectness of the action assessed but the degree of novelty. Growth in a variety of possible situations (in other words, the complication of the environment/task) affects the performance oppositely: it reduces the likelihood of correct action in the case of reinforcement learning and increases the probability of an unbeknown situation.

Humans and animals undoubtedly have such mechanisms developed in the process of evolution: an innate knowledge-seeking mechanism so simple that it can be determined by the genome and an information exchange mechanism based on the use of autonomously formed knowledge about the environment. The capabilities of humans are much higher than those of animals due to a larger amount of memory, a longer learning period, and better innate algorithms.

The mentioned simple but reliable algorithms for searching for new knowledge, including the detection of unknown objects/structures, the detection of unknown cause-and-effect relationships, and repeated sequences of events, are described in previous chapters. Our experiments allow us to assert that these algorithms lack the limitations inherent in reinforcement learning and are also radically more efficient.

In the AGI system, unlike animals and humans, the volume of initially embedded learning algorithms and the size of initially embedded knowledge does not have significant restrictions dictated by the genome size. However, the possibility of autonomous learning is necessary when it comes to human-level AI. In most application areas, simpler systems that do not reach human-level intelligence and do not require the ability to autonomously learn can be enough.

The use of knowledge mining can also be helpful in the case of complex activities and environments where the effort of reinforcement learning becomes unacceptably large.

SUMMATION

• New knowledge is acquired by mankind as a result of the realization by individuals of the ability to knowledge mining.

• The resource required by reinforcement learning in comparison with knowledge mining grows with the increasing complexity of the environment and tasks.

• Knowledge transfer (including reinforcement learning) and knowledge mining require some innate knowledge, but such congenital knowledge nature in these cases is significantly different.

• The use of knowledge mining can be helpful even in the case when such functionality is not required during the operation of the AI system.