There are many definitions of what is meant by AGI; although they differ in sound, they are nevertheless close in meaning. I like the definition on the Peter Voss website https://agiinnovations.com/what-is-agi:
"Computer systems based on AGI technology ('AGIs') are specifically engineered to be able to learn. They are able to acquire a wide range of knowledge and skills via learning, similar to the way we do. Unlike current computer systems, AGIs do not need to be programmed to do new tasks. Instead, they are simply instructed and taught by humans. Additionally, these systems can learn by themselves both implicitly 'on-the-job', and explicitly by reading and practicing. Furthermore, just like humans, they resiliently adapt to changing circumstances."
But the definition describes what the AGI developer should get as a result. And what is required to provide this required output? In HOW TO LEARN PERMANENTLY and ARCHITECTURE, the subject of discussion was the "parts" and "units" that make up the overall design of the AGI system. Here we will discuss the fundamental capabilities that these components should provide.
As stated in the definition above, an AGI system must be able to learn. In technological terms, this primarily means that it is necessary to have a memory to remember what the system has learned. This seemingly trivial aspect, however, has very non-trivial features. Since it is implicitly meant that “learning” means “learning anything, as people do,” it requires the technical ability to memorize information of an arbitrary type - visual, textual, audio, information about actions, events, dependencies, processes, algorithms, and etc. For almost every kind of information, there are specialized data presentation technologies and methods for manipulating them. Still, it is unrealistic to obtain the required more or less universal system by simply combining several specialized ones. Some specific universal core is required, linking all subsystems into one whole. The most suitable option for these purposes is to use a combination of a semantic graph with storage of quantitative data, textual information, etc.
The previous chapters have discussed many aspects of processing information to decide what actions to take. Such processing includes, in particular, object recognition, classification, forecasting, and planning. There is, however, an aspect that is "left out" of modern discussions in the field of AGI: where do the objects that the system operates in the AI system come from - pieces and positions in chess, Go and Atari games, cats and cookies in image recognition systems, words in AI systems focused on working with texts? These are just those logical entities, the relationships between which are stored in the Semantic Knowledge Storage. Having carefully analyzed the currently successfully applied narrow AI systems based on artificial neural network technologies, we will find that all logical objects and concepts used in one way or another have a human as their original source. Training datasets, ontologies, expert systems rules, statements in the CYC - everywhere, directly or indirectly, a set of elementary objects that the AI system operates lead to a human. When we talk about a person's ability to learn, we explicitly or implicitly mean the ability to generate new logical objects by observing the world around us, not only to receive them from other knowledge holders in the process of "learning with a teacher" (the opposite would mean the innateness of all knowledge and their constant volume for mankind). Broadcast promises to develop existing AI systems to human-level AI by increasing their size and unspecified complication of the structure bypassed this aspect with complete silence. Thus, one of the critical issues remains entirely out of the public eye of AI developers, leaving us with little real chance that current developments will lead to the creation of AGI.
This is not to say that this aspect does not attract attention: AI developers have long formulated the "grounding problem" https://en.wikipedia.org/wiki/Symbol_grounding_problem, which at first glance concerns the problem under discussion. But the problem of linking logical concepts/objects/symbols with objects/processes of the environment (real or virtual) in which the AI system operates is not the same as the problem of generating logical objects. If a logical thing is generated in one way or another, it certainly points to the corresponding environment object ab initio and directly or indirectly determines the method of detecting the object, so the grounding problem simply does not arise.
The generation of new objects/concepts is a formulation of rules that allow the object to be detected. The source of new information from the environment that allows generating new concepts/logical objects are sensors or virtual counterparts that supply the AI system. Unlike messages coming from outside using a communication protocol operating with a predefined set of symbols/concepts, they contain raw "atomic" quantitative information about the environment: brightness and color of an image pixel, instantaneous sound intensity, etc. Formally, this is a set of elementary predefined (by the design of sensors) objects. At each moment, any complex observed object is represented by a subset of atomic objects corresponding to the observed picture as a whole. The number of potentially observable objects is unlimited.
Statistical methods for detecting dependencies - primarily clustering - obviously cannot be the basis of a universal process for generating new concepts/objects at once for two reasons: the current set of data from sensors at different times corresponds to different groups of objects, and the same set of objects may be described by very different sets of sensory data (the same scene observed from different positions).
The potential unlimitedness of the number of different objects in the environment requires using a certain universal principle of object detection and its corresponding implementation in the form of a universal algorithm. The specifics of a particular environment and a set of sensors for such a universal algorithm are the algorithm's parameters. Of course, different objects have different complexity of their description and detection. The limited resources of the AI system determine the limit of the complexity of objects that it can detect in a dynamic environment.
Thus, a prerequisite for the successful design of an AGI system is the presence in this system of two things: a universal way of representing the generated knowledge and a universal algorithm for detecting unknown objects in the environment. The degree of universality can vary; a specific environment and a particular set of sensors are acceptable starting points.
A universal approach to detecting unknown objects is sketched in one of the previous chapters: AGI: STRUCTURES DISCOVERING
Let me copy-paste here the point on your post from Barry Smith "What is missing in his account is the absence of an AGI will. To have AGI the machine would have to want things, have intentions, ...
See attached extract from 9781003310105.pdf
BS" see https://groups.google.com/g/ontolog-forum/c/-nl9WyPa9qc/m/2Zm4HfrNAwAJ