LANGUAGE AND "Language Models"

A language is traditionally understood as a set of rules that allows one individual to present information in a way that would enable it to be correctly interpreted by any other individual who knows this language.

Language is a means of information exchange and is reduced to a set of exchange rules divided into syntactic, grammar, and semantic rules. All more or less complex languages, including both technical (programming languages, for example) and natural languages, allow the representation of unlimited information of arbitrary content, thereby ensuring the exchange of a wide variety of information.

With the advent of computers, tools have appeared for generating correct texts in different languages, including spell checkers and more advanced tools (for example, grammarly.com) that can analyze the semantics of texts and thereby help ensure the correct transmission of information to those who read it.

Thus, language is a tool for forming the presentation of information and for interpreting information presented by a text in the corresponding language. That is, a language is a communication tool. Knowledge of the language fully ensures the correctness of the presentation of information (that is, the text's correspondence to the author's intentions) and the correctness of the interpretation of the text by the reader. The knowledge of both communicating parties is stored in a specific internal representation, which can differ from each other arbitrarily. Accordingly, the use of the language is, on the one hand, encoding some available information according to the rules of the language and, on the other hand, extracting information, that is, converting it into the internal representation of the system.

An important aspect of the traditional notion of "language" is that the information to be encoded by one party and interpreted by the other party is not part of the language: memoirs of Winston Churchill are not part of the English language, and the browser source code is not part of the programming language. If all the information intended to be represented by the language were part of the language, then the communication of parties who are fully proficient in the language would become nonsense: both parties already know the same thing.

In connection with the development of computer tools focused on using natural language - GPT * and the like - a new term began to be used - "language model". Traditionally, a model is understood as a kind of simplification of the existing; in this case, this is clearly not the case. On the contrary, the “language model” contains much more than a language in the classical sense of the term, namely information extracted from the training text sets. That is, "language model" is a euphemism that denotes the aggregate of a language and some knowledge represented by texts in this language. All systems currently labeled with the term "language model" do not have the means of analyzing and incorporating textual information obtained during operation into an internal set of knowledge. In fact, the "language model" is a model of a person who, due to an unknown illness, remembers everything read earlier but has lost the ability to memorize new things and make logical conclusions using the available information. This analogy helps to quickly find answers to whether the corresponding system can bring harm and/or benefit, whether it is worth demonstrating to the general public, etc.