ISSN 0869-6632 (Print)
ISSN 2542-1905 (Online)


The article published as Early Access!

This is an open access article distributed under the terms of Creative Commons Attribution 4.0 International License (CC-BY 4.0).
Full text PDF(Ru):
Language: 
Russian
Article type: 
Article
UDC: 
530.182
EDN: 

Dynamics of recurrent neural networks with piecewise linear activation function in the context-dependent decision-making task

Autors: 
Kononov Roman Andreevich, Institute of Applied Physics of the Russian Academy of Sciences
Maslennikov O.  V., Institute of Applied Physics of the Russian Academy of Sciences
Nekorkin Vladimir Isaakovich, Institute of Applied Physics of the Russian Academy of Sciences
Abstract: 

The purpose The aim of this paper is to investigate the dynamic mechanism of solving the cognitive task of context-dependent two-alternative decision-making, developed in the process of reinforcement learning by recurrent neural networks, and to develop a methodology for analyzing such models based on the theory of dynamical systems.

Methods. An ensemble of neural networks with a piecewise linear activation function is constructed. The models were optimized using the reinforcement learning method of proximal policy optimization. The structure of a trial with constant stimuli over a long stage
allows us to treat inputs as system parameters and consider the system as autonomous during finite time intervals.

Results. The dynamic mechanism of two-alternative decision in terms of attractors of autonomous systems is uncovered and described. Possible types of attractors in the model under consideration are described and the distribution of types of attractors in the
ensemble of models relative to the parameters of the cognitive task is studied. In the obtained networks, a stable division into functional populations is revealed for the ensemble of models. The process of evolution of the composition of these populations is studied. Based on the obtained understanding of the dynamic mechanism, a two-dimensional network was constructed that solves the simplified problem of two-alternative decision without context.

Conclusion. The proposed approach allows us to qualitatively describe the mechanism of solving the problem in terms of attractors. Such description allows us to study the dynamics of functional models and identify the populations behind dynamic objects. This approach makes it possible to track the evolution of the attractors of the system and the corresponding populations in the learning process.
 

Acknowledgments: 
This work was supported by the Russian Science Foundation, grant № 23-72-10088.
Reference: 

-

Received: 
29.06.2024
Accepted: 
04.09.2024
Available online: 
06.12.2024