We believe that in this changing and noisy world, the brevity of memory is necessary. In a changing environment, forgetting is adaptive because it allows more flexible behavior; In a noisy world, forgetting is adaptive because it prevents over-fitting of special events. (2) Based on this view, the permanence of memory is not always useful. For example, for short-lived or unusual aspects in the world, the permanence of memory will be harmful, because it may lead to inflexible behavior and incorrect prediction; Persistence is useful only when those aspects of experience are relatively stable and new experiences are predicted. (3) Therefore, only through the interaction of permanence and transience can memory show its real purpose: to use past intelligence to guide decision-making (Du Dai and Carruthers, 2005; Schacter et al., 2007). (4) Next, we review the calculation cases that use timeliness to increase the flexibility of behavior and promote generalization. In addition, we also determine the similarity between the use of temporality in calculation and its realization in the brain.
Neural network:
For neural networks using distributed representation, new learning is a great challenge (French,1999; Lewandowski and Li; McCluskey and Cohen,1989; Ratcliffe, 1990). There are two challenges: new learning may overwrite previous memories (that is, catastrophic interference); New learning will be hindered by existing memories (that is, active mutual interference) (Burgess et al.,1991; McCluskey and Cohen,1989; Palm, 2013; Siegel and hassell Mo, 2002). This is the dilemma of "stability and plasticity" in neural networks (Abraham and Robins, 2005; Carpenter and Grosberg, 1987). According to the traditional view, the persistence of memory and the flexibility of behavior are incompatible, because a network that is good at maintaining persistent memory will find it difficult to learn new information, especially information that conflicts with previous experience.
Recently, however, the neural network model using external storage devices or synapses has changed on multiple time scales, which challenges the universality of this dilemma (Graves et al., 2016; Kirkpatrick et al., 2017; Santoro et al., 20 16). In addition, another strategy that the brain can use to solve this problem is to use orthogonal representation to sparsely encode experiences, which may be caused by the process of pattern separation (Yassa and Stark, 20 1 1). Context dependence of memory is an example of this strategy: by maintaining orthogonal patterns, memories encoded in a particular context are more likely to be expressed in that context than in other contexts (Ma Ren et al., 20 13). This strategy maximizes the number of patterns that can be stored in the neural network without interference (Amari, 1989).
Brain:
However, in a dynamic environment, it is very important to discard outdated information no matter how limited the capacity is (Kraemer and Golding, 1997). If the environment has changed, but our memory has not changed, then we may stick to the old memory and hurt ourselves. Therefore, transience can promote decision-making by eliminating outdated information, so that organisms can respond to changes in their environment more effectively.
Recent studies have provided evidence that forgetting is necessary for flexible behavior in dynamic environment (Dong et al., 2016; Epp et al., 2016; Shuai et al., 20 10). Shuai and his colleagues trained flies to distinguish two smells (A and B) and found that inhibiting Rac 1 can slow down forgetting. Drosophila group which inhibited RAC 1 showed impaired reverse learning (A- or B+), indicating that the retained memory affected the new learning. In the fruit fly group that activated RAC 1, the result was just the opposite, and forgetting the old memory promoted reverse learning. The results of this model are extended to five different flies, which are designed to express mutations related to autism spectrum disorders, which also interfere with Rac activities. All these fruit flies with impaired Rac function show impaired forgetting, which in turn impairs reverse learning (Shuai et al., 20 10).
Other studies have shown the same results. EPP et al. (2065438+June) studied reverse learning (mediated by neurogenesis) after forgetting. In the experiment, they trained mice to find a fixed platform in the water maze, and then retrained them in the same maze, but the platform was moved to the opposite position. In this way, mice with enhanced hippocampal neurogenesis can find a new platform position more effectively (enhanced hippocampal neurogenesis will lead to forgetting the original position); However, when the level of hippocampal neurogenesis decreases, the opposite pattern is observed, because the inhibition of neurogenesis maintains the memory of the original position and interferes with the learning of the new position.
Similar results were also observed in the task of situation-smell pairing (Epp et al., 20 16). After training, the increase of neurogenesis will lead to forgetting the paired association that has been learned, but it will help the subsequent reverse learning. However, this promotion does not apply to any learning, and the benefits of new learning can only be observed when there is obvious conflict with the original learning. For example, if mice with increased nerves are trained in a new environment-odor pair, they will not show any benefit. These findings indicate that adult hippocampal neurogenesis promotes forgetting, and forgetting enhances behavioral flexibility by removing or weakening outdated information. Related papers on the relationship between neurogenesis and flexibility are: Burghart et al. (2012); Garthe et al. (2009), (2016); Luu et al (2012); Swan, etc (2014); And Winocur et al. (20 12).
Reference file:
Azevedo et al., 2009: We found that the average adult male brain contains 86 1 8 1 100 million nerve cells (neurons) and 8,469.8 billion nerve cells (non-neurons). In terms of the number of neurons and non-neurons, the human brain is an equally enlarged primate brain.
Amari, 1989: When most components of the coding mode to be stored are 0 and only a few components have a ratio of 1, this coding scheme is called sparse. The storage capacity and information capacity of sparse coded associative memory are analyzed in detail, and it is proved that it is directly proportional to the number of neurons n logn. Compared with the general non-sparse coding scheme (about 0. 15n), the proportional relationship is very large.
Du Dai and Carruthers, 2005: Research shows that memory may be the imprint of the past, which is very important for the future cognitive process.
Schacter et al., 2007: Imagining the future depends largely on the neural mechanism of being able to remember the past. These findings lead to the concept of prospective brain, that is, a key function of the brain is to use stored information to imagine, simulate and predict possible future events. According to this idea, we believe that processes such as memory can be effectively re-conceptualized.
French, 1999: This paper studies the causes, consequences and various solutions of catastrophic forgetting in neural networks. This review will consider how the brain can overcome this problem, and will also discuss the impact of this solution on distributed connection networks.
McCloskey and Cohen, 1989: This paper discusses catastrophic interference in connectionist networks. When the network is trained in sequence, new learning may cause disastrous interference to old learning. The analysis of the causes of interference shows that at least some interference will occur when the new learning may change the weights involved in the old learning. The simulation results only show that the interference is catastrophic in some specific networks.
Ratcliff, 1990: Evaluate the multi-layer memory connection model based on the encoder model by using the back propagation learning rule. These models are applied to the standard recognition and memory process, and the items are studied in turn, and then their retention rates are tested. Sequential learning in these models leads to two main problems. First of all, well-learned information will be quickly forgotten with the learning of new information. Second, the difference between learning projects and new projects either decreases with the progress of learning or is non-monotonous. In order to solve these problems, we studied the network operation in the multi-layer model and several variants of the multi-layer model, including a model with pre-learning memory and a context model, but none of them solved these problems. The problems discussed provide limitations for the connectionist model of human memory and task, in which not all the information to be learned can be obtained in the learning process.
Burgess et al., 199 1: A neural network model can be established to record the results of human memory experiments on the learning item list. This paper summarizes the psychological experiment of learning list. Hopfield- parisi neural network is used to simulate many simple features of sequence effect in sequence recall. The functional relationship between the recall rate of an item and its quantity, position in the list and similarity is studied by simulation. More complex experiments involve different types of projects, and related activity patterns are used for modeling. By considering the weight distribution and signal-to-noise ratio parameters, the working principle of the model is understood.
Palm, 20 13: This paper introduces the theory, practice and technical development of neural associative memory in recent 40 years. The importance of sparse coding of associative memory mode is pointed out. The application of associative memory network in large-scale brain modeling is also mentioned.
Siegle and Hasselmo, 2002: Connectionism model is considered as a promising tool to understand the nature of psychological disorders and guide their evaluation. Specifically, the connectionist model can guide the following aspects of the evaluation process: understanding which structures are related to evaluation, designing methods to evaluate these structures, and understanding individual differences in evaluation data.
Abraham and Robins, 2005: Memory maintenance is widely considered to involve the long-term maintenance of synaptic weights in related neural circuits during learning. However, although the recent technological progress is exciting, this intuitive and attractive assumption cannot be confirmed by experiments. Artificial neural networks provide an alternative method because they allow continuous monitoring of the weights of individual connections during learning and maintenance. In this model, if the network wants to keep the previously stored materials while learning new information, it needs to constantly change the connection weight. Therefore, the duration of synaptic changes does not necessarily define the persistence of memory; On the contrary, it may be necessary to adjust the balance between synaptic stability and synaptic plasticity in order to obtain the best memory retention in real neuronal circuits.
Carpenter and Grossberg, 1987: Adaptive vibration structure is a kind of neural network, which can self-organize stable pattern identification codes in real time to respond to any input pattern sequence. An adaptive vibration structure ART2 is introduced. ART2 architecture embodies the solutions of various design principles such as stability-plasticity trade-off, search-direct access trade-off, matching-reset trade-off and so on.
Graves et al., 20 16: Artificial neural network has obvious advantages in sensory processing, sequential learning and reinforcement learning, but its ability to express variables and data structures and store data for a long time is limited due to lack of external memory. Here we introduce a machine learning model called Differentiable Neural Computer (DNC), which is composed of a neural network that can read and write external memory matrix, similar to the random access memory in traditional computers. Like a traditional computer, it can use memory to represent and manipulate complex data structures, but like a neural network, it can learn to do so from data. The results show that DNC has the ability to solve complex structured tasks that neural networks can't accomplish without external read-write memory.
Kirkpatrick et al, 20 17: The ability to learn tasks in a sequential way is very important for the development of artificial intelligence. So far, neural networks have not been able to do this. We have proved that it is possible to overcome this limitation and train the network, so that it can maintain its expertise in tasks that it has not experienced for a long time. We remember the results of the old tasks by selectively slowing down the learning of the weights that are important to those tasks, which proves that our method is scalable and effective.
Santoro et al., 20 16: In the process of system integration, there is a transition from detail-dependent plot memory to general schema memory. This conversion is sometimes called "memory conversion". Here, we show the advantage that memory conversion has not been paid attention to before, that is, it enhances the ability of strengthening learning in dynamic environment. We developed a neural network, which was trained to find rewards in foraging tasks where the reward position changed. The network can use the memory of a specific location (scene memory) and the statistical model of the location (schematic memory) to guide its search. Our work puts forward the theoretical question of why memory conversion will happen again, shifting the focus from avoiding memory interference to strengthening reinforcement learning across multiple time scales.
Yassa and Stark, 20 1 1: The ability to distinguish similar experiences is an important feature of situational memory. For a long time, this ability has been considered to require hippocampus, and the calculation model shows that it depends on pattern separation. However, experimental data on the role of hippocampus in pattern separation have not been obtained until recently. This paper summarizes several types of data. We discuss the effects of aging and adult neurogenesis on pattern separation, emphasize several challenges of cross-species and cross-pathway connection, and propose future research directions.
Ma Ren et al., 20 13: Context surrounds events and gives them meaning; They are very important for recalling the past, explaining the present and predicting the future. In fact, the brain's ability to contextualize information allows great cognitive and behavioral flexibility. Studies on the regulation and disappearance of Pavlov's fear in rodents and humans show that neural circuits including hippocampus, amygdala and medial prefrontal cortex participate in the process of learning and memory, thus realizing situational dependence behavior.
Kraemer and Golding, 1997: Summarized the research status of human adaptive forgetting, and put forward the viewpoint of animal adaptive forgetting. The discussion includes the theoretical premise of forgetting, the review of the selective phenomenon of animal adaptive forgetting, the description of the possible mechanism (recoverability) of this forgetting, and the influence of this analysis on the psychological and neurobiological methods of memory.
Dong et al, 20 16: In this study, the behavioral flexibility of Drosophila melanogaster was measured by reverse learning task, and the effects of functional deletion mutations of homologues of several autism risk genes in Drosophila melanogaster were determined. Mutations in five autism risk genes with different molecular functions all lead to similar behavior inflexibility phenotype, which shows the reversal of learning disabilities. These reverse learning defects are due to the inability to forget, or more precisely, the inability to activate Rac 1(Ras-related C3 botulinum toxin substrate 1) dependent forgetting. Therefore, the behavior-induced activation of Rac 1 dependent forgetting can aggregate autism risk genes.
Epp et al., 20 16: By controlling the level of hippocampal neurogenesis, we found that neurogenesis regulation is a form of active intervention. The increase of hippocampal neurogenesis weakens the existing memory, thus promoting the coding of new and conflicting information in mice. On the contrary, the reduction of neurogenesis stabilizes the existing memory and hinders the coding of new and conflicting information. These results indicate that reducing active interference is an adaptive benefit of neurogenic amnesia.
Shuai et al. 20 10: the initial memory will disappear quickly if it is not consolidated. This memory decline is considered to be due to the inherent instability of newly acquired memory or the interference of subsequent information. This paper reports the role of Rac-dependent forgetting mechanism of G protein in Drosophila in passive memory decline and interfering forgetting. The inhibition of Rac activity leads to the delay of early memory decline, from a few hours to more than one day, which blocks the forgetting caused by interference. On the contrary, the increase of Rac activity in mushroom somatic neurons will accelerate memory decline. This forgetting mechanism does not affect the acquisition of memory and does not depend on the memory formation mechanism mediated by adenylate cyclase in turnip cabbage. Endogenous Rac activation is induced at different time scales, memory is gradually lost in passive decline, and acute memory disappears in reverse learning. We believe that the role of Rac in actin cytoskeleton remodeling may be related to memory loss.
Burghart et al. (20 12): Hippocampus participates in memory separation, which is a neural process that uses pattern separation and allows cognitive flexibility. We evaluated the role of adult hippocampal neurogenesis in cognitive flexibility by using variants of active avoidance task and two independent methods, namely, excision of adult-born neurons, local X-ray irradiation in hippocampus and gene ablation of glial fibrillary acidic protein positive neural precursor cells. The results show that when adult neurogenesis needs to change the learning response to stimulus-induced memory, it is helpful to cognitive flexibility.
Garthe et al. (2009): Although great progress has been made in the past few years, the specific contribution of newborn granulosa cells to adult hippocampal function is still unclear. We assume that in order to solve this problem, we must pay special attention to the specific design, analysis and interpretation of learning tests. Therefore, we designed a behavioral experiment to predict that new neurons may be particularly related to learning conditions, in which new aspects appear in familiar situations, which puts high demands on the quality of (re-) learning in the reference memory version of the water maze. The task inhibition of temozolomide (TMZ) on adult neurogenesis leads to highly specific learning disabilities. Mice were tested in Morris water maze with hidden platform (6 times a day for 5 days, and the platform position was reversed on the fourth day). The test was conducted four weeks after the end of four treatment cycles to minimize the number of new neurons that could be recruited during the test. The decrease of neurogenesis did not change the long-term enhancement of CA3 and dentate gyrus, but eliminated the LTP in dentate gyrus belonging to newborn neurons. TMZ has no obvious side effects in the test, and both the treatment group and the control group have learned to find a hidden platform. However, the qualitative analysis of the search strategy shows that the mice in the treatment group did not move forward to the spatial accurate search strategy, especially when learning to change the target position (inversion). Therefore, it seems that new neurons in dentate gyrus are necessary to increase the flexibility of hippocampus-dependent learning quality parameters. We found that the lack of adult granulosa cells, in particular, led to the inability of animals to accurately locate hidden targets, which was also related to the special role of dentate gyrus in generating measurement results rather than just environmental structure maps. Because adult hippocampal neurogenesis has been inhibited and highly specific behavioral defects have been found, cellular hippocampal plasticity can be associated with clearly defined assumptions in the theoretical model.
Garthe et al. (20 16): We have proved here that living in a stimulating environment (ENR) can improve some key indexes of water maze learning, which have been proved to depend on adult hippocampal neurogenesis in previous functional loss experiments. By analyzing the strategy of mice looking for hidden platforms in water maze, it is found that ENR promotes the acquisition of tasks by increasing the probability of using effective search strategies. When the escape platform moves to a new position, ENR also enhances the animal's behavioral flexibility. Temozolomide can reduce adult neurogenesis, and it can eliminate the influence of ENR on acquisition and flexibility without affecting other aspects of water maze learning. These characteristic effects and interdependencies were not found in the parallel experiment of the second neurogenic behavioral stimulus-random wheel rotation. Because the histological evaluation of adult neurogenesis must be the end point measurement, we can only infer the level of neurogenesis during the whole experiment. The end point of this study is behavior parameters. Although the relationship between physical activity and precursor cell proliferation, learning and survival of new neurons has been well established, the relationship between the specific functional effects described here and the dynamic changes of stem cell niche remains to be solved. However, our research results support the hypothesis that adult neurogenesis is a key mechanism and the basis for leading an active life and enriching experience.
Luu et al (20 12: Adult neurogenesis in dentate gyrus of hippocampus plays an important role in learning and memory. However, the exact contribution of new neurons to hippocampal function is still controversial. New evidence shows that neurogenesis is important for pattern separation and interference reduction when similar projects must be studied at different times. In this study, we use the recently developed olfactory memory task with these specific characteristics to directly test this prediction. In this task, rats learn two lists of highly interfering odor pairs one after another in the same or different environments. Consistent with our hypothesis, focal cranial irradiation leads to the decrease of dentate gyrus nerve selectivity, which significantly weakens the ability to overcome interference when learning the second list. The ability to learn a single odor list is not affected. We also found that radiation had no effect on learning in hippocampus-dependent spatial alternating tasks. Although both tasks involve learning interference, the time process of learning interference projects is different. Learning the odor list of interference is carried out sequentially in several sessions, while learning the spatial position of interference is carried out simultaneously in each session. Therefore, the gradual increase of new neurons may provide a pattern separation mechanism for olfactory tasks rather than maze tasks. These findings prove the role of neurogenesis in solving interference, and they are consistent with the model, indicating the key role of neurogenesis in pattern separation.
Winocur et al. (20 12): Under the condition of high interference or low interference, rats were given low dose radiation to inhibit hippocampal neurogenesis or sham treatment. Half of the mice are engaged in running activities, while the other half are not. In non-runners, irradiation has no effect on learning, and there is no memory discrimination reaction under low interference conditions, but irradiation treatment increases their susceptibility to interference, resulting in the loss of memory of previous learning discrimination. The irradiated rats who participated in running activities showed enhanced nerve growth and protection against memory damage. The results show that adult hippocampal cells play an important role in distinguishing conflict memory from situation-dependent memory, which further proves the importance of neurogenesis in hippocampal sensitive memory tasks. This result is consistent with the calculation model of hippocampal function, which illustrates the central role of neurogenesis in regulating interference effects in learning and memory.