Artificial Neural Networks With to sleep they learn better; We already knew that the brain in animals uses the trick of sleep to organize memory and learned data, why can’t machines do the same?

Each person needs 7 to 13 hours of sleep every 24 hours, depending on their age. During this period, many interactions take place: the continuation of heart rate, breathing and metabolism, and finally, the regulation of hormonal levels; You may think that your whole body is relaxed, but the ruler of your body is the brainstill continues its activity!

Corresponding author Bazhnov and colleagues in the November 18, 2022 issue of PLOS Journal of Computational Biology discuss how biological models can help reduce risk. Catastrophic forgetting They discuss artificial neural networks and their use in a wide range of researches. Co-authors are Ryan Golden and Jean-Eric Delanois, both at UC San Diego. and Pavel Sanda, Institute of Computer Science, Czech Academy of Sciences.

Read more: Lack of sleep can cause blindness

Artificial neural networks learn better by sleeping, but how?

In the background of this research, Bazhnov and his colleagues have reported how sleep promotes logical memory, the ability to remember unconscious or indirect connections between objects, people, or events, and protects against forgetting old memories.

In this regard, Dr. Maxim Bazhnov, professor of medicine and sleep researcher at the University of California San Diego School of Medicine, said:

“The brain is very isolated during sleep and tries to organize what we have learned during the day. In fact, sleep helps organize memories in the most efficient way possible.”

As you may know, artificial neural networks use the architecture of the human brain to improve various technologies and systems, from basic science to medicine to finance and social media.

In some ways, they achieve superhuman performance, such as amazing computing speed, but they fail in one important aspect that a young child is better than: namely, when neural networks learn sequentially, adding new information on top of information. They rewrite the former, which ultimately leads to the phenomenon that Catastrophic forgetting is named.

In contrast, the human brain is constantly learning and accumulating new data on top of existing knowledge, and it usually learns even better when it associates the learned exercises with the sleep cycle! Bazhnov

So why don’t neural networks do that?

Sleep mechanism in artificial neural networks

Scientists used the model of spiking neural networks, which artificially imitates natural nervous systems; In other words, instead of the information flowing continuously, it is transmitted as discrete events (cluster-like) at certain points in time.

The researchers found that catastrophic forgetting was reduced when the spiking networks were trained for a new exercise, accompanied by occasional periods that deviated from the natural training curve. Going out of the natural line means something like sleep in animals.

According to the study authors, sleep allows neural networks like the human brain to retrieve old memories without directly using old training data.

Memories in the human brain with Patterns of weight support (Synaptic weight) – the strength or range of communication between two neurons – are shown. As far as we know in computer science and neurology, synaptic weight is the place where two or more neurons are connected, and their range and power of connection is very strong.

Bazhnov said:

When we learn new information, neurons fire in a certain order, which increases the number of synapses between them and creates a stronger connection. During sleep, the spiking neural network spontaneously repeats the states it acquired during our waking hours, which leads to this process. Reactivation Or they say replay.

Artificial neural networks learn with sleep not to suffer from catastrophic forgetting!

Synaptic plasticity And the capacity to change or mold is still present during sleep and can strengthen the synaptic weight patterns that indicate memory and prevent forgetting or help the process of transferring accumulated knowledge from old tasks to new tasks.

When Bazhnov and colleagues applied this approach to artificial neural networks, they observed that networks With this method, no longer suffering Catastrophic forgetting They do not

This means that these networks can learn continuously like humans or animals, and as a result do not suffer from “catastrophic forgetting”!

Also, understanding how the human brain processes information during sleep can be used to strengthen people’s memory; In other words, strengthening the sleep rhythm probably leads to better memory.

In other projects, we use computer models to develop optimal strategies for applying stimulation during sleep, such as auditory sounds, that increase sleep rhythms and improve learning.

Read more:

Your opinion about What is improved sleep learning in artificial neural networks? Please leave your comments in the comments section duplicate Share and Technology News Follow with us.

Source:ScienceDaily