¿Qué es el Reservoir Computing?
Articles,  Blog

¿Qué es el Reservoir Computing?


Hello! I know you know that today I’m going to talk about reservoir computing basically basically because it’s the title of the video but I want to start with something a little simpler The first thing we are going to do is to understand the difference between the way of thinking of the human mind and a computer. For example, if I ask now what is the square root of 4, you and a computer will answer me that it is 2. However if you asked the question of what is the square root of a very large number or how much is 3 raised to 34 you would have some more difficulty. The computer doesn’t care, it doesn’t matter whether it deals with a small number or a large number: the operations it performs
are the same However, if we were now to ask the question whether a person is happy or sad, what is a very simple task for us, would be extremely complicated for a computer. Which brings us to the first point: computers are capable of performing algorithms efficiently, i.e. tasks developed one after the other. However, they fail when faced with more abstract tasks such as facial recognition. So what is it that makes our brain different and can perform these tasks efficiently? Although we have more and more information about the areas and mechanisms that act in our brain, to give you an idea, already in 1986 we had managed to build a brain map of a worm with 302 neurons and more than 8000 connections, in fact, we know very little about our brain. What we do know is that our brain is constituted by an immense quantity of neurons that when they receive information, they process it and transform it in order to transmit it to other neurons and so on. In other words, the information is not transmitted sequentially but is distributed and processed in parallel in different neurons. Well, this kind of architecture has proven to be efficient
for tasks such as recognition facial or text, among others, then, the
objective would be to construct nodes or artificial neurons that mimic
the neurons of our brain, that is, we want the information to go into
our network of artificial neurons and propagate so then we can get an
output signal. These structures are used today for very demanding tasks
as facial recognition, text recognition,
financial forecasting, recovery of optical signals and a long etcetera. And although there are several techniques that are inspired by this idea, today I will
talk about the one that is called Reservoir Computing. This is the classic architecture
of a reservoir computer. Here we see three layers: one input layer, one main layer
which we call reservoir (which is a network that connects the nodes to each other and
sometimes even with themselves) and an output layer. What makes the reservoir computing different
of other machine learning techniques is that both the connections input-reservoir and the connections within the reservoir are
kept fixed and the only thing that keeps updating are the output connections. I want you to understand the importance of this: unlike
other methods, no time is wasted into improving internal connections, and you do not need to modify or even know the internal structure. In addition,
we can carry out various tasks in parallel, i.e., if we
introduce an object for classifying it, we couldn’t just
update the connections to know its color, but also its shape,
size… However, in order to make this technique work, we need our reservoir to meet certain characteristics that we are going to explain as follows and that they are: projection of the
input into a larger dimension, a non-linear function and also short-term memory. What are these three properties for? To project the
input information in several dimensions, what we need is a
network that has many nodes so that each one of them offers us a different
projection
of the information that we are enterin. And why is this
important? Let’s imagine we have to separate the yellow stars and the
red balls in a linear way. In 2 dimensions it is not possible to separate them with
a single straight line, we would need two. However, when we project our
problem in three dimensions it’s possible to find a plane that separates them. Here the input signal would be the balls red and yellow stars, the transformation the problem into three
dimensions would be done by the reservoir, and
the plane separating the figures would be calculated in the output layer. This action of classifying objects would be analogous to that of classifying the state
one person: happy or sad. Watch out, I want you to realize that this is a very
simple model that helps us to understand the importance of high
dimensionality but when we we work with reservoir computing,
we go to very high dimensions, in a way that the input information is
separated by hyper planes. Although this feature is also very important,
we also need a non-linear function, why? Very simple, if we have several functions,
the sum of all of them could be understood as another linear transformation. However, we want to be able to access more
complex behaviors. Finally, we need our reservoir to have short-term memory. An example where this is evident is in the predictive keyboard of
our mobile: when we talk by whatsapp, after writing “how” and “are”
our keyboard will offer “you” as a prediction. This is important, predictions do not depend
on what we’ve said hours before but on the current context of the conversation. To do this, we need the information to stay within our network and this will be
achieved by means of connections between nodes or with themselves. Then, to sum it up,
we need our reservoir to have three characteristics: the
first, that it has many nodes to generate high dimensionality, the
second, it has to have a non-linear function in order to be able to access more
complex problems, and the third, it has to have short-term memory, and this is achieved with
connections among the nodes or connecting nodes with themselves. In our IFISC group we have seen how to simplify in a drastic way the traditional
architecture
Reservoir Computing To do this, we have only one non-linear
node and copies of this node distributed through a loop.
In such a way, we meet the three characteristics that I have just mentioned
previously. This simplification has allowed the construction of simple
and very diverse experimental systems since there is no imposition
on the non-linear function that is used. In fact, almost any
non-linear function is equally useful. There are implementations in photonic, electronic,
optoelectronic or mechanical devices that have proven to be efficient
in tasks such as time-series prediction or the recovery of optical signals. How far can you go with this technology? The truth is that I
don’t have an answer to this question. However, for those of you who are
interested, I have posted in the description of the video several links to know
the maths behind the method, the different experimental implementations,
or even the various perspectives that can be taken in order
to imitate the behavior of the neurons in the brain. That’s all, I hope
that with this you’ve got an idea of what is reservoir computing and
when it can be used.

3 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *