Hello! I know you know that today I’m going to talk about reservoir computing basically basically because it’s the title of the video but I want to start with something a little simpler The first thing we are going to do is to understand the difference between the way of thinking of the human mind and a computer. For example, if I ask now what is the square root of 4, you and a computer will answer me that it is 2. However if you asked the question of what is the square root of a very large number or how much is 3 raised to 34 you would have some more difficulty. The computer doesn’t care, it doesn’t matter whether it deals with a small number or a large number: the operations it performs

are the same However, if we were now to ask the question whether a person is happy or sad, what is a very simple task for us, would be extremely complicated for a computer. Which brings us to the first point: computers are capable of performing algorithms efficiently, i.e. tasks developed one after the other. However, they fail when faced with more abstract tasks such as facial recognition. So what is it that makes our brain different and can perform these tasks efficiently? Although we have more and more information about the areas and mechanisms that act in our brain, to give you an idea, already in 1986 we had managed to build a brain map of a worm with 302 neurons and more than 8000 connections, in fact, we know very little about our brain. What we do know is that our brain is constituted by an immense quantity of neurons that when they receive information, they process it and transform it in order to transmit it to other neurons and so on. In other words, the information is not transmitted sequentially but is distributed and processed in parallel in different neurons. Well, this kind of architecture has proven to be efficient

for tasks such as recognition facial or text, among others, then, the

objective would be to construct nodes or artificial neurons that mimic

the neurons of our brain, that is, we want the information to go into

our network of artificial neurons and propagate so then we can get an

output signal. These structures are used today for very demanding tasks

as facial recognition, text recognition,

financial forecasting, recovery of optical signals and a long etcetera. And although there are several techniques that are inspired by this idea, today I will

talk about the one that is called Reservoir Computing. This is the classic architecture

of a reservoir computer. Here we see three layers: one input layer, one main layer

which we call reservoir (which is a network that connects the nodes to each other and

sometimes even with themselves) and an output layer. What makes the reservoir computing different

of other machine learning techniques is that both the connections input-reservoir and the connections within the reservoir are

kept fixed and the only thing that keeps updating are the output connections. I want you to understand the importance of this: unlike

other methods, no time is wasted into improving internal connections, and you do not need to modify or even know the internal structure. In addition,

we can carry out various tasks in parallel, i.e., if we

introduce an object for classifying it, we couldn’t just

update the connections to know its color, but also its shape,

size… However, in order to make this technique work, we need our reservoir to meet certain characteristics that we are going to explain as follows and that they are: projection of the

input into a larger dimension, a non-linear function and also short-term memory. What are these three properties for? To project the

input information in several dimensions, what we need is a

network that has many nodes so that each one of them offers us a different

projection

of the information that we are enterin. And why is this

important? Let’s imagine we have to separate the yellow stars and the

red balls in a linear way. In 2 dimensions it is not possible to separate them with

a single straight line, we would need two. However, when we project our

problem in three dimensions it’s possible to find a plane that separates them. Here the input signal would be the balls red and yellow stars, the transformation the problem into three

dimensions would be done by the reservoir, and

the plane separating the figures would be calculated in the output layer. This action of classifying objects would be analogous to that of classifying the state

one person: happy or sad. Watch out, I want you to realize that this is a very

simple model that helps us to understand the importance of high

dimensionality but when we we work with reservoir computing,

we go to very high dimensions, in a way that the input information is

separated by hyper planes. Although this feature is also very important,

we also need a non-linear function, why? Very simple, if we have several functions,

the sum of all of them could be understood as another linear transformation. However, we want to be able to access more

complex behaviors. Finally, we need our reservoir to have short-term memory. An example where this is evident is in the predictive keyboard of

our mobile: when we talk by whatsapp, after writing “how” and “are”

our keyboard will offer “you” as a prediction. This is important, predictions do not depend

on what we’ve said hours before but on the current context of the conversation. To do this, we need the information to stay within our network and this will be

achieved by means of connections between nodes or with themselves. Then, to sum it up,

we need our reservoir to have three characteristics: the

first, that it has many nodes to generate high dimensionality, the

second, it has to have a non-linear function in order to be able to access more

complex problems, and the third, it has to have short-term memory, and this is achieved with

connections among the nodes or connecting nodes with themselves. In our IFISC group we have seen how to simplify in a drastic way the traditional

architecture

Reservoir Computing To do this, we have only one non-linear

node and copies of this node distributed through a loop.

In such a way, we meet the three characteristics that I have just mentioned

previously. This simplification has allowed the construction of simple

and very diverse experimental systems since there is no imposition

on the non-linear function that is used. In fact, almost any

non-linear function is equally useful. There are implementations in photonic, electronic,

optoelectronic or mechanical devices that have proven to be efficient

in tasks such as time-series prediction or the recovery of optical signals. How far can you go with this technology? The truth is that I

don’t have an answer to this question. However, for those of you who are

interested, I have posted in the description of the video several links to know

the maths behind the method, the different experimental implementations,

or even the various perspectives that can be taken in order

to imitate the behavior of the neurons in the brain. That’s all, I hope

that with this you’ve got an idea of what is reservoir computing and

when it can be used.

## 3 Comments

## Ingo Fischer

Gracias, Irene! Muy bien!

## Alejandro Alvarez

Excelente 🙂

## daniel gomez alonso

Leilo les tengo leilos