In this article, I'll explain what Recurrent Neural Networks (RNNs) are, how they work, and what you can do with them. I will also show an excellent example of music generation using artificial intelligence.
However, before we talk about RNN, we need to explain the concept of data streams.
data streams
As the name implies, data streams are a collection of data in different states over time, so that the succession of data forms something larger. These are some examples:
- All the different notes that make up a song.
- All the words that make up a sentence.
- All the images that make up a video.
Take, for example, all the images that make up the following video clip.
Individually, they can only tell us that they are images of a guy playing golf.
But if we look at all the images in the correct order (from left to right), we can see the swing taking place. This is a “sequence” of data that must be analyzed as a whole if we want to make sense of it and conclude that all these images represent a “swing”.
So what are recurrent neural networks?
If you have been reading my blog, you must be familiar with neural networks by now, if you are not, I recommend read this article first.
Neural networks are a lot of fun and easy to use, but they have a little problem. They can only take a fixed-size input and provide a fixed-size output... In other words, they can't parse streams of data.
If you take the image classification example, the neural network will input a vector (X) of all the pixel values of the image and the output will be a classification vector (Y) that will represent a prediction of what the image is.
Although it may seem incredible, machine learning professionals often refer to these models as “vanilla” neural networks, since they have a “one-to-one” architecture and for them they are “regular” and “no big mystery”. ..
vanilla relationship
Recurrent Neural Networks (RNN), on the other hand, are a more perverted type of artificial neural network in which the result(s) depend(s) on the input(s).
This means that the prediction will take into account all the previous information that has been given to it.
The autocomplete feature on our smartphones is a great example of what a recurrent neural network can do.
Every time you type a word, the software will take all the previous words you have typed to predict which one you are most likely to type next. Giving an RNN to just the first word and letting him predict the rest can be a lot of fun.
You can test that if you change any of the typed words in the middle of a sentence, your phone will propose different words. This is how you know that the function takes into account the entire data stream in the order you provide it and that it's not just giving you random words.
The autocomplete function is an example of a "Many-to-One" RNN, because the software needs to parse many words in the correct order through time to arrive at a conclusion… What will be the next word?
But there are other types of RNN architectures:
- One entrance for many exits
- Many entrances and one exit
- Many entrances and many exits
- Many inputs and many outputs (with different lengths)
I hope the image below gives you a better understanding of these different architectures.
The “NN” cell represents a neural network and can be as deep as you like, even with hundreds of hidden layers.
Some scholars and researchers often represent RNNs with a different diagram, but the concept is exactly the same.
I personally prefer the “unrolled” diagram that I presented first because I think it is easier to understand.
Training an RNN
Training an RNN is like training a vanilla neural network.
- You start by setting the parameters (weights and bias) randomly.
- You do a forward propagation to calculate the cost.
- You do back propagation to find the parameters that minimize the model error.
- You repeat the same process many times until your model is as accurate as possible (smallest error)
If you are new to the world of machine learning and you did not understand anything that he writes before, nor the previous image, do not worry.
The only thing you need to understand is that you have to take into account the results of the previous NN when calculating the result of the current NN and so on until the end.
In other words, the model takes all previous data streams into account, when it trains, but also when it provides you with predictions.
Generating music with RNN
By now, you should understand the basic concept of recurrent neural networks. As you can see, you can train a model with words to predict the next ones or you can train it with Shakespare poems to generate its own poetry.
But I personally find that one of the most amazing examples of RNN is generating music, one note at a time.
If you are new to my blog, please understand that my articles are intended for a non-technical audience, so I will not cover the implementation details of this project, nor will I provide the code I used. There are hundreds of tutorials out there on how to achieve what I did, so if you are interested in doing the same, you can simply google it or contact me for more details.
In this article, I will only provide a general idea of what I did.
- I collected different clips of Jazz music
- I used them to train an RNN
- I gave the RNN a random first note so that it could predict the following notes.
- The model spawned some jazz music!
This is the final result I put on youtube.http://www.youtube.com/embed/V1a3gYwo2mA?modestbranding=0&html5=1&rel=0&autoplay=0&wmode=opaque&loop=0&controls=1&autohide=0&showinfo=0&theme=dark&color=red
The images in the video were also generated with AI using a technique called "neural transfer" which I will definitely cover in a different article.
So that's all for now. There are other interesting concepts behind RNNs such as “memory” and bidirectional RNNs that I will cover in other articles as well. I hope you enjoyed reading about this topic and leave your comments below.
