Neural Turing Machines

https://arxiv.org/abs/1410.5401

TLDR; https://www.youtube.com/watch?v=_H0i0IhEO2g

Essentially an architecture with a controller (FNN, RNN, LSTM) that has read/write access to a memory in order to learn a particular task. The controller gets an input, can read/write to memory block and then output a result.

When reading and writing to the memory matrix, introspective attention is used. Separate attention weights for reading, erasing and writing. All of this is end-to-end differentiable, so theoretically we can learn very complex tasks.

model

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s