LSTM unit and GRU unit are the most common units in RNN model, and their contents are composed of three structures: input gate, forgetting gate and output gate.
The functions of LSTM and GRU units are almost the same, but the only difference is that:
In contrast, it is easier to use GRU units.
Quasi-recursive neural network unit is the basic unit of RNN model, which is faster than LSTM unit.
QRNN units were published on 20 16. It replaces the traditional circular structure with convolution operation, and its network structure is between RNN and CNN.
The convolution structure inside QRNN can calculate the sequence data in matrix mode at the same time, and it is no longer necessary to calculate the sequence data in sequence order like the circular structure. It uses parallel operation mode instead of serial operation, which improves the operation speed. In training, convolution structure is also more stable than circular structure.
In practical application, QRNN units can be arbitrarily replaced by existing units in RNN model.
For more information, please refer to the paper:
Quasi-recursive neural network
SRU element is the basic element of RNN model. Its function is similar to QRNN element, and it also improves the speed of LSTM element.
In order to output the results, the LSTM unit must manipulate the samples one by one in order. This operation mode makes the unit unable to play its greatest role in the environment of multi-computer parallel computing.
SRU unit was published on 20 17. It maintains the cyclic structure of LSTM cells, and improves the operation speed by adjusting the operation order (putting matrix multiplication outside the serial cycle and adding multiplication in the serial cycle).
If you need to study the deeper theory of SRU elements, you can refer to the following papers:
Simple Recursive Unit with Highly Parallel Recursion
For more usage of the function tf.contrib.rnn.SRUCell, please refer to the official help document.
/tensor flow/tensor flow/blob/r 1. 15/tensor flow/contrib/rnn/python/ops/rnn _ cell . py # l 2738-l 28 16