Abstract:
Aimed at the complex structure and parameter redundancy of recurrent neural networks such as LSTM, the structure of recurrent neural networks is studied and analyzed. In order to improve the structural rationality of the recurrent neural network and reduce the amount of calculation of network parameters, a weight activity evaluation algorithm is proposed that evaluates the activity of the basic unit of the network and screens its structure. Through experiments and tests on arrhythmia data, the differences in the weight activity of the LSTM network and the change characteristics of weights and gradients were analyzed. The experimental results show that this algorithm can better optimize the structure of the recurrent neural network and reduce the redundancy of network parameters.