A novel type of activation function in artificial neural networks: Trained activation function
Yükleniyor...
Tarih
2018-03
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Elsevier
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Attribution-NonCommercial-ShareAlike 3.0 United States
Attribution-NonCommercial-ShareAlike 3.0 United States
Özet
Determining optimal activation function in artificial neural networks is an important issue because it is directly linked with obtained success rates. But, unfortunately, there is not any way to determine them analytically, optimal activation function is generally determined by trials or tuning. This paper addresses, a simpler and a more effective approach to determine optimal activation function. In this approach, which can be called as trained activation function, an activation function was trained for each particular neuron by linear regression. This training process was done based on the training dataset, which consists the sums of inputs of each neuron in the hidden layer and desired outputs. By this way, a different activation function was generated for each neuron in the hidden layer. This approach was employed in random weight artificial neural network (RWN) and validated by 50 benchmark datasets. Achieved success rates by RWN that used trained activation functions were higher than obtained success rates by RWN that used traditional activation functions. Obtained results showed that proposed approach is a successful, simple and an effective way to determine optimal activation function instead of trials or tuning in both randomized single and multilayer ANNs.
Açıklama
Anahtar Kelimeler
Activation Function, Artificial Neural Network, Random Weight Artificial Neural Network, Trained Activation Function
Kaynak
WoS Q Değeri
Q1
Scopus Q Değeri
Q1
Cilt
99
Sayı
Künye
Ertuğrul, Ö F. (2018). A novel type of activation function in artificial neural networks: Trained activation function. Neural Networks, 99, pp. 148-157. https://doi.org/10.1016/j.neunet.2018.01.007
