| Activation Functions z (2008) | |||||||||||||||
Abstract | |||||||||||||||
| We deal with computational issues of loading a xed-architecture neural network with a set of positive and negative examples. This is the rst result on the hardness of loading a simple 3-node architecture which do not consist of the binary-threshold neurons, but rather utilize a particular continuous activation function, commonly used in the neural network literature. We observe that the loading problem is polynomial-time if the input dimension is constant. Otherwise, however, any possible learning algorithm based on particular xed architectures faces severe computational barriers. Similar theorems have already been proved by Megiddo and by Blum and Rivest, to the case of binary-threshold networks only. Our theoretical results lend further suggestion to the use of incremental (architecture-changing) techniques for training networks rather than xed architectures. Furthermore, they imply hardness of learnability intheprobably-approximately-correct sense as well. 1 | |||||||||||||||
Publication details | |||||||||||||||
| |||||||||||||||