Publication View

Backpropagation can give rise to spurious local minima even for networks without hidden layers (1989)

Abstract

We give an example of a neural net without hidden layers and with a sigmoid transfer function, together with a training set of binary vectors, for which the sum of the squared errors, regarded as a function of the weights, has a local minimum which is not a global minimum. The example consists of a set of 125 training instances, with four weights and a threshold to be learnt. We do not know if substantially smaller binary examples exist. 1

Publication details

Download	http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.80.9216
Source	http://www.math.rutgers.edu/~sontag/FTP_DIR/complex_systems.pdf
Contributors	CiteSeerX
Repository	CiteSeerX - Scientific Literature Digital Library and Search Engine (United States)
Type	text
Language	English
Relation	10.1.1.52.795, 10.1.1.62.6437, 10.1.1.50.8957, 10.1.1.41.1536, 10.1.1.51.3409, 10.1.1.47.3534, 10.1.1.50.5279, 10.1.1.16.9031, 10.1.1.49.789, 10.1.1.52.7292, 10.1.1.54.9909, 10.1.1.48.8868, 10.1.1.57.1663, 10.1.1.50.5976, 10.1.1.53.5802, 10.1.1.41.4485, 10.1.1.5.112, 10.1.1.62.6665, 10.1.1.86.2848, 10.1.1.99.1294, 10.1.1.39.6570, 10.1.1.41.3529, 10.1.1.25.6280, 10.1.1.28.1342