Publication View

Backpropagation Can Give Rise To Spurious Local Minima Even For Networks Without Hidden Layers (1989)

Abstract

We give an example of a neural net without hidden layers and with a sigmoid transfer function, together with a training set of binary vectors, for which the sum of the squared errors, regarded as a function of the weights, has a local minimum which is not a global minimum. The example consists of a set of 125 training instances, with four weights and a threshold to be learnt. We do not know if substantially smaller binary examples exist. Sussmann's work was partially supported by NSF grant DMS83-01678-01, and by the CAIP Center, Rutgers University, with funds provided by the New Jersey Commission on Science and Technology and by CAIP's industrial members Sontag's work was partially supported by NSF grant DMS88-03396, by US Air Force grants AFOSR-85-0247 and AFOSR-88-0235, and by the CAIP Center, Rutgers University, with funds provided by the New Jersey Commission on Science and Technology and by CAIP's industrial members. 1 Introduction Backpropagation (bp) is one of the most widely...

Publication details

Download	http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.55.7378
Source	http://www.math.rutgers.edu/~sontag/FTP_DIR/complex_systems.ps.gz
Contributors	CiteSeerX
Repository	CiteSeerX - Scientific Literature Digital Library and Search Engine (United States)
Type	text
Language	English
Relation	10.1.1.52.795, 10.1.1.62.6437, 10.1.1.50.8957, 10.1.1.41.1536, 10.1.1.47.3534, 10.1.1.51.3409, 10.1.1.50.5279, 10.1.1.16.9031, 10.1.1.49.789, 10.1.1.52.7292, 10.1.1.54.9909, 10.1.1.48.8868, 10.1.1.57.1663, 10.1.1.50.5976, 10.1.1.53.5802, 10.1.1.41.4485, 10.1.1.5.112, 10.1.1.62.6665, 10.1.1.86.2848, 10.1.1.99.1294, 10.1.1.39.6570, 10.1.1.41.3529, 10.1.1.25.6280, 10.1.1.28.1342