| Polynomial Bounds for VC Dimension of Sigmoidal Neural Networks (1995) | |||||||||||||||
Abstract | |||||||||||||||
| . We introduce a new method for proving explicit upper bounds on the VC Dimension of general functional basis networks, and prove as an application, for the first time, the VC Dimension of analog neural networks with the sigmoid activation function oe(y) = 1=1+ e \Gammay to be bounded by a quadratic polynomial in the number of programmable parameters. 0 Introduction The most commonly used activation function in various neural networks applications is the sigmoid oe(y) = 1=1 + e \Gammay (cf. [HKP91]). We refer to [AB92], [M93], and [MS93] for all the necessary background on the computation by neural networks and the VC dimension (particularly, to the connection between their computational power, and the sample complexity). In Maass's 1993 lecture notes [M93], Open Problem 10 (see also [GJ93] and [MS93]) asks: Is the VC-dimension of analog neural nets with the sigmoid activation function oe(y) = 1=1 + e \Gammay bounded by a polynomial in the number of programmable parameters ? (I... | |||||||||||||||
Publication details | |||||||||||||||
| |||||||||||||||