9 Improved Temporal Difference Methods with Linear Function Approximation (2009)
Dimitri P. Bertsekas, Angelia Nedich, Vivek S. Borkar
Editor’s Summary: This chapter considers temporal difference algorithms within the context of infinite-horizon finite-state dynamic programming problems with discounted cost and linear cost...
A Power Optimal Scheduling Algorithm on a Point to Point Wireless Link (2008)
Nitin Salodkar, Abhijeet Bhorkar, Abhay Kar, Vivek S. Borkar
Abstract. In this paper we consider the problem of transmitting packets over a point to point wireless link where the objective is to minimize the average transmitted power subject to a constraint on...
A Learning Algorithm for Risk-Sensitive Cost (2008)
Basu, Arnab, Bhattacharyya, Tirthankar, Borkar, Vivek S
A linear function approximation-based reinforcement learning algorithm is proposed for Markov decision processes with infinite horizon risk-sensitive cost. Its convergence is proved using the "o.d.e....
Vivek S. Borkar, Govindan Rangarajan
Abstract. Many complex adaptive systems contain a large diversity of specialized components. The specialization at the level of the microscopic degrees of freedom, and diversity at the level of the...
Mathematical Programming Embeddings of Logic (2008)
Vivek S Borkar, Vijay Chandru, Sanjoy K Mitter
Abstract. Can theorem proving in mathematical logic be addressed by classical mathematical techniques like the calculus of variations? The answer is surprisingly in the affirmative and this approach...
Capacity of Markov Channels: A Stochastic Control Approach (2008)
Vivek S. Borkar, Arzad A. Kherani, Vinod Sharma
The problem of maximizing the input-output mutual information rate for a Markov channel is cast as a problem of controlling a partially observed controlled Markov chain on a suitable state space with...
Optimal Random Access in Networks with Two-Way Traffic (2008)
Eitan Altman, Vivek S. Borkar, Arzad A. Kherani, Eitan Altman Ý
We consider a random access network in which the nodes need to optimize their channel access rates. The nodes are assumed to be rational and interested in their performance seen as a transmitter as...
Learning Decentralized Goal-based Vector Quantization (2007)
In this paper we consider the following generalization of classical vector quantization: to (vector) quantize or, equivalently, to partition the domain of a given function such that each cell in the...
Randomized Neural Networks for Learning Stochastic Dependences (2007)
We consider the problem of learning the dependence of one random variable on another, from a finite string of i.i.d. copies of the pair. The problem is first converted to that of learning a function...
Power Efficient Scheduling under Delay Constraints over Multi-user Wireless Channels (2007)
Salodkar, Nitin, Karandikar, Abhay, Borkar, Vivek S.
In this paper, we consider the problem of power efficient uplink scheduling in a Time Division Multiple Access (TDMA) system over a fading wireless channel. The objective is to minimize the power...
Lattice Approximation in the Stochastic Quantization of (04)2 Fields (2007)
Borkar, Vivek S., Mitter, Sanjoy K.
The Parisi-Wu program of stochastic quantization [8] involves construction of a stochastic process which has a prescribed Euclidean quantum field measure as its invariant measure. This program was...
A Simulation-Based Algorithm for Ergodic Control of Markov Chains Conditioned on Rare Events (2006)
Bhatnagar, Shalabh, Borkar, Vivek S, Akarapu, Madhukar
We study the problem of long-run average cost control of Markov chains conditioned on a rare event. In a related recent work, a simulation based algorithm for estimating performance measures...
Discrete-Time Controlled Markov Processes With Average Cost Criterion: A Survey (2006)
Arapostathis, Aristotle, Borkar, Vivek S., Fernandez-Gaucherand, Emmanuel, Ghosh, Mrinal K., Marcus, Steven I.
This work is a survey of the average cost control problem for discrete time Markov processes. We have attempted to put together a comprehensive account of the considerable research on this problem...
A Simulation-Based Algorithm for Ergodic Control of Markov Chains Conditioned on Rare Events (2006)
Bhatnagar, Shalabh, Borkar, Vivek S, Akarapu, Madhukar
We study the problem of long-run average cost control of Markov chains conditioned on a rare event. In a related recent work, a simulation based algorithm for estimating performance measures...
Power Optimal Opportunistic Scheduling in Fading Wireless Channel (2006)
Abhijeet Bhorkar, Abhay Kar, Vivek S. Borkar
1 In this paper, we propose a power optimal opportunistic scheduling scheme for a multiuser single hop Time Division Multiple Access (TDMA) system. We formulate the problem of minimizing average...
A Simulation-Based Algorithm for Ergodic Control of Markov Chains Conditioned on Rare Events (2006)
Shalabh Bhatnagar, Vivek S. Borkar, Madhukar Akarapu, Shie Mannor
We study the problem of long-run average cost control of Markov chains conditioned on a rare event. In a related recent work, a simulation based algorithm for estimating performance measures...
Power Optimal Opportunistic Scheduling in Fading Wireless Channel (2006)
Abhijeet Bhorkar, Abhay Kar, Vivek S. Borkar
1 In this paper, we propose a power optimal opportunistic scheduling scheme for a multiuser sin-gle hop Time Division Multiple Access (TDMA) system. We formulate the problem of minimizing average...
Controlled diffusion processes (2005)
This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic...
Controlled diffusion processes (2005)
This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic...
On De Finetti Coherence and Kolmogorov Probability (2004)
Vivek S. Borkar, Vijay R. Konda, Goldman Sachs, Sanjoy K. Mitter
This article addresses the problem of existence of a countably additive probability measure in the sense of Kolmogorov that is consistent with a probability assignment to a family of sets which is...
Multiscale Chaotic SPSA and Smoothed Functional Algorithms for Simulation Optimization (2003)
Shalabh, Bhatnagar, Borkar, Vivek S
The authors propose a two-timescale version of the one-simulation smoothed functional (SF) algorithm with extra averaging. They also propose the use of a chaotic simple deterministic iterative...
Ergodic Control of Partially Degenerate Diffusions in a Compact Domain (2003)
Borkar, Vivek S, Ghosh, Mrinal K
The problem of ergodic control of a reflecting diffusion in a compact domain is analysed under the condition of partial degeneracy, i.e. when its transition kernel after some time is absolutely...
Multiscale Chaotic SPSA and Smoothed Functional Algorithms for Simulation Optimization (2003)
Shalabh, Bhatnagar, Borkar, Vivek S.
The authors propose a two-timescale version of the one-simulation smoothed functional (SF) algorithm with extra averaging. They also propose the use of a chaotic simple deterministic iterative...
Ergodic Control of Partially Degenerate Diffusions in a Compact Domain (2003)
Borkar, Vivek S, Ghosh, Mrinal K
The problem of ergodic control of a reflecting diffusion in a compact domain is analysed under the condition of partial degeneracy, i.e. when its transition kernel after some time is absolutely...
Shalabh Bhatnagar, Vivek S. Borkar, Shalabh Bhatnagar, Vivek S. Borkar
On behalf of:
Shalabh Bhatnagar, Vivek S. Borkar
The authors propose a two-timescale version of the one-simulation smoothed functional (SF) algorithm with extra averaging. They also propose the use of a chaotic simple deterministic iterative...
Mathematical Programming Embeddings of Logic (2002)
Borkar, Vivek S, Chandru, Vijay, Mitter, Sanjoy K
Can theorem proving in mathematical logic be addressed by classical mathematical techniques like the calculus of variations? The answer is surprisingly in the affirmative, and this approach has...
Mathematical Programming Embeddings of Logic (2002)
Borkar, Vivek S, Chandru, Vijay, Mitter, Sanjoy K
Can theorem proving in mathematical logic be addressed by classical mathematical techniques like the calculus of variations? The answer is surprisingly in the affirmative, and this approach has...
Parijat Dube, Vivek S. Borkar, D. Manjunath
Abstract — We consider a system of identical parallel queues served by a single server and distinguished only by the price charged at entry. A Poisson stream of customers joins the queue by a...
Parijat Dube, Vivek S. Borkar, D. Manjunath
We consider a system of identical parallel queues served by a single server and distinguished only by the price charged at entry. A Poisson stream of customers joins the queue by a greedy policy that...
Differential Join Prices for Parallel Queues: Social (2002)
Optimality Dynamic Pricing, Parijat Dube, Vivek S. Borkar, D. Manjunath
We consider a system of identical parallel queues served by a single server and distinguished only by the price charged at entry. A Poisson stream of customers joins the queue by a greedy policy that...
Optimal sequential vector quantization of Markov sources (2001)
Vivek S. Borkar, Sanjoy K. Mitter, Sekhar Tatikonda
Abstract. The problem of sequential vector quantization of a stationary Markov source is cast as an equivalent stochastic control problem with partial observations. This problem is analyzed using the...
Randomized Neural Networks for Learning Stochastic Dependences (1999)
Borkar, Vivek S, Gupta, Piyush
We consider the problem of learning the dependence of one random variable on another, from a finite string of independently identically distributed (i.i.d.) copies of the pair. The problem is first...
An Analog Scheme for Fixed-Point Computation - Part II: Applications (1999)
Soumyanath, K, Borkar, Vivek S
In a companion paper [6] we presented theoretical analysis of an analog network for fixed-point computation. This paper applies these results to several applications from numerical analysis and...
Evolutionary games with two timescales (1999)
Borkar, Vivek S, Jain, Sanjay, Rangarajan, Govindan
We consider a two timescale model of learning by economic agents wherein active or 'ontogenetic' learning by individuals takes place on a fast scale and passive or 'phylogenetic' learning by society...
Randomized Neural Networks for Learning Stochastic Dependences (1999)
Borkar, Vivek S, Gupta, Piyush
We consider the problem of learning the dependence of one random variable on another, from a finite string of independently identically distributed (i.i.d.) copies of the pair. The problem is first...
An Analog Scheme for Fixed-Point Computation - Part II: Applications (1999)
Soumyanath, K, Borkar, Vivek S
In a companion paper [6] we presented theoretical analysis of an analog network for fixed-point computation. This paper applies these results to several applications from numerical analysis and...
Ergodic control of partially observed Markov chains (1998)
The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear...
Asynchronous stochastic approximations (1998)
The asymptotic behavior of a distributed, asynchronous stochastic approximation scheme is analyzed in terms of a limiting nonautonomous differential equation. The relation between the latter and the...
Borkar, Vivek S., Jain, Sanjay, Rangarajan, Govindan
Many complex adaptive systems contain a large diversity of specialized components. The specialization at the level of the microscopic degrees of freedom, and diversity at the level of the system as a...
Dynamics of Individual Specialization and Global Diversification in Communities (1998)
Borkar, Vivek S., Jain, Sanjay, Rangarajan, Govindan
We discuss a model of an economic community consisting of $N$ interacting agents. The state of each agent at any time is characterized, in general, by a mixed strategy profile drawn from a space of...
A Unified Framework for Hybrid Control: Model and Optimal Control Theory (1998)
Branicky, Michael S, Borkar, Vivek S, Mitter, Sanjoy K
Complex natural and engineered systems typically possess a hierarchical structure, characterized by continuous-variable dynamics at the lowest level and logical decision-making at the highest,...
Asynchronous stochastic approximations (1998)
The asymptotic behavior of a distributed, asynchronous stochastic approximation scheme is analyzed in terms of a limiting nonautonomous differential equation. The relation between the latter and the...
A Unified Framework for Hybrid Control: Model and Optimal Control Theory (1998)
Branicky, Michael S, Borkar, Vivek S, Mitter, Sanjoy K
Complex natural and engineered systems typically possess a hierarchical structure, characterized by continuous-variable dynamics at the lowest level and logical decision-making at the highest,...
Dynamics of individual specialization and global diversification in communities (1998)
Borkar, Vivek S, Jain, Sanjay, Rangarajan, Govindan
We discuss a model of an economic community consisting of N interacting agents. The state of each agent at any time is characterized, in general, by a mixed strategy profile drawn from a space of s...
Optimal Sequential Vector Quantization of Markov Sources (1998)
Borkar, Vivek S, Mitter, Sanjoy K, Tatikonda, Sekhar C
The problem of optimal sequential vector quantization of Markov sources is cast as a stochastic control problem with partial observations and constraints, leading to useful existence results for...
Dynamics of individual specialization and global diversification in communities (1998)
Borkar, Vivek S, Jain, Sanjay, Rangarajan, Govindan
We discuss a model of an economic community consisting of N interacting agents. The state of each agent at any time is characterized, in general, by a mixed strategy profile drawn from a space of s...
Optimal Sequential Vector Quantization of Markov Sources (1998)
Borkar, Vivek S, Mitter, Sanjoy K, Tatikonda, Sekhar C
The problem of optimal sequential vector quantization of Markov sources is cast as a stochastic control problem with partial observations and constraints, leading to useful existence results for...
A unified framework for hybrid control: Model and optimal control theory (1998)
Michael S. Branicky, Vivek S. Borkar, Senior Member, Sanjoy K. Mitter
Abstract — Complex natural and engineered systems typically possess a hierarchical structure, characterized by continuousvariable dynamics at the lowest level and logical decision-making at the...
A unified framework for hybrid control: Model and optimal control theory (1998)
Michael S. Branicky, Vivek S. Borkar, Senior Member, Sanjoy K. Mitter
Abstract — Complex natural and engineered systems typically possess a hierarchical structure, characterized by continuousvariable dynamics at the lowest level and logical decision-making at the...
Asynchronous Stochastic Approximations (1998)
. The asymptotic behavior of a distributed, asynchronous stochastic approximation scheme is analyzed in terms of a limiting nonautonomous di#erential equation. The relation between the latter and the...
Stochastic approximation with two time scales (1997)
Asymptotic behaviour of a two time scale stochastic approximation algorithm is analysed in terms of a related singular ordinary differential equation.
Stochastic approximation with two time scales (1997)
Asymptotic behaviour of a two time scale stochastic approximation algorithm is analysed in terms of a related singular ordinary differential equation.
Occupation measures for controlled Markov processes: characterization and optimality (1996)
Bhatt, Abhay G., Borkar, Vivek S.
For controlled Markov processes taking values in a Polish space, control problems with ergodic cost, infinite-horizon discounted cost and finite-horizon cost are studied. Each is posed as a convex...
Stochastic Processes that Generate Polygonal and Related Random Fields (1996)
Borkar, Vivek S, Mitter, Sanjoy K
A reversible, ergodic, Markov process taking values in the space of polygonally segmented images is constructed. The stationary distribution of this process can be made to correspond to a Gibbs-type...
Stochastic Processes that Generate Polygonal and Related Random Fields (1996)
Borkar, Vivek S, Mitter, Sanjoy K
A reversible, ergodic, Markov process taking values in the space of polygonally segmented images is constructed. The stationary distribution of this process can be made to correspond to a Gibbs-type...
Mathematical Programming Embeddings of Logic (1996)
Borkar V. S, Chandru V, Micciancio D, Mitter S. K, Vivek S. Borkar, Vijay Chandru, ...
Serious studies on spatial embeddings of logic were initiated by Robert Jeroslow (cf. [16]) a lit-
Ergodic control of Markov Chains with Constraints – The General Case (1994)
The problem of controlling a Markov chain on a countable state space with ergodic or ’long run average’ cost is studied in the presence of additional constraints, requiring finitely many (say, m)...
Ergodic control of Markov Chains with Constraints – The General Case (1994)
The problem of controlling a Markov chain on a countable state space with ergodic or ’long run average’ cost is studied in the presence of additional constraints, requiring finitely many (say, m)...
White-Noise Representations in Stochastic Realization Theory (1993)
It is proved that any random sequence can be exhibited as the output of a stochastic dynamical system driven by white noise. Further refinements are obtained for Markov, stationary, and ergodic...
White-Noise Representations in Stochastic Realization Theory (1993)
It is proved that any random sequence can be exhibited as the output of a stochastic dynamical system driven by white noise. Further refinements are obtained for Markov, stationary, and ergodic...
On Extremal Solutions to Stochastic Control Problems (1991)
We identify two solutions of a controlled diffusion if the corresponding one-dimensional marginals of the state and control process agree.The extreme points of the set of such equivalence classes are...
A Remark on Control of Partially Observed Markov Chains (1991)
A new state variable is introduced for the problem of controlling a Markov chain under partial observations, which, under a suitably altered probability measure, has a simple evolution.
Ergodic and adaptive control of nearest-neighbor motions (1991)
Borkar, Vivek S, Ghosh, Mrinal K
The self-tuning approach to adaptive control is applied to a class of Markov chains called nearest-neighbor motions. These have a countable state space and move from any state to at most finitely...
On Extremal Solutions to Stochastic Control Problems (1991)
We identify two solutions of a controlled diffusion if the corresponding one-dimensional marginals of the state and control process agree.The extreme points of the set of such equivalence classes are...
A Remark on Control of Partially Observed Markov Chains (1991)
A new state variable is introduced for the problem of controlling a Markov chain under partial observations, which, under a suitably altered probability measure, has a simple evolution.
Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey (1991)
Arapostathis, Aristotle, Borkar, Vivek S., Fernandez-Gaucherand, E., Ghosh, M.K., Marcus, S.I.
This work is a survey of the average cost control problem for discrete-time Markov processes. We have attempted to put together a comprehensive account of the considerable research on this problem...
Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey (1991)
Arapostathis, Aristotle, Borkar, Vivek S., Fernandez-Gaucherand, Emmanuel, Ghosh, Mrinal K., Marcus, S.I.
This work is a survey of the average cost control problem for discrete-time Markov processes. We have attempted to put together a comprehensive account of the considerable research on this problem...
Controlled Markov chains with constraints (1990)
We consider the ergodic control of a Markov chain on a countable state space with a compact action space in presence of finitely many (say, m) ergodic constraints. Under a condition on the cost...
Controlled diffusions with constraints (1990)
Borkar, Vivek S, Ghosh, Mrinal K
Controlled diffusion problems with classical cost structures (ergodic, discounted, etc.) are considered when finitely many additional cost functionals of the same type are required to lie in...
Controlled diffusions with constraints (1990)
Borkar, Vivek S, Ghosh, Mrinal K
Controlled diffusion problems with classical cost structures (ergodic, discounted, etc.) are considered when finitely many additional cost functionals of the same type are required to lie in...
Controlled Markov chains with constraints (1990)
We consider the ergodic control of a Markov chain on a countable state space with a compact action space in presence of finitely many (say, m) ergodic constraints. Under a condition on the cost...
Optimal control of diffusion processes (1989)
Abstract: This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic...
A Fresh Look at Markov Decision Processes. (1987)
This paper develops a new framework for the study of Markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced measures on the...
Control of Markov Chains with Long-Run Average Cost Criterion II. (1987)
The long-run average cost control problem for discrete time Markov chains on a countable state space is studied in a very general framework. Necessary and sufficient conditions for optimality in...
A Fresh Look at Markov Decision Processes. (1987)
This paper develops a new framework for the study of Markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced measures on the...
Control of Markov Chains with Long-Run Average Cost Criterion II. (1987)
The long-run average cost control problem for discrete time Markov chains on a countable state space is studied in a very general framework. Necessary and sufficient conditions for optimality in...
A Further Remark On Dynamic Programming For Partially Observed Markov Processes
Vivek S. Borkar, Amarjit Budhiraja
In [5], a pair of dynamic programming inequalities were derived for the `separated' ergodic control problem for partially observed Markov processes, using the `vanishing discount' argument....