### Abstract

We consider a linear regression game in which the covariates are known in advance: at each round, the learner predicts a real-value, the adversary reveals a label, and the learner incurs a squared error loss. The aim is to minimize the regret with respect to linear predictions. For a variety of constraints on the adversary's labels, we show that the minimax optimal strategy is linear, with a parameter choice that is reminiscent of ordinary least squares (and as easy to compute). The predictions depend on all covariates, past and future, with a particular weighting assigned to future covariates corresponding to the role that they play in the minimax regret. We study two families of label sequences: box constraints (under a covariate compatibility condition), and a weighted 2- norm constraint that emerges naturally from the analysis. The strategy is adaptive in the sense that it requires no knowledge of the constraint set. We obtain an explicit expression for the minimax regret for these games. For the case of uniform box constraints, we show that, with worst case covariate sequences, the regret is O(d log T), with no dependence on the scaling of the covariates.

Original language | English |
---|---|

Journal | Journal of Machine Learning Research |

Volume | 40 |

Issue number | 2015 |

Publication status | Published - Jan 1 2015 |

Event | 28th Conference on Learning Theory, COLT 2015 - Paris, France Duration: Jul 2 2015 → Jul 6 2015 |

### Fingerprint

### All Science Journal Classification (ASJC) codes

- Software
- Control and Systems Engineering
- Statistics and Probability
- Artificial Intelligence

### Cite this

*Journal of Machine Learning Research*,

*40*(2015).

**Minimax fixed-design linear regression.** / Bartlett, Peter L.; Koolen, Wouter M.; Malek, Alan; Takimoto, Eiji; Warmuth, Manfred K.

Research output: Contribution to journal › Conference article

*Journal of Machine Learning Research*, vol. 40, no. 2015.

}

TY - JOUR

T1 - Minimax fixed-design linear regression

AU - Bartlett, Peter L.

AU - Koolen, Wouter M.

AU - Malek, Alan

AU - Takimoto, Eiji

AU - Warmuth, Manfred K.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - We consider a linear regression game in which the covariates are known in advance: at each round, the learner predicts a real-value, the adversary reveals a label, and the learner incurs a squared error loss. The aim is to minimize the regret with respect to linear predictions. For a variety of constraints on the adversary's labels, we show that the minimax optimal strategy is linear, with a parameter choice that is reminiscent of ordinary least squares (and as easy to compute). The predictions depend on all covariates, past and future, with a particular weighting assigned to future covariates corresponding to the role that they play in the minimax regret. We study two families of label sequences: box constraints (under a covariate compatibility condition), and a weighted 2- norm constraint that emerges naturally from the analysis. The strategy is adaptive in the sense that it requires no knowledge of the constraint set. We obtain an explicit expression for the minimax regret for these games. For the case of uniform box constraints, we show that, with worst case covariate sequences, the regret is O(d log T), with no dependence on the scaling of the covariates.

AB - We consider a linear regression game in which the covariates are known in advance: at each round, the learner predicts a real-value, the adversary reveals a label, and the learner incurs a squared error loss. The aim is to minimize the regret with respect to linear predictions. For a variety of constraints on the adversary's labels, we show that the minimax optimal strategy is linear, with a parameter choice that is reminiscent of ordinary least squares (and as easy to compute). The predictions depend on all covariates, past and future, with a particular weighting assigned to future covariates corresponding to the role that they play in the minimax regret. We study two families of label sequences: box constraints (under a covariate compatibility condition), and a weighted 2- norm constraint that emerges naturally from the analysis. The strategy is adaptive in the sense that it requires no knowledge of the constraint set. We obtain an explicit expression for the minimax regret for these games. For the case of uniform box constraints, we show that, with worst case covariate sequences, the regret is O(d log T), with no dependence on the scaling of the covariates.

UR - http://www.scopus.com/inward/record.url?scp=84984693648&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84984693648&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84984693648

VL - 40

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

IS - 2015

ER -