Isotonic regression #

Isotonic regression is a generalisation of the usual linear regression model where the monotonicity of the observations is strictly enforced onto the learned function.

Therefore, instead of a simple linear function being fit using least squares, a piecewise linear function is fit by solving the quadratic problem \[ \min\sum_i\omega_i(y_i-\hat y_i)^2\quad\mbox{s.t.}\quad \hat y_i\leq \hat y_j\quad\mbox{whenever}\quad x_i\leq x_j. \]

It has fewer assumptions than a simple linear regression model: in particular in does not assume linearity and can be useful when little is known about the relationship between \(x\) and \(y\) other than the fact that it should be monotonic.

  • Implementation in scikit-learn.
  • Wikipedia.
  • An example plot showing the main idea.