Comparing methods for determining best fit
About this video
A faulty meter creates one extreme outlier in wind turbine data, and the video compares three ways of fitting a line of best fit. Least squares regression is the most sensitive to the outlier because it squares distances, median-median regression is very resistant, and least absolute regression sits in between.
You'll be able to
- explain why squaring vertical distances makes least squares regression sensitive to outliers.
- explain why a median-median line is resistant to outliers.
- rank least squares, least absolute and median-median regression by how sensitive they are to an outlier.
Key words
- outlier
- A data value that is far away from the rest of the data.
- least squares regression
- A method that finds the line minimizing the sum of the squared vertical distances from the points to the line.
- median-median regression
- A method that splits the data into three sections, finds the median x and y in each, and fits a line to those medians.
- least absolute regression
- A method that finds the line minimizing the absolute values of the vertical distances from the points to the line.
- resistant
- Describes a statistic or method that barely changes when an extreme value is added.
Transcript
- [Teacher] We're told that for two weeks, Inti records data on the speed in revolutions per minute of a small wind turbine versus the electrical power it produces. One day, a faulty meter reports a power value far greater than the other readings, creating a single extreme outlier.
Which methods for determining a line of best fit are least sensitive to this outlier? It says pick two answers. So pause this video and have a go at that before we do it together. All right, in a least squares regression, you try to find a line that goes through all of the data points that minimizes the squared vertical distance between each of those points and the line.
Now if you have an outlier, something that is very far from that line and you're thinking about the square distance, something far away, when you square it, it's vertical distance is squared, it's really going to pull up. If it's way above the line, it would pull up on it.
If it's way below the line, it would pull down on it. So this one is going to be very sensitive to outliers, at least relative to the other two. If we look at a median-median regression there, you typically will divide the data into three sections and you find the median in both the X and the Y direction, each of those three sections, and you fit a line for those medians.
So median in general, as a statistic is very, or as a parameter, is very resistant to outliers. For example, if I have the three data points, one, three, and five, the median is three. If that one, or if that five became a 5 million, that median is still three.
So this is gonna be very resistant to outliers, least absolute regression. So this is based about fitting a line that minimizes the absolute value, vertical distance between the points and the line. It will be sensitive to the outliers, but not as sensitive as the least squares regression where you're squaring the distance.
Here, you're just taking the absolute value of the distance. So something that's far off it will tug on the regression line, but not as much as least squares regression. So those are my two answers.
Video © Khan Academy, used under CC BY-NC-SA. Watch on Khan Academy. Puzzicle is not affiliated with Khan Academy.