Which statement best describes shrinkage in regression?

Which statement best describes shrinkage in regression?

• A. An increase in predictive accuracy when the model is applied to new data.
• B. The loss of predictive power when the model was derived from the population rather than the sample.
• C. A reduction in predictive power due to using too many predictors.
• D. An increase in variance of coefficient estimates in small samples.

Answer: (B)

Shrinkage in regression is about how well a model generalizes from the data it was fit on to new data. When you estimate a model from a sample, the coefficients pick up not only the true relationships but also random noise in that sample. As a result, the model’s predictions tend to be less accurate on new data than they are on the data used to fit it—the predictive power “shrinks” outside the original sample. The statement that best captures this idea is that there is a loss of predictive power when the model is derived from the population rather than the sample. In other words, using the sample to build the model introduces sampling error that reduces performance on new data, whereas a model derived from the full population would preserve predictive power. The other ideas don’t describe shrinkage as such: improved accuracy on new data would imply better generalization, not shrinkage; having too many predictors points to overfitting as a cause of reduced generalization but isn’t the definition of shrinkage itself; and higher variance of coefficient estimates in small samples describes sampling variability, not the generalization loss across new data.

Shrinkage in regression is about how well a model generalizes from the data it was fit on to new data. When you estimate a model from a sample, the coefficients pick up not only the true relationships but also random noise in that sample. As a result, the model’s predictions tend to be less accurate on new data than they are on the data used to fit it—the predictive power “shrinks” outside the original sample.

The statement that best captures this idea is that there is a loss of predictive power when the model is derived from the population rather than the sample. In other words, using the sample to build the model introduces sampling error that reduces performance on new data, whereas a model derived from the full population would preserve predictive power.

The other ideas don’t describe shrinkage as such: improved accuracy on new data would imply better generalization, not shrinkage; having too many predictors points to overfitting as a cause of reduced generalization but isn’t the definition of shrinkage itself; and higher variance of coefficient estimates in small samples describes sampling variability, not the generalization loss across new data.