We often use AIC to discern the best model among candidates.
Now suppose we have two (non-parametric) models, which use mass points and weights to model a random variable:
- model A uses 4 mass points to model a random variable (i.e. the height of men in Singapore)
- model B uses 5 mass points to mode the same random variable
We consider model A to be nested in model B. This is because model A is basically a version of model B, where one mass point is “de-activated”.
Thus, we must not use small differences in AIC or BIC alone to judge between these models. If the model with a constraint on one or more parameters (model A) is regarded as nested on within the model without the constraint (model B) , a chi-square difference test, or Likelihood Ratio (LR) test, is performed to test the reasonableness of the constraint, using a central chi-square with degrees of freedom equal to the number of parameters constraints.
However, under the null hypothesis, the parameter of interest takes its value on the boundary of the parameter space (next post). For this reason, the asymptotic distribution of the chi-square difference, or Likelihood Ratio (LR) statistic, is not that of a central chi-square distributed random variable with one degree of freedom. This boundary problem affects goodness of fit measures like AIC and BIC4. As a result, the AIC and BIC should be used heuristically, in conjunction with graphs and other criteria to evaluate estimates from the chosen model.
One thought on “Limits of Akaike Information Criteria (AIC)”
Hey there exceptional website! Does running a blog similar to this require a large amount
of work? I have absolutely no expertise in coding but I was hoping to start my own blog in the near future.
Anyway, should you have any recommendations or tips for new
blog owners please share. I understand this is off topic nevertheless
I just wanted to ask. Cheers!