This repo contains experimental code used to develop the theory in my job market paper. The repository focuses on improving the Vuong (1989) test with the bootstrap. The code is designed to be compatible with GenericLikelihoodModel
class in statsmodels
.
For a less experimental version of the selection tests see the following repository.
There are multiple versions of vuong_tests
and vuong_plots
in this repository. These modules contain the main code for running the test. I included the old versions so that the older examples would still be able to run. statsmodels
automatically compute these things with the GenericLikelihoodModel
class in statsmodels
. .
Each of the different tests are designed to work with statsmodels.api
. They take the likelihood ll1
of each model. They also involve the gradient grad1
, hessian hess1
, parameters params1
of each of the models. The following pseudo-code illustrates how to get the attributes from a generic likelihood model. The test requires a user defined setup_test
function for getting these attributes from the models. This way the code will work with user defined models outside of statsmodels
.
model1 = your_model(y,X)
model1_fit = model1.fit()
ll1 = model1.loglikeobs(model1_fit.params)
grad1 = model1.score_obs(model1_fit.params)
hess1 = model1.hessian(model1_fit.params)
params1 = model1_fit.params
The main versions of the test used in the paper are in vuong_tests4.py
. vuong_tests5.py
contains experimental code to avoid needing to compute the likelihood on each observation and other experimental bootstraps like the bias corrected bootstrap percentile interval and the bootstrap percentile-t interval (see Hansen chapter 10 section 10.17 and 10.18). vuong_tests6.py
simplifies this code.
Each version of the test returns 1 when model 1 is selected 2 when model 2 is selected and 0 when no model is selected.
ndVuong
is the main function with the test designed by Shi (2015).regular_test
does the classical version of the test. It has the option to include the numerator bias correction with thebiascorrect
parameter.two_step_test
is designed to run a two-step classical version of the test.bootstrap_test
is the main bootstrap version of the test
The following helper functions are included:
compute_eigen2
is designed to compute the numerator correction described in the paper.bootstrap_distr
computes all of the bootstrap test statistics with resampling.choose_c
chooses the tuning parameter for the module.
vuong_plots.py
is a module intended for visualizing the performance of the test and evaluate different properties of the test.
The main examples in the paper are:
overlapping_reg
corresponds to the first set of Monte Carlo examples.overlapping_reg
has its own readme explaining each of its' individual files.shi_ex2
correspond to the tables in the second Monte Carlo analysis section.shi
is a port of the matlab code written for Shi (2015) to python. I include the original matlab code for reference.
multiple_models
explores how the test might generalize to multiple models.splines
explores how the test might generalize to a non-parametric setting.diff_norms
explores how the test might generalize the mean squared error as a selection criteria.
auctions
contains examples similar to Donald and Paarsch (1993) involving testing functional form assumptions in auction models.logit_probit
involves testing functional form assumptions in discrete choice models.log_level
tests function form assumptions in the context of whether or not to take 'log' of the y variable in a regression model.missing
tests Tobit vs OLS.- subsampling explores an alternative subsampling bootstrap procedure similar to the one in Romano and Shaikh (2012).
nash
explores how the test might apply to different entry game models.