Please use this identifier to cite or link to this item:

Assessing the integrity of cross-validation : a case for small sample-based research

Authors Kim, Namwoon
Han, Jin Kyung
Issue Date 1997-07
Summary In the context of small sample-based research, investigators often encounter feasibility problem(s) in cross-validation endeavors. The problem(s), in general, can be traced to two criteria: (1) methodological (i.e., arising due to inefficient or limited portfolio of cross-validation methods); and (2) data-related (i.e., originating due to availability-constraints on pertinent research sample). In this paper, we address the former source of problem as the main contingency when the latitude on the latter problem source is exogenous to the investigator. Specifically, four resampling-class of validation methods applicable for small-sample context are examined: sample-repeat, bootstrap, jackknife, and introducing bootjack. Bootjack, developed by the authors, incorporates desirable features from two other methods (bootstrap and jackknife). The four techniques are put through a comparative analysis across sample-sizes of various magnitudes. Moreover, the methods are judged for the integrity of their prediction results, which consists of (1) unbiasedness in predictions; (2) consistency of predictions across different sample sizes; and (3) stability of prediction results. Finally, recommendations are made regarding small-sample validation predicament.
Language English
Format Working paper
Files in this item:
File Description Size Format
mktg97096.pdf 1595013 B Adobe PDF