A Reality Check for Data Snooping (paper)
Do we really know what we think we know? White’s famous paper on his “Reality Check” procedure:
“Data Snooping occurs when a given set of data is used more than once for purposes of inference or model selection. When such data reuse occurs, there is always the possibility that any satisfactory results obtained may simply be due to chance rather than to any merit inherent in the method yielding the results. Our new procedure, the Reality Check, provides simple and straightforward procedures for testing the null that the best model encountered has no predictive superiority over a given benchmark model…”
Related posts: