Scoring Predictions at Extreme Quantiles
share
Prediction of quantiles at extreme tails is of interest in numerous applications. Extreme value theory provides various competing predictors for this point prediction problem. An assessment of a set of predictors based on their predictive performance is commonly used to select the best estimate in a given situation. However, due to the extreme nature of this inference problem, it might be possible that the predicted quantiles are not seen in the historical records, therefore, making it challenging to validate the prediction with its realization. In this article, we propose two non-parametric scoring approaches to evaluate and rank extreme quantile estimates. These methods are based on predicting a sequence of equally extremal quantiles on different parts of the data. We then use the quantile scoring function to evaluate the competing predictors. The performance of the scoring methods are illustrated and compared with the conventional scoring method in a simulation study. The methods are then applied to cyber netflow data from Los Alamos National Laboratory and daily precipitation data at a station in California available from Global Historical Climatology Network.
READ FULL TEXT