7. PM-VAL-007 — Predictive Model Evaluation, Cross-Validation, Performance Benchmarking, Uncertainty Quantification, Explainable AI and Generalizability

Description

This Master Prompt establishes a rigorous methodological framework for evaluating the accuracy, reliability, interpretability, robustness, uncertainty and practical validity of predictive models developed using statistical, machine learning, deep learning, mechanistic and hybrid approaches. It guides researchers through internal validation, external validation, holdout testing, k-fold cross-validation, nested cross-validation, bootstrap validation, temporal validation, spatial validation, multicentre validation and independent dataset evaluation. The theme addresses regression metrics such as MAE, MSE, RMSE, MAPE and R², alongside classification metrics including accuracy, precision, recall, sensitivity, specificity, F1-score, ROC-AUC and precision–recall analysis, selected according to the prediction task. Particular attention is given to calibration, discrimination, prediction intervals, uncertainty estimation, confidence assessment, error propagation, model stability, distribution shift, domain transferability and robustness under changing conditions. Researchers are guided in applying explainable artificial intelligence techniques such as SHAP, permutation importance, partial dependence analysis, feature attribution and other interpretable modelling approaches. The theme also covers statistical comparison of competing models, ablation analysis, sensitivity testing, subgroup performance, fairness considerations, data leakage prevention, reporting transparency and reproducibility. Applications reflected in the uploaded literature include explainable thermal comfort prediction, clinical risk scoring, radiomics, property price prediction, structural performance and critical appraisals of AI-based predictive research. It emphasizes that high predictive accuracy alone is insufficient without evidence of validity, interpretability, calibration and generalization.