Description

This Master Prompt focuses on determining how much confidence can reasonably be placed in the evidence entering a review synthesis. It guides researchers in moving beyond the assumption that publication, peer review, citation frequency, journal reputation, or inclusion in a review automatically establishes evidential credibility. Instead, each included study should be examined according to criteria appropriate to its design, disciplinary context, research purpose and intended role in the synthesis. Appraisal may therefore consider methodological appropriateness, sampling and population adequacy, measurement validity and reliability, procedural transparency, analytical rigor, reporting completeness, reproducibility, data and code availability, limitations, potential sources of bias, applicability and research integrity. The uploaded literature demonstrates the value of purpose-specific appraisal: one systematic review of scholarly Knowledge Graph research used six a priori indicators covering gold standards, inter-annotator agreement, user-study protocols, provenance granularity, extrinsic evaluation and openness, while explicitly explaining why conventional formal risk-of-bias and certainty tools were not appropriate for its predominantly methodological and exploratory evidence base.

A major emphasis is placed on fitness-for-purpose appraisal rather than indiscriminate checklist application. Formal critical-appraisal, risk-of-bias or certainty-of-evidence instruments should be selected only when they correspond to the included study designs and the purpose of the review. Where heterogeneous evidence includes experiments, observational studies, qualitative investigations, modeling studies, technical validation studies, software evaluations, conceptual papers or mixed evidence, appraisal criteria may need to be differentiated while retaining a transparent common logic. The Master Prompt also requires researchers to distinguish poor methodological quality from poor reporting: an unreported procedure cannot automatically be assumed not to have occurred, but insufficient reporting may reduce the confidence with which the study can be interpreted or reproduced. This distinction is particularly important because the uploaded literature treats transparent reporting, appropriate searching, recency, critical appraisal and accessible study-level data as important considerations when determining whether an existing systematic review is sufficiently suitable to serve as an evidence base.