Citation

Kurashkin, Sergei O. and Tynchenko, Vadim S. and Borodulin, Aleksei S. and Hammoud, Ahmad and Tee, Connie (2026) Evaluation Rigor from Graph Neural Networks to Graph Foundation Models: A Systematic Review and a Four-Axis Reporting Standard. Machine Learning and Knowledge Extraction, 8 (7). p. 194. ISSN 2504-4990

Text Evaluation Rigor from Graph Neural Networks to Graph Foundation Models_ A Systematic Review and a Four-Axis Reporting Standard.pdf - Published Version Restricted to Repository staff only Download (857kB)

Abstract

Graph machine learning reports steady progress across node, graph, and link prediction, across temporal and hypergraph frontiers, and across the emerging class of graph foundation models. This review asks a prior question: when a method is reported to outperform the alternatives, how far does the evidence support the claim? We organize the answer around four axes of evaluation rigor: statistical rigor (seeds, dispersion, formal significance testing), baseline fairness (budget-parity tuning of trivial and structure-agnostic baselines), data integrity (leakage, duplication, negative sampling, contamination), and claim integrity (whether gains survive fair tuning and discriminative benchmarks). Drawing on a criterionbased corpus of 150 studies, of which 51 were read in full depth, we find a consistent picture. Only two of 25 methodologically central backbone studies apply a formal betweenmethod significance test, and reported gains repeatedly shrink or disappear once a trivial baseline is tuned to parity, a leaked split is repaired, or a pretrained model is evaluated on unseen data. We argue that these failures share one cause: the saturation of benchmarks that can no longer discriminate between methods. The principal output is a minimum reporting standard, a concrete four-axis checklist that authors and reviewers can apply at negligible cost.

Item Type:	Article
Uncontrolled Keywords:	Graph neural networks, evaluation rigor
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Rosnani Abd Wahab
Date Deposited:	03 Sep 2026 08:28
Last Modified:	03 Sep 2026 08:28
URII:	http://shdl.mmu.edu.my/id/eprint/16646

Downloads

Edit (login required)