OSF preregistered · Confirmatory Protocol v2 · Completed

Registered Confirmatory v2 Results

Incremental Predictive Value of Spatial Graph Structure in Hourly Istanbul Traffic Forecasting: A Confirmatory Multi-Season Study.

Research home OSF registration DOI Permanent GitHub archive Successful workflow run

Primary registered endpoint

Paired MAE difference−0.0181 km/hDGT Directed Road + Adaptive − Temporal MLP

Relative MAE difference−0.490%Negative values favor the registered graph model.

Primary p-value0.5000One-sided paired Wilcoxon; α = 0.05.

Primary conclusion: the preregistered H1 superiority hypothesis was not statistically supported. The 95% hierarchical bootstrap CI for the paired MAE difference was [−0.1760, +0.1120] km/h, which crosses zero.

Registered hypothesis family

Hypothesis	Comparison / effect	Raw p	Holm-adjusted p	Interpretation
H1 primary	+1h DGT Directed Road + Adaptive vs Temporal MLP: −0.0181 km/h; −0.490%	0.500000	Not applicable	Not supported at α = 0.05.
H2	+1h DGT Directed Road + Adaptive vs DGT Identity + Adaptive: +0.0207 km/h	0.903320	1.000000	Not significant.
H3	+1h minus +6h graph-benefit contrast: −0.1574 km/h; 95% CI [−0.2539, −0.0383]	0.013672	0.068359	Raw signal, but not significant after preregistered Holm correction.
H4 +2h	DGT Directed Road + Adaptive vs Temporal MLP: +0.0327 km/h	0.753906	1.000000	Not significant.
H4 +3h	DGT Directed Road + Adaptive vs Temporal MLP: −0.0390 km/h	0.347656	1.000000	Not significant.
H4 +6h	DGT Directed Road + Adaptive vs Temporal MLP: +0.1392 km/h	0.975586	1.000000	Not significant.

Data and preregistered feasibility rules

The final confirmatory analysis used 2024-05 and 2024-11. The frozen Protocol v2 feasibility process excluded 2024-02 and 2024-08 under the registered 64-node / 98% training-coverage rule. January 2025 was excluded from all confirmatory inference because it had been used in exploratory development.

The inferential unit was the target calendar day after node-level and forecast-origin aggregation and seed averaging, yielding 10 paired confirmatory days in the registered hypothesis tests.

Uncertainty and multiplicity

Effect uncertainty was estimated with a preregistered 10,000-replicate hierarchical bootstrap resampling confirmatory month and then target day within month, with fixed RNG seed 20260814. H2, H3 and the three H4 horizon-specific tests formed one secondary family and were adjusted using the Holm procedure.

Permanent evidence and provenance

The successful registered workflow output is permanently committed to the repository rather than relying only on time-limited GitHub Actions artifacts.

Research interpretation

The confirmatory evidence does not support a claim that the registered directed-road Dynamic Graph Transformer broadly outperforms the Temporal MLP. The scientifically relevant outcome is the transparent registered result itself: small and uncertain +1h benefit, non-significant primary inference, and a secondary short-vs-long horizon pattern that did not survive the preregistered multiplicity correction.