Before comparing two computational experiments, count the runs that failed
Count failed and missing runs before comparing computational experiments, so the denominator reflects the work that was planned.
Reading room
The original project descriptions remain available as dated research material. Separately authored editorial notes explore how to inspect an experiment, identify its information boundaries and connect a reported result with the procedure that produced it.
Count failed and missing runs before comparing computational experiments, so the denominator reflects the work that was planned.
A model evaluation is only useful when its information boundary matches the question being asked.