|Title||How Much Does Memory Layout Impact Performance? A Wide Study|
|Publication Type||Conference Paper|
|Year of Publication||2014|
|Authors||Oliveira, A., J-C. Petkovich, and S. Fischmeister|
|Conference Name||Proceedings of the International Workshop on Reproducible Research Methodologies (REPRODUCE)|
|Conference Location||Orlando, USA|
Memory layout variations are often cited as a threat to the reproducibility of computer performance experiments. Previous research has shown worst-case effects as large as 300% of execution time, among other surprising cases. While these are worrying results, how frequent and widespread are these large effects? To answer this question, this paper presents a wide-scale evaluation of memory layout effects on the performance of a large subset of the SPECCPU2006 benchmark suite on a wide array of diverse machines. We find that on average, these benchmarks are not as susceptible to memory layout effects as the worst-case analysis found in the literature suggests. Finally, we re-execute our own experiment and demonstrate why the reproducibility of an experiment's result should not depend on its data being perfectly reproducible.