The SC conference technical program leads the industry in many aspects of computational science delivering ground breaking technical papers, panels, posters and BOFs. Over the life of the conference, the field of computational science has matured, and is now often referred to as the third leg of science (alongside theory and experimentation).
To maintain and build on our leadership, we must develop workflows and practices that ensure robust, repeatable scientific results, even as the platforms we use for computational science continue to evolve. To support this, we propose to institute practices over the next five years that establish SC as the "gold standard" for computational science practice, as measured by the ability of a researcher to reproduce published research findings and validate scientific conclusions from simulations.
There are complex factors affecting reproducibility and replication in computational science, and there are also costs in both time and resources to institute the measures necessary to ensure reproducibility. For this reason, we are soliciting input from the SC technical community. This week, the SC program features multiple events promoting discussion on reproducibility challenges:
In 2016, we move to actions! The Student Cluster Challenge is being re-designed to focus on the replication of a previously published conference paper. Authors of technical papers accepted at SC15 (and previous years) are invited to submit proposals to reproduce their paper results. Submissions will be accepted soon - from December 1, 2015 to January 15, 2016 – and the selected paper’s software application will be featured as a competition application in the Student Cluster Competition at SC16.
We hope these steps will both improve our understanding of the challenges and raise awareness of the direction we are taking. Please share your thoughts and join us in transforming HPC at the exascale!
Michela Taufer is an associate professor, and holds the David and Beverly J.C. Mills Career Development Chair in the Department of Computer and Information Sciences at the University of Delware. Wilfred Pinhold is CEO of Concurrent Systems LLC and an adjunct professor at Portland State University.
No entries found