IEEE Computer – Reproducible Research

August 9, 2012

The July/August 2012 issue of IEEE Computer is dedicated to:

Reproducible Research


The Editorial:

Reproducible Research: Tools and Strategies for Scientific Computing

by Victoria Stodden, long term advocate of Reproducibility Research and Open Science, introduces a set of fascinating articles on multiple practical approaches to restoring reproducibility verification in scientific research.

 “This special issue is comprised of articles contributed by participants in a workshop held in Vancouver in July 2011 called “Reproducible Research: Tools and Strategies for Scientific Computing.”

One of our aims was to improve the visibility of the nascent group of tool builders working to facilitate really reproducible research in computational science.

This special issue focuses on tools and strategies for reproducible computational science.” 

The articles cover a diverse set of topics, and describe tools that are available today (many of them as Open Source software) to empower researchers to facilitate the reproducibility of their work.

They include:

The requirement for research publications to be reproducible goes back to the Dawn of Science, and the early days of the Royal Society, whose motto was (and still is):

Royal Society Coat of Arms“Nullius in Verba”
“Take Nobody’s word for it”

“It is an expression of the determination of Fellows of the Royal Society
to withstand the domination of authority
and to verify all statements by an appeal to facts determined by experiment.”


Unfortunately, this essential characteristic of scientific work has been lost today, in large measure due to the misplaced reward system that demands academics to publish large numbers of papers without consideration of whether the work is reproducible or not. The result of this sad state of affairs is that about 90% of scientific papers are not reproducible.

The Reproducible Research movement is working hard on restoring the practice of reproducibility verification by raising awareness, and by providing practical mechanisms for making this part of the daily routine of scientific work once again.

 From this collection of articles, two that are of great practical utilitiy are:

  • The CDE tool by Philip Guo, which enables the creation of portable Linux (executable) applications as self-contained packages. Think of a minimal Linux platform that is fully-customized for the sole purpose of running one executable.
  • The use of Virtual Appliances and Cloud Computing, by Bill Howe, as platforms for enabling the distribution of applications and data used during research work. This would enable others to verify the reproducibility of the research.

This special issue is a very valuable contribution to the Reproducible Research movement and by logical implication, to the Open Science movement.

Leave a Reply