Digital Finds

joseph dot reeves at thehumanjourney dot net

Good Science Requires Open Source

Feb 08, 2010 by Joseph Reeves

There's a fantastic article on the Guardian's website about further revelations emerging from the UEA "Climategate" affair. The conclusion? Release the source code of applications you use in scientific endeavours. I said something like this at TAG 2008:

The Guardian 5 February 2010: If you're going to do good science, release the computer code too

Computer code is also at the heart of a scientific issue. One of the key features of science is deniability: if you erect a theory and someone produces evidence that it is wrong, then it falls. This is how science works: by openness, by publishing minute details of an experiment, some mathematical equations or a simulation; by doing this you embrace deniability. This does not seem to have happened in climate research. Many researchers have refused to release their computer programs — even though they are still in existence and not subject to commercial agreements. An example is Professor Mann's initial refusal to give up the code that was used to construct the 1999 "hockey stick" model that demonstrated that human-made global warming is a unique artefact of the last few decades. (He did finally release it in 2005.)

So, if you are publishing research articles that use computer programs, if you want to claim that you are engaging in science, the programs are in your possession and you will not release them then I would not regard you as a scientist; I would also regard any papers based on the software as null and void.

I don't know if anyone has ranted at Professor Darrel Ince, but the reaction I got was fantastic; amongst a general shower of scorn, one person flat out called me a liar and another said that such an issue was too theoretical for a meeting of the Theoretical Archaeology Group.

Open Source Software is gaining momentum within archaeological practice, but perhaps not as quickly as it could be. The reason why? Probably because of the nature of archaeological practice rather than the ability of the tools. Many of the practitioners of archaeological computing (in the UK at least, probably also elsewhere) are ivory tower academics interested in producing published volumes. "Archaeological computing", for most, is little more than producing pretty pictures for books. It barely matters for most if you can access the original data, or the black box software that turned it into an image, because there is no element of experimentation or reproducibility required; you've simply produced an image for a book. Other over worked areas of archaeological computing, such as least cost path analysis, view sheds and calorific analysis of landscape exploitation have such little merit from an archaeological perspective that they barely warrant trying to justify by any scientific measures. I tried to this second point a few years back during my MA and, although the only linkable mention of it I can find is on Jo's blog, I think it went down a little better than when I tried to point out the obvious at TAG. Whilst phenomenology critics may enjoy my paper for demonstrating holes in the practice, it also points out that people don't just walk along the easiest paths they can find, nor are they necessarily interested by the fact they can see something impressive on the horizon.

There's plenty of MSc Archaeological Computing types around, but few of them actually do anything scientific. There's no scientific rigour within archaeological practice; no reproducibility, little experimentation, no openness. No scientific hypothesis to stand up and knock down. Archaeological theory is sometimes played with by archaeological computing practitioners, certainly nobody could work in a theoretical vacuum, but it's often implicit, if not shunned. Many people want to make maps within an accepted and uncritical theoretical wasteland that has largely failed to see the emergence of post-processual thinking.

Archaeological computing is, more often than not, behind the curve on scientific practice and archaeological thinking. Professor Ince highlights the need for openness and scientific rigour; let's just hope that people start doing more to work that would actually involve a need for this approach.

Post a Comment:
Comments are closed for this entry.