Comments on: On Unpublished Software

By: EpiGrad

EpiGrad — Fri, 10 Feb 2012 22:11:22 +0000

I’m not sure I agree. While I do appreciate when researchers make their code public, I’ve never actually found myself needing to rely on someone else’s code for a research project. If a paper relies on and focuses entirely on the software, I might understand, but I’ve always argued that the baseline level of presentation should be that the code can be rebuilt, not merely that it exists. This is, for no other reason, important to prevent the propagation of inefficient algorithm design, coding errors, or other design flaws.

Indeed, the one time I tried using someone else’s code, the amount of trouble I went to trying to shoehorn their software into my question entirely offset the time saved not coding it myself.

By: RSingh

RSingh — Fri, 10 Feb 2012 22:03:13 +0000

The practice of not including the code seriously hampers the dissemination of the published work.

Researchers also have limited time and the lack of code is an impediment to learning or utilising the published technique.

It should be made mandatory that the code is included, either within the paper or reference to a link on a website.

By: nsaunders

nsaunders — Fri, 10 Feb 2012 02:42:23 +0000

I read Jonathan’s post too and had no sympathy for the authors. Neither, though, did I have antipathy toward them – they are not the issue. What’s at fault here are the journals and the reviewers. Until they enforce some standards, authors will get away with bad practice. How a reviewer can declare the findings of a study to be valid without access to the analyses used is completely beyond my comprehension.

By: Justin Rising

Justin Rising — Fri, 10 Feb 2012 02:40:21 +0000

“I’ve had one collaborator essentially decide not to give out code because further analysis – that would appear in future papers – was already baked in, and they weren’t going to go through their code line by line to make sure some poor grad student’s project didn’t get scooped by someone reading the code carefully, or that in removing that stuff, they didn’t manage to otherwise break the software.”

This is the perfect example of something that’s easy to avoid by properly using a version control system (e.g., svn/git/etc.). Check in the version that went in to the original paper, make a tag/branch for it, and then you can make whatever modifications you want while still always having access to something that’s suitable for release.