I was giving a talk at the Digital Humanities Oxford Summer School on reproducibility this year and had an intriguing question. A review of a recent conference paper reminds me of this.
At the end of the lecture, I pose two questions:
Can someone on your group reproduce one of your results using available information on the web or in papers?
https://iaine.github.io/ReproResOxford/final.html
It is a good way of seeing if a piece of work is reproducible. During the discussion, the question of tacit knowledge was raised, this being the knowledge that you do not necessarily know that you know. Arguably, this might be picked up in such a test but not always. Each member of a group may have the same knowledge and forget that they have it, or have not explained it. It is hard to step out and to make this knowledge clear or to put it out there.
A review for a conference paper reminds me of this. I put in a paper on Joshua Steele that described some of the work that I have been doing with his notation. More of this to come but the review pointed out that we had barely mentioned how to read the notation. It is not easy to read and requires concentration. I am still finding things in it but I have also been working with it, more off than on, for 2 or 3 years. When I get my head into the work, it becomes a tacit knowledge.
So how do we begin to approach it? I do not know at the moment but getting other people to review is a start. Citations and provenance so that the argument can be followed are useful as well. I am parking this here as something to explore. However, the reviewer’s comment was really useful in bringing this up.
No Comments