Yesterday Times Higher Education published an interesting article by Matthew Gamble, a computer scientist working on web science questions. Gamble’s article addresses the need for Web 2.0 scholarship – the use of online metrics for evaluating science; piggy backing on other discussions in the field such as alt-metrics (which Gamble also mentions).
This discussion opens doors to a number of questions about knowledge production processes as well as what is valued in science and what should/could be measured as impact. These discussions were also the topic of the recent altmetrics workshop at the Web Science Conference in Koblenz, Germany in June 2011 (which I attended). The Altmetrics workshop itself was the first steps towards building a recognized community in science who were researching alternative metrics to science. The workshop brought together researchers from multiple disciplines and facilitated great discussions on a wide number of topics that look at understanding not only Web behaviors of scientists, but collection and disambiguation problems of Web data and how to understand the implications of science and knowledge production on the Web. Overall one of the best workshops I have attended, yet, that perfectly fit my area of growing expertise.
I presented some exploratory research on the validity of online metrics in science. The work was completed with my colleague Shenghui Wang, a talented computer scientist, who I developed a crawler with (she did the actual building, I did the informing) to investigate a community of scientists online. The title was – “Who are we talking about?: the validity of online metrics for commenting on science”. You can find the complete abstract here: http://altmetrics.org/workshop2011/birkholz-v0/. Paper is in the works.
Preliminary/exploratory results indicated that, in the sample of Dutch computer scientists and their co-authors from 2007 – March 2011, the higher your h-index (a measure of performance) the more likely you are to be found on LinkedIn, Slideshare and have a blog. Additionally the higher the citation score (a measure of tenure and performance) the more likely you are be on LinkedIn and have a blog. This suggests that among this community the measuring of web behaviors of a scientists own enterprise are representative of dynamics of scientist who have both a higher tenure and higher performance, thus when talking about implications of altmetrics and or analyzing behavior on these social media sites we need to be explicit about who we can generalize about and how these reflect to greater dynamics in science; as for this sample we can only reflect on the behaviors of high performance and tenured scientists. Further research needs to be completed to test this on other research communities and further develop recall precision techniques used in the web crawler to obtain the data on scientists’ presence on these sites; although we might suggest that if this holds true for other communities that altmetrics would provide a unique avenue for analyzing those leading the pack in their respective fields which would allow more immediate impact measures for understanding science overcoming the delay of impact measures that integrate citation.