Overview The Altmetrics Collection Jason Priem1, Paul Groth2*, Dario Taraborelli3 1 School of Information & Library Science, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States of America, 2 Department of Computer Science and The Network Institute, VU University Amsterdam, Amsterdam, The Netherlands, 3 The Wikimedia Foundation, San Francisco, California, United States of America Introduction methods to gather evidence of broader them–tracks with potential to show infor- impacts and provide more detail about the mal paths of influence with unprecedented What paper should I read next? Who science system: tracking acknowledge- speed and resolution. Many of these tools should I talk to at a conference? Which ments [10], patents [11], mentorships offer open APIs, supporting large-scale, research group should get this grant? [12], news articles [8], usage in syllabuses automated mining of online activities and Researchers and funders alike must make [13], and many others, separately and in conversations around research objects daily judgments on how to best spend their various combinations [14]. The emer- [21]. limited time and money–judgments that gence of the Web, a ‘‘nutrient-rich space Altmetrics [22,23] is the study and use are becoming increasingly difficult as the for scholars’’ [15], has held particular of scholarly impact measures based on volume of scholarly communication in- promise for new filters and lenses on activity in online tools and environments. creases. Not only does the number of scholarly output. Webometrics researchers The term has also been used to describe scholarly papers continue to grow, it is the metrics themselves–one could propose have uncovered evidence of informal joined by new forms of communication in plural a ‘‘set of new altmetrics.’’ impact by examining networks of hyper- from data publications to microblog posts. Altmetrics is in most cases a subset of links and mentions on the broader Web To deal with incoming information, both scientometrics and webometrics; it is [16–18]. An important strand of webo- scholars have always relied upon filters. a subset of the latter in that it focuses more metrics has also examined the properties At first these filters were manually com- narrowly on scholarly influence as mea- of article download data [7,19,20]. piled compendia and corpora of the sured in online tools and environments, rather The last several years, however, have literature. But by the mid-20th century, than on the Web more generally. presented a promising new approach to filters built on manual indexing began to Altmetrics may support finer-grained break under the weight of booming gathering fine-grained impact data: track- ing large-scale activity around scholarly maps of science, broader and more postwar science production. Garfield [1] equitable evaluations, and improvements and others pioneered a solution: automat- products in online tools and environments. These tools and environments include, to the peer-review system [24]. On the ed filters that leveraged scientists own other hand, the use and development of impact judgments, aggregating citations among others: altmetrics should be pursued with appro- as ‘‘pellets of peer recognition.’’ [2]. These citation-based filters have dra- N social media like Twitter and Facebook priate scientific caution. Altmetrics may face attempts at manipulation similar to matically grown in importance and have N online reference managers like CiteU- what Google must deal with in web search become the tenet of how research impact Like, Zotero, and Mendeley ranking. Addressing such manipulation is measured. But, like manual indexing 60 N collaborative encyclopedias like Wiki- may, in-turn, impact the transparency of years ago, they may today be failing to pedia altmetrics. New and complex measures keep up with the literature’s growing volume, velocity, and diversity [3]. N blogs, both scholarly and general- may distort our picture of the science audience system if not rigorously assessed and Citations are heavily gamed [4–6] and are painfully slow to accumulate [7], and N scholarly social networks, like Research- correctly understood. Finally, altmetrics may promote an evaluation system for Gate or Academia.edu overlook increasingly important societal scholarship that many argue has become and clinical impacts [8]. Most importantly, N conference organization sites like La- overly focused on metrics. they overlook new scholarly forms like nyrd.com datasets, software, and research blogs that Growing numbers of scholars are using Scope of this Collection fall outside of the scope of citable research these and similar tools to mediate their The goal of this collection is to gather an objects. In sum, citations only reflect formal interaction with the literature. In doing so, emerging body of research for the further acknowledgment and thus they provide only a they are leaving valuable tracks behind study and use of altmetrics. We believe it is partial picture of the science system [9]. Scholars may discuss, annotate, recom- mend, refute, comment, read, and teach a Citation: Priem J, Groth P, Taraborelli D (2012) The Altmetrics Collection. PLoS ONE 7(11): e48753. doi:10.1371/ journal.pone.0048753 new finding before it ever appears in the formal citation registry. We need new Editor: Christos A. Ouzounis, The Centre for Research and Technology, Hellas, Greece mechanisms to create a subtler, higher- Received October 1, 2012; Accepted October 4, 2012; Published November 1, 2012 resolution picture of the science system. Copyright: ß 2012 Priem et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The Quest for Better Filters Funding: These authors have no support or funding to report. The scientometrics community has not been blind to the limitations of citation Competing Interests: The authors have declared that no competing interests exist. measures, and has collectively proposed * E-mail:

[email protected]

PLOS ONE | www.plosone.org 1 November 2012 | Volume 7 | Issue 11 | e48753 The Altmetrics Collection greatly needed, as important questions N Validation of models of scientific The current collection includes articles regarding altmetrics’ prevalence, validity, discovery/recommendation based on that address many of these areas. It will distribution, and reliability remain incom- altmetrics publish new research on an ongoing basis, pletely answered. Importantly, the present collection, which has the virtue of being N Qualitative research describing the and we hope to see additional contribu- tions appear in the coming months. We scholarly use of online tools and online and open access, allows altmetrics environments look forward to building a foundation of researchers to experiment on themselves. early research to support this new field. The collection’s scope includes: N Empirically-supported theory guiding altmetrics’ use Author Contributions N Statistical analysis of altmetrics data N Other research relating to scholarly Wrote the paper: PG JP DT. sources, and comparisons to estab- impact in online tools and environ- lished sources ments. N Metric validation, and identification of biases in measurements References 1. Garfield E (1955) Citation indexes to science: a iour. Journal of Documentation 50: 165–196. 18. Vaughan L, Shaw D (2005) Web citation data for new dimension in documentation through asso- doi:10.1108/eb026929. impact assessment: a comparison of four science ciation of ideas. Science 123: 108–111. 11. Pavitt K (1985) Patent statistics as indicators of disciplines. Journal of the American Society for 2. Merton RK (1988) The Matthew Effect in innovative activities: Possibilities and prob- Information Science 56: 1075–1087. Science, II. ISIS 79: 606–623. lems. Scientometrics 7: 77–99. doi:10.1007/ 19. Bollen J, Van de Sompel H, Hagberg A, Chute R 3. Tenopir C, King D (2008) Electronic journals and BF02020142. (2009) A principal component analysis of 39 changes in scholarly article seeking and reading 12. Marchionini G, Solomon P, Davis C, Russell T scientific impact measures. PLoS ONE 4. patterns. DLib Magazine 14. Available: http:// (2006) Information and library science MPACT: doi:10.1371/journal.pone.0006022. www.dlib.org/dlib/november08/tenopir/ A preliminary analysis. Library and Information 20. Kurtz MJ, Eichhorn G, Accomazzi A, Grant CS, 11tenopir.html. Science Research 28: 480–500. Demleitner M, et al. (2005) The bibliometric 4. Falagas M, Alexiou V (2008) The top-ten in 13. Kousha K, Thelwall M (2008) Assessing the properties of article readership information. journal impact factor manipulation. Archivum impact of disciplinary research on teaching: An Journal of the American Society for Information Immunologiae et Therapiae Experimentalis 56: automatic analysis of online syllabuses. Journal of Science 56: 111–128. 223–226. doi:10.1007/s00005-008-0024-5. the American Society for Information Science 21. Priem J, Hemminger BH (2010) Scientometrics 5. Wilhite AW, Fong EA (2012) Coercive Citation in and Technology 59: 2060–2069. doi:10.1002/ 2.0: Toward new metrics of scholarly impact on Academic Publishing. Science 335: 542–543. asi.20920. the social Web. First Monday 15. Availa- doi:10.1126/science.1212540. 14. Martin BR, Irvine J (1983) Assessing basic ble:http://firstmonday.org/htbin/cgiwrap/bin/ 6. The PLoS Medicine Editors (2006) The Impact research?: Some partial indicators of scientific ojs/index.php/fm/article/view/2874/2570. Factor Game. PLoS Med 3: e291. doi:10.1371/ journal.pmed.0030291. progress in radio astronomy. Research Policy 12: 22. jasonpriem (2010) I like the term #articlelevelmetrics, 7. Brody T, Harnad S, Carr L (2006) Earlier Web 61–90. doi:16/0048-7333(83)90005-7. but it fails to imply *diversity* of measures. Lately, I’m usage statistics as predictors of later citation 15. Cronin B, Snyder HW, Rosenbaum H, Martin- liking #altmetrics. Available: https://twitter.com/#!/ impact. Journal of the American Society for son A, Callahan E (1998) Invoked on the Web. jasonpriem/status/25844968813. Accessed 2012 Oct Information Science and Technology 57: 1060– Journal of the American Society for Information 8. 1072. doi:10.1002/asi.20373. Science 49: 1319–1328. doi:10.1002/(SICI)1097- 23. Priem J, Taraborelli D, Groth P, Neylon C (2010) 8. Lewison G (2002) From biomedical research to 4571(1998)49:14[1319::AID-ASI9]3.0.CO;2-W. alt-metrics: a manifesto. Available:http://altmetrics. health improvement. Scientometrics 54: 179–192. 16. Almind TC, Ingwersen P (1997) Informetric org/manifesto/. Accessed 2011 August 15. doi:10.1023/A:1016005710371. Analyses on the World Wide Web: Methodolog- 24. Taraborelli D (2008) Soft peer review: Social 9. de Solla Price DJ, Beaver D (1966) Collaboration ical Approaches to ‘‘WEBOMETRICS.’’ Journal software and distributed scientific evaluation. in an invisible college. American Psychologist 21: of Documentation 53: 404–426. Proceedings of the 8th International Conference on the 1011–1018. 17. Thelwall M, Vaughan L, Bjo¨rneborn L (2005) Design of Cooperative Systems (COOP ’08). Carry-Le- 10. Cronin B, Overfelt K (1994) The scholar’s Webometrics. Annual Review of Information Rouet. Available: http://discovery.ucl.ac.uk/ courtesy: A survey of acknowledgement behav- Science and Technology 39. 8279/. PLOS ONE | www.plosone.org 2 November 2012 | Volume 7 | Issue 11 | e48753