Issue 38 – September 2014


10 years of research impact: top cited papers in Scopus 2001-2011

Gali Halevi and Henk Moed investigate what the most frequently cited articles were in Scopus from 2001-2011, in eight main research areas, and give their authors the chance to comment on their achievements.

Read more >

Scopus is celebrating 10 years since its launch. As the largest abstract and citation database of peer-reviewed literature available today, Scopus boasts 53 million records, 21,915 titles from 5,000 publishers. In this paper we aimed to identify some of the top cited papers indexed in Scopus across various disciplines between 2001 and 2011. In addition, we contacted the authors of these papers to seek their insight about why they think their papers are as highly cited as they are.

In order to achieve this, we conducted a comprehensive search on all Scopus data, limiting the results to articles published between 2001 and 2011. Scopus is the largest abstract and citation database of peer-reviewed literature, and features smart tools to track, analyze and visualize research. The initial search results yielded more than 13 million records (as of June 11, 2014). This set was further refined, to include only full research articles while excluding reviews, editorials or book chapters. The search results were then limited to one of Scopus' 26 subject categories at a time (see Table 1 for full list). Each set of articles under a subject category was sorted by “cited by” counts (i.e. citations), which enables the highly cited articles to be identified.

In this paper we review the following 8 subject areas and their top cited articles:

  • Agricultural and Biological Sciences
  • Arts and Humanities
  • Computer Science
  • Chemical Engineering
  • Energy
  • Engineering
  • Environmental Science
  • Medicine


Agricultural and Biological Sciences

The top cited article between 2001 and 2011 in Agricultural and Biological Sciences is:
Tamura, K., Dudley, J., Nei, M., Kumar, S.MEGA4: Molecular Evolutionary Genetics Analysis (MEGA) software version 4.0 (2007) Molecular Biology and Evolution, Vol. 24, No. 8, pp. 1596-1599.
Cited 17,359 times (as of June, 2014).

MEGA [Molecular Evolutionary Genetics Analysis] is a freely available software tool for conducting statistical analysis of molecular evolution and for constructing phylogenetic trees. MEGA is used by biologists in a large number of laboratories for reconstructing the evolutionary histories of species and inferring the extent and nature of the selective forces shaping the evolution of genes and species (1). This software was first developed by Sudhir Kumar and Koichiro Tamura in the laboratory of Dr. Masatoshi Nei (2). The first version of this software was released in 1993. As expected, the main disciplines citing this article are Agricultural and Biological Sciences, Biochemistry, Genetics and Molecular Biology, Immunology, Medicine and Veterinary Sciences. However, there are several interesting disciplines citing this software including Social Sciences, Arts and Humanities and Business, which may not seem directly related to the core research field of this software. A closer look at these citing disciplines reveals that the software has been used to track Ancient DNA in Anthropology and Archeology and to sketch the markup of civilization (3, 4) as well as study the phenomenon of the emergence and extinction of languages (5).

Comments from Prof. Kumar:

"This article described a useful software tool that enables comparative analysis of DNA and protein sequences from different individuals, strains, and species. Such analyses are becoming very important in this age of genomics, and increasingly larger numbers of scientists are using MEGA software to analyze their data."

Comments from Prof. Nei:

"MEGA4 is the fourth version of the MEGA, and in this version a new Maximum Composite Likelihood method of estimating evolutionary distances and other evolutionary parameters have been introduced. It has also been made usable in Linux and Intel-based Macintosh computers. Because of these new features, the MEGA4 article has been cited a large number of times. This improvement of the software was done primarily by Koichiro Tamura and Sudhir Kumar. Further improvement of the software was published later in the MEGA5 (2011) and MEGA6 (2013) articles."


Arts & Humanities

The top cited article between 2001 and 2011 in Arts & Humanities is:
McCall, L. The complexity of intersectionality (2005) Signs, Vol. 30, No. 3, pp. 1771-1800.
This article was cited 640 times (as of July, 2014).

This article discusses the complexity of studying the issue of intersectionality and offers different methods to do so. Intersectionality (or intersectionalism) is the study of intersections between forms or systems of oppression, domination or discrimination (6). The article was written by Leslie McCall, a professor at Northwestern University whose main areas of research include social inequality, economic and political sociology, methods, and social theory. This article is highly cited by research papers in Arts & Humanities and Social Sciences in the context of gender-related psychology, ethnic identity and feminism. Yet, it is also cited by Business and Management research focusing on women’s careers in business (7), workplace diversity (8) and women’s leadership skills development (9). Another interesting discipline citing this paper is Environmental Sciences, which refers to it in the context of gender-related client change adaptation (10) and gender migration patterns (11), to name two examples.

 Comments from Dr. McCall:

"I believe [the high citation count] has to do with interdisciplinary interest in the issue of intersectionality across a wide range of fields. I try to extend the usefulness of the concept for quantitative as well as qualitative research. The latter tends to dominate the study of intersectionality so this article has helped justify research in more quantitatively oriented fields."



The top cited article between 2001 and 2011 in the field of Energy is:
Allison, J., Geant4 developments and applications (2006) IEEE Transactions on Nuclear Science, Vol. 53, No. 1, pp. 270-278.
This article was cited 1,450 times (as of July 2014).

Geant4 is a software tool developed by scientists from all over the world. The article boasts 44 authors from various countries including UK, USA, Japan, Switzerland, Italy, Spain and Russia to name a few. Geant4 is a software toolkit for the simulation of the passage of particles through matter. It is used for a large number of experiments and projects in a variety of application domains, including high energy physics, astrophysics and space science, medical physics and radiation protection (22). The article was mostly cited by articles in the field of Physics and Astronomy and Engineering. In addition, a large number of citations were received from the field of Medicine where the toolkit is used to track the effect of materials on the human body (23).

Comments from Prof. Asai:

"“Geant4 developments and applications” is our second general publication followed by "Geant4 - A Simulation Toolkit", J.S. Agostinelli et al., Nuclear Instruments and Methods A, Vol. 506 (2003) 250-303. Geant4 is a software toolkit for simulating elementary particle passing through and interacting with matter. Its areas of application include high energy, nuclear and accelerator physics, as well as studies in medical science, space science and material science, which are rapidly expanding."



Chemical Engineering

The top cited article between 2001 and 2011 in the field of Chemical Engineering is:
Kreuer, K.D. On the development of proton conducting polymer membranes for hydrogen and methanol fuel cells. (2001) Journal of Membrane Science, Vol. 185, No. 1, pp. 29-39.
This article was cited 1,689 times (as of July 2014).

Proton conducting polymer membranes are of general interest because such membranes can be used to conduct protons in fuel cells, which convert, for example hydrogen or methanol into electrical energy and show promise as low emission power sources. So far, the benchmark membrane material was Nafion, a sulfonated tetrafluoroethylene based fluoropolymer-copolymer discovered in the late 1960s by Walther Grot of DuPont which is not only used in fuel cells, but also in other electrochemical devices, chlor-alkali production, metal-ion recovery, water electrolysis, plating, surface treatment of metals, batteries, sensors, Donnan dialysis cells, drug release, gas drying or humidification, and superacid catalysis for the production of fine chemicals (17). The paper actually reveals structure/property relationships for Nafion and alternative hydrocarbon ionomers, and it presents improved proton conducting polymer membranes (a/k/a polymer electrolyte membranes), along with methods for the manufacture thereof (16). The article even provided visions about membranes conducting protons in the absence of any humidification. Due to the wide range of applications and the need for better membranes, this article was found to be highly cited by Chemistry, Materials Science, Chemical Engineering and Energy.


Comment from Prof. Kreuer:

"I am aware of the impact this paper has generated in the community.

This is a pioneering work making, for the first time, a semi-quantitative connection between morphology (microstructure) and transport (proton conductivity, water transport) of fuel cell membranes (hydrocarbon versus PFSA). The disclosed differences provide rationales for explaining many other properties. The materials are highly relevant for fuel cell and other electrochemical applications, and the paper provides clear guidelines for optimizing such materials."


Computer Science

The top cited article between 2001 and 2011 in the field of computer science is:
Lowe, D.G.Distinctive image features from scale-invariant keypoints (2004) International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110.
This article was cited 15,797 times (as of July 2014).

The paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene by using object recognition algorithm. The algorithm was published by David Lowe in 1999. Applications of this algorithm include object recognition, robotic mapping and navigation, image stitching, 3D modeling, gesture recognition, video tracking, individual identification of wildlife and match moving. The algorithm is patented in the US; the owner is the University of British Columbia (18). In addition to being highly cited in related disciplines such as Engineering and Mathematics, this article and the method described are also cited by Health, Decision and Social Sciences fields. In Health Sciences the method is used for organ imaging (19), while in Social Sciences it is used to track the processing and interpretation of visual images by humans, to give an example (20). Examining Decision Sciences in the context of this article, the method has been used to study decision processing based on visual recognition, such as street signs (21).

Comments from Prof. Lowe:

"The reasons for the high citations include the fact that it describes a useful algorithm for other researchers in computer vision to match images in a way that wasn't available previously. In addition, the method is very efficient compared to previous approaches, so it is widely used in practice which leads to further citations."



The top cited article between 2001 and 2011 in the field of Engineering (focusing on Condensed Matter Physics) is:
Geim, A.K., Novoselov, K.S.The rise of graphene (2007) Nature Materials, Vol. 6, No. 3, pp. 183-191.
This article was cited 11,102 times (as of July 2014).

Graphene is pure carbon in the form of a very thin, nearly transparent sheet, one atom thick. It is remarkably strong for its very low weight (100 times stronger than steel) and it conducts heat and electricity with great efficiency. It was first produced in the lab in 2004 (24). This article discusses the nature and uses of Graphene and the emergence of a new paradigm of 'relativistic' condensed matter physics.

Citing articles are from a wide spectrum of sciences including Materials Sciences, Chemistry, Energy, Pharmacology, Computer Science and so forth, in all of which Graphene is used, studied and developed. Graphene is probably a good example of basic research leading to a technological innovation. Thus, examining citations to this article in Social Sciences, one notices that this article is cited by papers describing the global Graphene research front (25), patenting trends (26) and the use of Graphene in technological developments (27) to name a few.

Comment from Prof. Geim:

"This paper should be viewed in combination with our paper “Electric field in atomically thin carbon films” (Science, 2004). Both are equally well cited as laying foundations for graphene research, a Nobel-prize winning subject."


Environmental Science

The top cited article between 2001 and 2011 in the field of Environmental Sciences is:
Kolpin, D.W., Furlong, E.T., Meyer, M.T., Thurman, E.M., Zaugg, S.D., Barber, L.B., Buxton, H.T. Pharmaceuticals, hormones, and other organic wastewater contaminants in U.S. streams, 1999-2000: A national reconnaissance (2002) Environmental Science and Technology, Vol. 36, No. 6, pp. 1202-121.
This article was cited 3,279 times (as of July 2014).

The article was written by US Geological Survey researchers who utilized five newly developed analytical methods to measure concentrations of 95 OWCs (organic wastewater contaminants) in water samples from a network of 139 streams across 30 states during 1999 and 2000. This study represented the first national-scale investigation of pharmaceuticals and other OWCs in streams of the U.S. The results of the study demonstrate the prevalence of pharmaceuticals and other OWCs in U.S. streams and the importance of obtaining data on metabolites to fully understand not only the fate and transport of OWCs in the hydrologic system, but also their ultimate overall effect on human health and the environment. As it touches on a wide range of environmental issues, this article is cited by articles in Chemistry, Agriculture, Medicine, Earth Sciences and so forth. However, it is worth noting its citations in law and regulations articles which fall under Social Sciences (28) as well as Economy and Business related articles which look at policy issues related to OWCs (29).

Comments from Mr. Kolpin:

"Yes, I was aware that our ES&T article from 2002 was being highly cited by the scientific community. In fact, this research was noted as the most frequently cited paper in the field of environmental science since 2010 and was prominently used in the article “Top-cited articles in environmental sciences: Merits and demerits of citation analysis” (Khan, M.A. and Ho, Y-S., Sci. Total Environ., v. 431, p. 122-127).

There are probably multiple factors for the number of citations this paper has received, but I think the primary reason is that it has turned out to be a seminal paper on the occurrence of contaminants of emerging concern (CECs) in water resources and was the first national-scale study of such compounds conducted in the United States. If you look at the number of papers published annually on the topic of CECs you can see that since 2002 (the year our paper was published) there has been a continual and dramatic increase in the number of papers being published each year. This increasing trend in CEC papers published annually documents the ever increasing interest by the scientific community in the rapidly evolving topic of CECs. Thus, even though the percentage of papers citing our 2002 ES&T papers may be slowly decreasing with time it is likely being offset by the total number of papers being published on the topic (keeping the number of citations for our 2002 paper at a healthy pace)."



The top cited article between 2001 and 2011 in the field of Medicine is:
Rossouw, J.E.,  Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the women's health initiative randomized controlled trial (2002) Journal of the American Medical Association, Vol. 288, No. 3, pp. 321-333.
This article was cited 9,723 times (as of July 2014).

The paper assesses the major health benefits and risks of the most commonly used combined hormone preparation estrogen plus progestin in the United States and found that the overall health risks exceeded benefits from use of combined hormone preparation. The study was conducted by a group of scientists from the Division of Women's Health Initiative at the National Heart, Lung/Blood Institute in the USA.

This article is seen to be cited in disciplines other than medicine-related ones, including Social Sciences and Arts & Humanities. Although the article reports on a specific experiment related to drug prescription and its effect on women’s health, it evoked a wider discussion which is seen in studies relating to health policy, women psychology and narratives relating to menopause (30, 31).

Comments from Prof. Rossouw:

"We are aware that this article was and continues to be highly cited. The findings overturned many decades of conventional wisdom, in particular that hormone therapy would prevent cardiovascular disease and that the benefits would outweigh the risks. As a result of this perception of benefit, menopausal hormone therapy was being prescribed to millions of women for chronic disease prevention in addition to its established role in treatment of vasomotor symptoms. After the contrary findings were published, prescriptions for estrogen plus progestin hormone therapy declined by 75% in the first 18 months and have continued to decline. Nationally, breast cancer rates have declined in parallel with hormone prescriptions. In short, the article had a substantial impact on medical practice and on public health."



It is noticeable that 4 out of the 10 articles featured here describe the development of computer software. The practice of citing computer software when used in a study is a part of this phenomenon. Regardless of the subject field, the computational tools developed and written about are highly cited.

Out of the 10 selected articles, 6 are the result of a scientific collaboration between two or more researchers. Collaboration is seen across institutions and countries which could be a result of a common global concern to damaging phenomena related to the environment.

The analysis of citing disciplines shows that research, regardless of its disciplinary origin, crosses subject-specific domains and has impact on a wide range of areas, some of which are quite surprising. It is plausible that the growing ability of researchers to be exposed to and read a wider range of literature encourages the transfer of knowledge from one discipline to another.


Subject Article Link
Agricultural and Biological Sciences MEGA4: Molecular Evolutionary Genetics Analysis (MEGA) software version 4.0
Arts and Humanities The complexity of intersectionality
Biochemistry, Genetics and Molecular Biology Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCT method
Business, Management and Accounting User acceptance of information technology: Toward a unified view
Chemical Engineering Processable aqueous dispersions of graphene nanosheets
Chemistry UCSF Chimera - A visualization system for exploratory research and analysis
Computer Science Distinctive image features from scale-invariant keypoints
Decision Sciences To parcel or not to parcel: Exploring the question, weighing the merits
Earth and Planetary Sciences First-year Wilkinson Microwave Anisotropy Probe (WMAP) observations: Determination of cosmological parameters
Economics, Econometrics and Finance Evolving to a New Dominant Logic for Marketing
Energy Geant4 developments and applications
Engineering The rise of graphene
Environmental Science Pharmaceuticals, hormones, and other organic wastewater contaminants in U.S. streams, 1999-2000: A national reconnaissance
Immunology and Microbiology Improved prediction of signal peptides: SignalP 3.0
Materials Science The SIESTA method for ab initio order-N materials simulation
Mathematics A fast and elitist multiobjective genetic algorithm: NSGA-II
Medicine Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the women's health initiative randomized controlled trial
Neuroscience Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain
Pharmacology, Toxicology and Pharmaceutics Minimal criteria for defining multipotent mesenchymal stromal cells. The International Society for Cellular Therapy position statement
Physics and Astronomy Statistical mechanics of complex networks
Psychology Using thematic analysis in psychology
Social Sciences User acceptance of information technology: Toward a unified view
Veterinary Reproductive Loss in high-producing dairy cattle: Where will it end (ADSA foundation scholar award)
Dentistry Stem cell properties of human dental pulp stem cells

Table 1 - Full List of Top Cited Articles in Scopus (Data Collected July 2014)



(3) Bon, C., Berthonaud, V., Fosse, P., Gély, B., Maksud, F., Vitalis, R., Philippe, M., van der Plicht, J., Elalouf, J.-M. (2011) “Low regional diversity of late cave bears mitochondrial DNA at the time of Chauvet Aurignacian paintings”.Journal of Archaeological Science, Vol. 38, No.8, pp. 1886-1895. Retrieved from
(4) Oliveira, H.R., Civáň, P., Morales, J., Rodríguez-Rodríguez, A., Lister, D.L., & Jones, M.K. (2012) “Ancient DNA in archaeological wheat grains: Preservation conditions and the study of pre-Hispanic agriculture on the island of Gran Canaria (Spain)”,Journal of Archaeological Science, Vol. 39, No.4, pp. 828-835. Retrieved from
(5) Eric, W. (2010) “Do languages originate and become extinct at constant rates?”,Diachronica, Vol. 27, No.2, pp. 214-225.
(7) Jyrkinen, M. (2014) “Women managers, careers and gendered ageism”, Scandinavian Journal of Management, Vol. 30, No.2, pp. 175-185.
(8) Kokot, P. (2014) “Structures and relationships: Women partners' careers in Germany and the UK”, Accounting, Auditing and Accountability Journal, Vol. 27, No.1, pp. 48-72.
(9) Levac, L. (2013) “'Is this for real?' participatory research, intersectionality, and the development of leader and collective efficacy with young mothers”, Action Research, Vol. 11, No. 4, pp. 423-441.
(10) Carr, E.R., & Thompson, M.C. (2014) “Gender and climate change adaptation in agrarian settings: Current thinking, new directions, and research frontiers”, Geography Compass, Vol. 8, No. 3, pp. 182-197.
(11) Júlíusdóttir, M., Skaptadóttir, U.D., & Karlsdóttir, A. (2013) “Gendered migration in turbulent times in Iceland”, Norsk Geografisk Tidsskrift, Vol. 67, No.5, 266-275.
(12) Zhou, J., Rau, P.-L.P., & Salvendy, G. (2014) “Older adults’ use of smart phones: An investigation of the factors influencing the acceptance of new functions”,Behaviour and Information Technology, Vol. 33, No.6, pp. 552-560.
(13) Liew, E.J.Y., Vaithilingam, S., & Nair, M. (2014) “Facebook and socio-economic benefits in the developing world”, Behaviour and Information Technology, Vol. 33, No.4, pp. 345-360.
(14) Saleh, A.M., Haris, A., & Ahmad, N. (2014) “Towards a UTAUT-based model for the intention to use solar water heaters by Libyan households”, International Journal of Energy Economics and Policy,Vol. 4, No.1, 26-31.
(15) Abdekhoda, M., Ahmadi, M., Dehnad, A., & Hosseini, A.F. (2014) “Information technology acceptance in health information management”, Methods of Information in Medicine, Vol. 53, No.1, pp. 14-20.
(19) Wu, H., Wang, D., Shi, L., Wen, Z., & Ming, Z. (2014) “Midsagittal plane extraction from brain images based on 3D SIFT”,Physics in Medicine and Biology, Vol. 59, No.6, pp. 1367-1387
(20) Laeng, B., Bloem, I.M., D'Ascenzo, S., & Tommasi, L. (2014) “Scrutinizing visual images: The role of gaze in mental imagery and memory”, Cognition, Vol. 131, No.2, pp. 263-283.
(21) Liu, H., Liu, Y., & Sun, F. (2014) “Traffic sign recognition using group sparse coding”,Information Sciences, Vol. 266, pp. 75-89.
(23) Tendeiro, D., Lopes, G., Vieira, P., & Santos, J.P. (2014) “Monte Carlo simulation of laser beams interaction with the human eye using Geant4”,BioMedical Engineering Online, Vol. 13, No. 1.
(25) Arora, S.K., Youtie, J., Shapira, P., Gao, L., & Ma, T.T. (2013) “Entry strategies in an emerging technology: A pilot web-based study of graphene firms”, Scientometrics, Vol. 95, No.3, pp. 1189-1207.
(26) Lehman, K. (2011) “Reviews of science for science librarians: Graphene”,Science and Technology Libraries, Vol. 30, No. 2, pp. 132-142.
(27) Pham, C.H., & Fayerberg, R. (2011) “Current trends in patenting graphene and graphene-based inventions”,Nanotechnology Law and Business, Vol. 8, No.1, pp. 10-17.
(28) Brands, E. (2014) “Siting restrictions and proximity of concentrated animal feeding operations to surface water”,Environmental Science and Policy,Vol. 38, pp. 245-253.
(29) Halden, R.U. (2014) “On the need and speed of regulating triclosan and triclocarban in the United States”,Environmental Science and Technology,Vol. 48, No.7, pp. 3603-3611.
(30) Bird, C.E. (2014) “Will extending the women's health initiative lead to better research and policy?”, Women's Health Issues,Vol. 24, No. 1, e3-e4.
(31) Nosek, M., Kennedy, H.P., & Gudmundsdottir, M. (2012) “Distress during the menopause transition: A rich contextual analysis of midlife women's narratives”, SAGE Open, Vol. 2, No.3, pp. 1-10.
(32) Wilson, S.M., DeMarco, A.T., Henry, M.L., Gesierich, B., Babiak, M., Mandelli, M.L., Miller, B.L., Gorno-Tempini, M.L. (2014) “What role does the anterior temporal lobe play in sentence-level processing? Neural correlates of syntactic processing in semantic variant primary progressive aphasia”, Journal of Cognitive Neuroscience,Vol. 26, No.5, pp. 970-985.
(33) van der Laan, L.N., de Ridder, D.T.D., Charbonnier, L., Viergever, M.A., & Smeets, P.A.M. (2014) “Sweet lies: neural, visual, and behavioral measures reveal a lack of self-control conflict during food choice in weight-concerned women”, Frontiers in Behavioral Neuroscience,Vol. 8 (MAY).
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

A decade’s trends in virology research

Matthew Richardson illustrates the trends that have influenced the field of Virology, the study of viruses, over the past 10 years, using bibliometric analysis and visualization techniques.

Read more >

One advantage bibliometric analysis brings is the ability to put a large quantity of research into perspective. Papers can of course be read individually, and the use of cited references in the literature allows an interested reader to get a wider background on the specific concepts found within, and how the understanding of these has changed over time. However, the sheer scale of work produced in a given field means that the only way to illustrate the broadest trends affecting an entire field is through analyzing the bibliographic data of these papers in bulk. In this article we illustrate the trends that have influenced the field of Virology, the study of viruses, over the past 10 years.

Visualizing the topics in Virology

In an earlier issue of Research Trends we introduced term maps as a method for exploring the topics published in a group of journals (1). These maps, developed in collaboration with the CWTS research group, present a two-dimensional view of the topical terms used in the titles and abstracts of a publication; when aggregated across a journal, or a large group of journals, you can then make use of the fact that a term is more likely to appear in the same paper as a related term to group together those which are most highly related. Using all of the textual data available in titles and abstracts, this allows you to produce a thorough view of which topics are researched and how they interact with one another to form the broader structure of a field.

In the term maps following, we use all journals that are categorized in Scopus within the Virology subject category. Although it is still possible that virology-related content is published outside these journals, for instance in a broad-based Medicine or Microbiology journal, this analysis catches the great majority of relevant research across a wider range of journals than a small selection would allow.

As we wish to compare the field at a gap of 10 years’ time, we have used the two time periods 2000–02 and 2010–12. The use of three consecutive years of publications in each map allows us to obtain a more thorough view of what is being published, and so to use more accurate co-occurrence relationships between terms in the maps.

The term maps and selection of topics

Figure 1 shows a term map for Virology content published in the years 2000–02. This covers 14,158 articles, reviews and conference papers. This map is a co-occurrence cluster map, showing both the position of each term (the relative location is determined by their co-occurrence in title and abstracts, so that the closer the terms are positioned the more often they tend to co-occur) and the main cluster they belong to (distinguished by one of four colors). The final element of the data shown is the frequency with which a term is found in this field: the larger the term appears, the more papers contain that term within the title or abstract.

Virology fig 1

Figure 1 – Journal term co-occurrence map for the field of Virology, using a set of 14,158 papers published from 2000 to 2002. Colors used to distinguish clusters of related terms. Data source: Scopus

This map forms a circular structure which is common to many such networks, and is composed of four main groupings of topics. The most common terms are those relating to primary care and clinical research in the green cluster (‘patient’, ‘case’, ‘therapy’); epidemiology, outbreak investigation and phylogenetics in the blue cluster (‘isolate’, ‘genotype’, ‘phylogenetic analysis’, ‘outbreak’); molecular biology and genetics in the red cluster (‘transcription’, ‘open reading frame’, ‘nucleotide’), and cell biology of disease in the yellow cluster (‘T cell’, ‘IFN’, ‘CD4’).

Figure 2 shows a term map based on the same selection of journals, 10 years later: this includes 24,691 Virology papers published in 2010–12. This represents a huge increase in content over the earlier time period, with more than 10,000 additional papers. As might be expected, similar phrases appear as common terms: for instance, ‘patient’, ‘domain’, ‘case’, ‘isolate’.  More interesting are the broader changes in the structure of the field, and changing trends in the less frequent, more specific topics. Topics such as HCV (hepatitis C virus) and HPV (human papillomavirus) are far more visible in the center of the map, pointing to the increasing quantity but also interdisciplinarity of this research.

While the main clusters remain present and intact in this later map, the circular structure is not as contained; the green cluster relating to primary care and clinical research, and the yellow cluster relating to cell biology of disease, no longer link together quite so closely as in the 2000–02 period. This finding is surprising, given that in recent years we have seen a strong focus on interdisciplinary research, translational medicine and closing the loop between ‘bench’ research and ‘bedside’ care.

Virology fig 2

Figure 2 – Journal term co-occurrence map for the field of Virology, using a set of 24,691 papers published from 2010 to 2012. Colors used to distinguish clusters of related terms. Data source: Scopus


In Figure 3, selected virus-related terms have been identified and annotated on the 2010–12 Virology map. Rather than being confined to any particular cluster, these virus topics are scattered throughout the map according to the types of papers they occur in most frequently. This finding illustrates the fact that different virus families are predominantly used in very different kinds of studies, relating to the different clusters of the map. Related terms appear close to one another, as expected: for instance, hepatitis B and hepatitis C are close to one another, in the green (clinical) cluster, while influenza A is towards the top of the map along with the subtypes H5N1 and H1N1.


Virology fig 3

Figure 3 – Journal term co-occurrence map for the field of Virology, using a set of 24,691 papers published from 2010 to 2012. Colors used to distinguish clusters of related terms and annotations provided for selected virus-related terms. Data source: Scopus


As demonstrated here, term maps provide a useful overview of a field and allow you to examine the broader structural changes that affect it over time. In contrast, in the analysis that follows SciVal is used for more detailed analysis of individual topics with various metrics.


Research trends in the past decade

Taking some of the virus terms identified from our term map, it is possible to construct research areas in SciVal based around these topics and then compare them to one another by a variety of measures. One example is provided in Figure 4: here we see trends in scholarly output from 2004 to 2013 for five different research areas, covering research on hepatitis B and C, human papillomavirus, the H1N1 strain of influenza A, and coronavirus. The first three were included as they show high quantities of research but also extremely strong growth throughout the decade.  H1N1 on the other hand starts with minimal activity but then grows quickly to a peak of 568 papers in 2011. This growth in activity follows the 2009-10 H1N1 (swine flu) pandemic (2). Coronavirus research follows a different trend: while it starts relatively high in 2004 with more than 600 papers, it then declines steadily until there were fewer than 300 papers published in 2011. After this point there is another increase in activity, with 395 papers in 2013. The two different periods of higher interest in coronaviruses seem likely to be related to two distinct viruses: first SARS-CoV, a global epidemic which occurred in 2002–03; and towards the end of the period MERS-CoV, which was first identified in 2012 (3, 4).


Virology fig 4

Figure 4 – Trends in scholarly output for a selection of virus-related topics, counting articles, reviews and conference papers published per year. Source: SciVal


Field-weighted citation impact (FWCI) is a citation metric showing the citation activity around a group of papers, taking into account subject field, article type and year of publication, and so offering a robust comparison to the expected level of citation impact (which is assigned a level of 1.0). Looking across the full set of virus topics highlighted in Figure 3, three in particular stand out as having extremely strong spikes of citation impact in the past decade: the influenza A subtypes H5N1 and H1N1, and coronavirus. These times of activity coincide with the timing of public outbreaks even more closely than the publication trends shown in Figure 4. The year 2004, in which H5N1 research has an FWCI of over 10 times the expected level, saw major outbreaks of the virus strain across Asia (5, 6);  2009, in which H1N1 research reached an FWCI of 9.33 times the expected value, saw cases of the virus affecting people in the US and around the world (2); and the coronavirus MERS-CoV was first identified in 2012, coinciding with an upturn in impact continuing into 2013 and 2014 (which shows early signs of a similarly high FWCI but is not shown here due to the incompleteness of the data) (4).


Virology fig 5

Figure 5 – Trends in field-weighted citation impact for a selection of virus-related topics. Source: SciVal



While the publication and citation trends shown for specific virus topics reflect wider public interest at times of virus outbreaks, bibliometric analysis such as shown in this article allows for detailed comparison of the amount of research in different areas but also the way it is carried out. The insights available through term maps are even more difficult to draw from mainstream media or individual scholarly papers; using these visualizations we can view the full structure of a subject area and see how this has changed over time. Virology, a fast-moving field with topics that naturally rise and fall in interest as outbreaks occur, is particularly apt for this kind of illustration of hot topics over time.



(1) Van Weijen, D. (2013) “Trends in pediatrics: Overview of research trends from 2007–2011”, Research Trends, Issue 34, September 2013. Available at:


VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Graphene: ten years of the ‘gold rush’

In this article, Andrew Plume investigates whether a new approach to assigning ‘credit’ for article authorship can answer the question: “Who are the authors of high-impact graphene research”?

Read more >

Since the publication of the famous paper (1)on the ‘sticky-tape method’ for preparing graphene in October 2004 (which helped win authors, Andre Geim and Konstantin Novoselov the 2010 Nobel Prize in Physics), the field of graphene research has seen phenomenal growth in terms of published research articles likened to a ‘gold rush’ (2). With the entrance of so many new and established researchers into the field, we investigate if a new approach to assigning ‘credit’ for article authorship can answer the question: “Who are the authors of high-impact graphene research”?

Graphene is a material comprising carbon atoms packed together in a two-dimensional sheet just one atom thick, and may be the thinnest material in the universe. This unique structure gives graphene some very surprising physical properties – it is some 100 times stronger than steel and conducts heat and electricity at high efficiency. Prior to its isolation by Geim and Novoselov in 2004, it existed only in theoretical models; as such, the field of graphene research can be considered to have appeared almost overnight.

Figure 1 shows the exponential increase in the number of research articles published on graphene in the decade between 2004 and 2013. Using this corpus of literature as a self-defining research field, we have applied a recently-published method for assigning authorship credit to understand who the high-impact authors in graphene research are. Most current approaches to identifying and ranking high-impact authors fail to account for the invisible credit structures which operate in author bylines in most fields of research. Instead, most analyses assume that each author has a full and equal stake in the creation of a research article, and this follows to the assignment of the credit for that article also. While much previous work has been done to examine the intricacies of fractional assignment of credit to authors (e.g. Moed (3) and Stallings et al. (4)), there has recently been renewed interest in algorithmic methods to fractionally assign authorship credit in a way that recognises these unstated community norms. Some of the most recent work along these lines has been published by Nils T. Hagen at the University of Nordland, Norway, and it is this approach which serves at the inspiration for the present study (5).

Graphene fig1

Figure 1 - Scholarly output (articles only) published in the period 2004-13 from a search for “graphene” in the titles, abstracts or keywords. Source: SciVal.

The present study aims to compare three methods of assigning authorship credit to the authors of the corpus of research articles on graphene defined above and examine the differences in the resulting lists of high-impact researchers. The first method is the standard ‘full count’ method – each author on the article receives a full count for each article they appear on, and also the full citation credit. The second method is ‘fractional’, where each author gets an equal portion of the credit with all other co-authors; an author on a single-author paper gets 1, while one on a 4-author paper gets 0.25; citation credit is assigned in the same way. For an examination of the rise of fractional authorship over time, see “Publish or perish? The rise of the fractional author…” , also in this issue (6).  Finally, the ‘harmonic’ method (as developed by Hagen, (5)) instead assigns additional weight to the first and last authors and diminishing weights to each additional author in the middle, and assigns citations same way also. As a vital and important research front, graphene research is typically published in well-known peer-reviewed journals and as such we have assumed that all of the most important research (and researchers) in this topic are represented in the Scopus database.

Citations in this analysis are counted on a 3-year basis; i.e. citations to each article are counted in the same year as publication plus the two following years; i.e. 2011 papers have their cites counted in the period 2011-2013; since the field is therefore self-defining, it is not necessary to field-weight the citation data as we may assume that citation practices within graphene research are reasonably homogenous. Because of the use of this 3-year citation window, this analysis considers only those articles published from 2005 to 2011, focussing on the period of expansion of the field in the wake of Geim and Novoselov’s landmark 2004 publication (1). Importantly, since the corpus is defined as research articles containing the word “graphene” in the title, abstract or keywords, it ignores all other articles on non-graphene topics published by the same authors; by design, these results answer the very specific question “who are the authors of high-impact graphene research?”, and not “who are the high-impact authors working on graphene?

If each author on every paper is represented in this analysis, when these lists are sorted by citations per article many of those appearing at the top are authors of single well-cited papers who may not (yet) represent career researchers. To account for this, a productivity threshold was applied to allow authors with relatively lower productivity in graphene research to appear in these lists; in Figure 2 this was set at a relatively ‘relaxed’ minimum of 7 articles in the 7 year period 2005-11 (i.e. on average, 1 article per author per year) for the full count method, and at 2 authorship credits for the fractional and harmonic methods (i.e. on average, less than 0.3 article credits per author per year).

Graphene fig2

Figure 2 -  Top 25 authors of graphene articles 2005-11: ‘relaxed’ productivity threshold. Source: Scopus.

It is clear from a glance that while the three methods have a few authors in common, where the same author does appear in more than one list their rankings are quite variable (see for instance the variability in ranking of the two Nobelists Geim and Novoselov in each list, for example). It is also clear that at this ‘relaxed’ productivity threshold, authors who are newer to the field are likely to appear but may not be as well-recognised as leading figures in the field by other graphene researchers.

In Figure 3, the productivity threshold was increased to focus only on authors with relatively high productivity in graphene research; this ‘stringent’ threshold was set at a minimum of 28 articles in the 7 year period 2005-11 (i.e. on average, 4 articles per author per year) for the full count method, and at 7 authorship credits for the fractional and harmonic methods (i.e. on average, 1 article credit per author per year). In these lists there is a somewhat greater degree of agreement between the results overall than in the ‘relaxed’ threshold lists, but especially for the very top names (the two Nobelists head all three lists, for example); below that, the three lists begin to differ and names in one or two lists are absent from the other(s).

Graphene fig3

Figure 3 - Top 25 authors of graphene articles 2005-11: ‘stringent’ productivity threshold. Source: Scopus.

It is difficult for anyone not working directly in a field of research to know who the ‘best’ researchers working in that field are, and recognising this we have not sought to make a value judgement here on the which list correlates most closely with peer esteem. Instead, the question remains open to those working on graphene to answer: which researchers are recognised as the ‘highest impact’ in the field, and which list reflects this most closely?

As early as 2008, Andre Geim himself has noted the tendency for graphene to attract large numbers of researchers: “With graphene, each year brings a new result, a new sub-area of research that opens up and sparks a gold rush” (6). Here we have applied a fresh approach to assigning author credit for published research articles to the field of graphene as one way of demonstrating who has made their fortune on the research frontier. It is important to note however that, owing to the inherent complexity in the research enterprise (especially at the frontier of knowledge), simplistic interpretations of author rankings may be dangerous insofar as they may reinforce the status quo and lead to a form of consensus-reaching which may ultimately limit the expansion of knowledge. Instead - as always - metrics informed by expert opinion are preferable.



(1) Novoselov, K.S., Geim, A.K., Morozov, S.V., Jiang, D., Zhang, Y., Dubonos, S.V., Grigorieva, I.V., Firsov, A.A. (2004) “Electric field in atomically thin carbon films”, Science, vol. 306, issue 5696, pp. 666-669.
(2) Plume, A., (2010) Buckyballs, nanotubes and graphene: On the hunt for the next big thing, Research Trends issue 18, July 2010,
(3) Moed, H.F. (2000) “Bibliometric Indicators Reflect Publication and Management Strategies” Scientometrics 47(2) pp. 323-346
(4) Stallings, J., Vance, E., Yang, J., Vannier, M.W., Liang, J., Pang, L., Liang Dai, Ye, I., and Wang, G. (2013) “Determining scientific impact using a collaboration index”, Proceedings of the National Academy of Sciences (doi:10.1073/pnas.1220184110)
(5) Hagen, N.T. (2014) “Counting and comparing publication output with and without equalizing and inflationary bias” Journal of Informetrics 8(2) pp. 310-317.
(6) Plume, A. & Van Weijen, D. (2014) Publish or perish? The rise of the fractional author…, Research Trends Issue 38, September 2014. 
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Publish or perish? The rise of the fractional author…

Andrew Plume and Daphne van Weijen investigate how the pressure researchers feel to publish their work has affected co-authorship patterns over the past 10 years. Are researchers publishing more unique articles or co-authoring more articles?

Read more >

“Publish or perish” is a common phrase used to describe the pressure researchers feel to publish their research findings in order to stay relevant and be successful within the academic community. It’s been around a very long time, although the origins of the phrase are somewhat unclear. Some researchers attribute the phrase to Kimball C. Atwood III, who is said to have coined the phrase in 1950 (1, 2). A 1996 article by Eugene Garfield (3) traces the phrase back to at least 1942, while according to Wikipedia (4) the term was used even earlier, in a 1932 non-academic book by Harold Jefferson Coolidge (5). The phenomenon has become a focus of academic research itself, as a search for the phrase in Scopus retrieved 305 documents published on the topic from 1962 to date. On average, more than 20 articles per year were published on the topic over the past 5 years (2009 – 2013), with 37 articles alone published in 2013. Nonetheless, it seems clear that researchers suffer from this phenomenon on an increasing scale.

One common belief is that as a result of the rise of the “publish or perish” culture, and in order to remain successful in academia, each researcher is publishing more and more articles every year. But is this true? Are researchers publishing more unique articles or co-authoring more articles? One of the earliest studies in our literature search that tried to answer this question, by F.P. De Villiers, was published in 1984 and focused on changes in authorship in the South African Medical Journal from 1971 to 1982 (6). Results of the study indicated that:

“the mean number of authors per article increased from 1.77 in 1971 to 2.35 in 1982, while the proportion of articles with only 1 author decreased from 60.8% to 40.8%. Possible reasons for this are mentioned, of which the pressure to publish may not be the least.” (6).

Although this sounds intuitively plausible, these results were restricted to articles published in a single journal, and in only one research area, about 30 to 40 years ago. Since then, we’ve seen an increase in papers authored by an extremely large number of researchers, most notably the ATLAS collaboration papers published in 2008 (2,926 authors) (7) and 2012 (3,171 authors) (8) and a Nature article on the Initial Sequencing and Analysis of the Human Genome by the International Human Genome Sequencing Consortiumwith about 2,900 authors published in 2009 (9). But the question remains how researchers are currently dealing with the increased pressure to publish. In other words, are individual researchers actually writing more articles every year, or are there just more authors writing more collaboratively? To answer this question we collected trend data from Scopus for 2003 – 2013 and checked different characteristics of authorship patterns over time; the data simply counted the number of articles (articles, reviews and conference papers) published each year and the count of authorships and unique author names associated with these. Here we use the term ‘authorships’ to define the occurrence of an individual on an article, while the concept of a ‘unique author’ reflects an individual who has appeared on one or more articles in a given period (here a single year).


Main findings

Results of our analysis show that there has been a consistent growth in the number of articles published over the past decade; from 1.3 million in 2003 to 2.4 million in 2013 (see Figure 1). At the same time, the number of authorships has increased at a far greater rate from 4.6 million in 2003 to 10 million in 2013.

Authorship fig1

Figure 1 - Growth in volume of articles published, authorships and unique authors from 2003 – 2013. Source: Scopus.


Over the past ten years or so, the number of authorships per unique author (2.31 in 2013) has increased while the number of articles per unique author (0.56 in 2013) has declined (see Figure 2), while the total number of articles published per year has increased (see Figure 1). At the same time, the average number of authorships per article has increased from 3.5 to 4.15 authors from 2003 to 2013, which suggests that authors are collaborating and co-authoring more now than they were 10 years ago. (At the same time, the percentage of single authored papers has declined from 20% in 2003 to 13% in 2013; data not shown).

 Authorship fig2

Figure 2 -  Authorship patterns over time (2003 – 2013). Source: Scopus.


In other words, the number of authorships per article is rising: 10 years ago, an average paper had about 3.5 authors, now it has over 4 authors. This rise in ‘fractional authorship’ (the claiming of credit for authorship of a published articles by more than one individual) is most likely driven by research collaboration, and is an efficient mechanism by which each author can increase their apparent productivity from the same underlying research contributions (i.e. articles per unique author) of 0.56 articles per unique author per year.

This means that a single author can produce a single authored article once every two years or a co-authored article with one other author every year. Now, with the rise of ‘fractional authorship’ or fractional contributions to papers, we’re seeing that the way in which authors are using this half a paper’s capacity per year is changing. A given author may achieve this output by appearing as ninth author on 5 different paper (5 x 0.1 authorships per paper), instead of co-authoring as second author on a pair of 4-author papers per year (2 x 0.25 authorships per paper).

These findings build on earlier observations (10) in which the increases in authorships per article (at 1.9% mean annual growth rate in the period 1980-2002), authorships per unique author (at 1.2%) were contrasted by a decline in article per unique author (at -0.7%). In the current data, the comparable rates are 1.8%, 0.9% and -0.8%; suggesting the continuation of a long-term trend stretching back not just one decade but at least three.

These findings are confirmed by research in several specialty fields, including software engineering, where the average number of authors per paper has risen on average by about 0.4 authors per decade from 1970 to 2012 (11), and pediatric surgery, which has seen a marked increase in papers authored by 6 or more authors and also in multi-national papers (12).

If each active author does not increase their fractional article output each year, what is driving the observed volume increase in research outputs globally? Here, the answer is quite simple – the research workforce is growing at a similar rate year-on-year to the volume of article production (at about 3-4% p.a.; data not shown), and so new entrants into research fields are responsible for creating new knowledge which eventually sees publication in the peer-reviewed literature.


Despite opinions to the contrary, these data suggest that there has been no apparent increase in overall productivity per active author over the last decade. Instead, authors are using their authorship potential more wisely by becoming more collaborative in the way they work, which is driving an apparent inflation in each author’s productivity as well as author bylines. Instead, the underlying driver of the volume increase in articles published is simply the introduction of new entrants/authors into the market. That is not surprising, as the total population of researchers globally continues to rise every year, and they become increasingly subject to the principles of "publish or perish": and so the cycle continues.



(1) Research Trends (2010) Did you know…  “Publish or perish” has been worrying researchers for 60 years?  Research Trends, issue 16, March 2010.
(2) Sojka, R.E. and Mayland, H.F. (1991) Driving Science With One Eye On the Peer Review Mirror
(3) Garfield, Eugene, (1996). "What Is The Primordial Reference For The Phrase 'Publish Or Perish'?"The Scientist 10 (12): 11.
(4), accessed July 7th, 2014.
(5) Coolidge, Harold Jefferson, (1932) Archibald Cary Coolidge: Life and Letters, p. 308 (source: Wikipedia). 
(6) De Villiers, F.P. (1984) South African Medical Journal, Volume 66, Issue 23, 8 December 1984, Pages 882-883.
(7) The ATLAS Collaboration et al (2008). JINST 3 S08003 doi:10.1088/1748-0221/3/08/S08003
(8) The ATLAS Collaboration et al (2012). Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC, Physics Letters B, Volume 716, Issue 1, 17 September 2012, Pages 1–29, DOI: 10.1016/j.physletb.2012.08.020.
(9) International Human Genome Sequencing Consortium(2009). Initial sequencing and analysis of the human genome,Nature, V412, 565.
(10) Moed, H.F. (2005). Citation Analysis in Research Evaluation. Dordrecht (Netherlands): Springer. ISBN 1-4020-3713-9, 346 pp.
(11) Fernandes, J.M. (2014) Authorship trends in software engineering. Scientometrics, DOI 10.1007/s11192-014-1331-6.
(12) Pinter, A. (2014), Changing Authorship Patterns and Publishing Habits in the European Journal of Pediatric Surgery: A 10-Year Analysis, European Journal of Pediatric Surgery, [Epub ahead of print]
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

A quick look at references to research data repositories

In this contribution, Sarah Huggett investigates whether there is a way to estimate the visibility of research data in the published literature, and presents some initial findings.

Read more >


While published papers are one of the most visible outputs of the research process, in a way they are only the tip of the iceberg: the research workflow is composed of much more than meets the eye of the external observer (see Figure 1).

Data fig 1

Figure 1 - Researcher Workflow. Source: Elsevier’s Response to HEFCE’s call for evidence: independent review of the role of metrics in research assessment


Most scholarly research uses data in one guise or another, and recently there have been calls for data to become more systematically visible research output rather than remain a background variable of academic endeavors. For instance, the Force 11 community movement, which aims to support the advancement of scholarly communications, has issued eight Data Citation Principles that stress the importance of data being “considered legitimate, citable products of research” (1). These principles highlight main citation issues, such as access, unique identification, and interoperability and flexibility. Research Trends’ curiosity was piqued: could there be a way to estimate the visibility of research data in the published literature?



Researchers may make data available in data repositories, and authors may subsequently reference these data in their scholarly outputs. So how could these data citations be analyzed?

One of the challenges mentioned by Force11 is unique identification: researchers may refer to datasets they cite by various names; however, the web addresses of the repositories in which the data reside can be used as reliable identifiers. So first, a list of data repositories was needed; this was extracted from databib (a website describing itself as “a searchable catalog / registry / directory bibliography of research data repositories”) in June 2014. This yielded 971 results (see examples in text box) of data repositories in various fields, countries, and of various sizes. Notably nearly half of the listed repositories originate from the USA (see Figure 2).

  Data fig2

 Figure 2 - Geographical distribution of data repositories. Source: databib


Examples of data repositories from the databib list:

1000 Genomes (Thousand Genomes) (A deep catalog of human genetic variation)
DataONE (Data Observation Network for Earth)
Marine Geoscience Data System
Ontario Data Documentation, Extraction Service and Infrastructure (ODESI)
Sloan Digital Sky Survey
World Data Center


Second, papers citing these repositories websites needed to be identified. The Scopus advanced search function allows searching the reference fields of papers for websites, which was done for all URLs on the databib list, truncating the addresses and using wildcards as appropriate. The records of the papers identified as containing the URLs in their reference lists were then extracted.
There are two main potential caveats to this approach:

  • If an author fails to include the website to the references or mentions the website in the full text but not the references, their papers will not be retrieved by this search method.
  • Some of the websites listed by databib are more than just data repositories. If a researcher references the website with a purpose other than data citation, then their paper will still be retrieved by this search method.



This analysis returned 178,909 1996-2014 documents, with a whopping 19% annual growth (CAGR) between 2009-2013, leading to over 30,000 papers in 2013. Most of the documents are articles (113,618 articles with 24% 2009-2013 CAGR), conference papers (37,410 conference papers with 7% 2009-2013 CAGR), and reviews (19,334 reviews with 16% 2009-2013 CAGR) (see Figure 3).

Data fig3

Figure 3 - 1996-2013 documents citing bibdata websites. Source: Scopus


These documents received 1,879,964 citations, and a word cloud of 2013 papers’ document titles (see Figure 4) shows the preponderance of health-related topics.

Data fig 4

Figure 4 - word cloud of words of titles of 2013 documents citing bibdata websites. Source: Scopus and tagxedo



The visibility of research data as estimated by references to data repositories in the published literature has seen strong growth in recent years. The topics covered by these papers are preponderantly centered on health-related issues. This currently topical issue is seeing initiatives aiming to further integrate research data into the more traditional outputs of research that are scholarly communications (1). There are still challenges ahead, in particular regarding unique identification and meta-data integration, which would allow more rigorous and accurate bibliometrics analyses. Nevertheless, with current computational storage capacities and increasing demand from the research community, the future of research data currently appears full of potential promises.



(1)    Force 11 - “Joint Declaration of Data Citation Principles”, accessed at in August 2014.
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

The black eagle soars: Germany’s bibliometric trends 2004-2013

In this piece, Stephanie Oeben and Sarah Huggett take a bibliometric investigate Germany’s research performance during the past decade, and discuss trends in German publication output and its citation impact.

Read more >

Since the scientific revolution, Germany has been a major contender in Science and Technology, and throughout the 19th Century, German was a preponderant language in scholarly communications around the globe. Although two World Wars took their toll on Germany’s scientific progress, in the modern era the country is still the home or birthplace of many Nobel Laureates. In today’s world, Germany remains a major scientific hub, producing over 6% of the world’s scholarly output in 2012, and German scholars are particularly active in disciplines such as Mathematics and Physical Sciences (1). In recent years, the country has seen a fairly steady rise in internal R&D expenditure, approaching 80 billion Euros in 2012 (2). Germany exceeded 10% of the world’s citations in 2012, leading to high relative citation impact of its research in all fields. German research also leads to technological innovations – Germany is second only to the USA in patent citation share (1). In this piece Research Trends takes a bibliometric look at trends in German research during the past decade.


Germany now

In the past five years (2009-2013), 497,212 Germany-based authors published 726,090 papers which were cited 5,045,807 times, resulting in a Field Weighted Citation Impact (FWCI) of 1.43. The country is highly internationally collaborative, with 48.3% of 2013 German scholarly papers resulting from international collaborations (source: SciVal).


Citations accrue to published articles over time, as articles are first read and subsequently cited by other authors in their own published articles. Citation practices, such as the number, type and age of articles cited in the reference list, may also differ by research field. As such, in comparative assessments of research outputs, citations must be counted over consistent time windows, and publication and field-specific differences in citation frequencies must be accounted for. Field-weighted citation impact is an indicator of mean citation impact, and compares the actual number of citations received by an article with the expected number of citations for articles of the same document type (article, review or conference proceeding paper), publication year and subject field. Where the article is classified in two or more subject fields, the harmonic mean of the actual and expected citation rates is used. The indicator is therefore always defined with reference to a global baseline of 1.0 and intrinsically accounts for differences in citation accrual over time, differences in citation rates for different document types (reviews typically attract more citations than research articles, for example) as well as subject-specific differences in citation frequencies overall and over time and document types. It is one of the most sophisticated indicators in the modern bibliometric toolkit. (1)


Germany 2004-2013

Germany has seen increases in international collaboration over time, as have several of its European neighbors (see Figure 1). The UK in particular has seen a higher increase rate than Germany in the past decade: while the UK was less internationally collaborative than Germany in 2004, by 2013, nearly half of its scholarly output (49.7%) was the result of international collaboration. That same year, more than half of the scholarly outputs of France and the Netherlands were internationally collaborative. Meanwhile, Spain and Italy show parallel increasing trends but lower percentages of international collaboration over the whole period, whilst Poland, the least internationally collaborative country selected, shows overall decreases in international collaboration over time, amounting to less than a third of its 2013 output.

Germany fig1

Figure 1 - Germany and selected European countries’ 2004-2013 international collaboration percentages. Source: SciVal (Scopus data)

Germany’s scholarly output has grown to reach 137,865 papers in 2013. Among its selected European neighbors it is second only to the UK, which published about 10,000 more papers that same year. Other selected European countries also see growth over time, but their scholarly outputs remain significantly below that of Germany and the UK (see Figure 2).

Germany fig2

Figure 2 - Germany and selected European countries’ 2004-2013 scholarly output. Source: SciVal (Scopus data). (Note: Owing to usual indexing lags for some recently-published content at the time of data extraction (mid 2014), the 2013 data point may not reflect a complete view of the final 2013 publication outputs of each country shown).

Some of the German outputs show high and increasing citability; for instance, German publications that are amongst the top 1% cited papers rose strongly over time, to reach nearly 2.4% of the country’s scholarly output in 2013. For comparison, 2.5% of the UK’s scholarly output was in the top 1% cited papers in 2013, and a significantly higher 3.1% of the Netherlands’ (see Figure 3). Germany and the UK have higher absolute numbers of papers in the top 1% cited papers than the Netherlands, but normalizing for output size reveals that a higher proportion of the Netherlands’ scholarly output is in the top 1% cited papers.

Germany fig3

Figure 3 - Proportion of 2004-2013 German and selected European countries’ publications in the top 1% cited papers. Source: SciVal (Scopus data).

Germany’s growth is not limited to the top cited outputs either, as demonstrated by the rising trend of Germany’s FWCI, from an already high 1.27 in 2004 to an impressive 1.49 in 2013. The Netherlands and the UK have higher FWCIs across the whole decade, and so does Italy in 2013 (1.60). Although in 2004 Italy’s FWCI was inferior to Germany’s, it has seen strong increases over the past 10 years, catching up to Germany in 2010 and 2011 before clearly overtaking it in 2012 and 2013, when it even marginally surpassed the UK’s (see Figure 4).


Germany fig4
Figure 4 - Germany and selected European countries’ 2004-2013 FWCI. Source: SciVal (Scopus data).

Finally, looking at the language diversity of scholarly publications, research has shown that non-English outputs tend to have lower citation impact (3). Taken together with the steadily decreasing proportion of German research published in German (see Figure 5), this may help explain some of the increase observed in FWCI.


Germany fig5

Figure 5 - Proportion of German-language German output 2004-2013. Source: Scopus.



Germany’s academic achievements are long-standing, and despite some historical turbulence, Germany has managed to maintain its status as one of the main scientific powers in Europe and on the global scene. Compared to selected European neighbors, Germany remains a solid contender with a robust performance, in particular in terms of output, even though in the last decade it has been overtaken by the UK in terms of international collaboration and by Italy in terms of FWCI. Recent trends such as increases in funding and output bode well for the bibliometrics future of the country, while boosting international collaboration could help further improve the nation’s citation impact (4).



(1)    BIS report
(2)    Federal Statistical Office
(3)    Van Leeuwen, T.N., Moed, H.F., Tijssen, R.J.W., Visser, M.S., van Raan, A.F.J. (2001) “Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance”, Scientometrics, Vol. 51, No. 1, pp. 335-346.
(4)    Science Europe & Elsevier (2013), “Comparative Benchmarking of European and US Research Collaboration and Researcher Mobility”, retrieved from documents and speeches/SE and Elsevier Report Final.pdf; The Royal Society (2011), “Knowledge, networks and nations: Global scientific collaboration in the 21st century”, (J. Wilson, L. Clarke, N. Day, T. Elliot, H. Harden-Davies, T. McBride, … R. Zaman, Eds.) (p. 113). London: The Royal Society. Retrieved from




VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Tracking scientific development and collaborations – The case of 25 Asian countries

Henk Moed and Gali Halevi explain how a country’s current stage of scientific development can be determined through the use of a bibliometric model, and illustrate its use by examining 25 countries in Asia.

Read more >

Bibliometric indicators based on publications in international, peer reviewed journals can be used to characterize the current stage of a country’s scientific development.  A simple bibliometric model for different phases of development of a country’s national research system distinguishes four phases: (1) pre-development; (2) building-up; (3) consolidation and expansion; and (4) internationalization (see Figures 1, 2). The model assumes that during the various phases of a country’s scientific development, the number of published articles in peer reviewed journals shows a more or less continuous increase, although the rate of increase may vary substantially over the years. But a bibliometric indicator measuring the share of a country’s internationally co-authored articles discriminates between the various phases in the development.

  1. Pre-development phase:
    In this phase the level of research activity in a country is low. Research oriented towards the international research front is carried out by a limited number of researchers only.  There is no clear research policy and structural funding of research. Activities result from initiatives by a limited number of active researchers, who may in some years seek collaborations with foreign colleagues. The publication output is low.  From a statistical point of view, indicators are based on low numbers and may show large annual fluctuations. This is especially true for the percentage of internationally co-authored articles.
  1. Building-up phase:
    Researchers in the country start establishing projects with foreign research teams, often funded by foreign or international agencies, and focusing on a particular topic. They begin collaborating with colleagues from more developed countries. Internationally co-authored articles constitute one of the outputs. National researchers enter international scientific networks. The role of the country’s authors in the collaboration is secondary rather than primary. The percentage of internationally co-authored articles relative to a country’s total publication output tends to increase, but is often not statistically significant, due to the fact that the absolute number of annual publications from a country is low, and the internationally co-authored papers may be concentrated in particular years.
  1. Consolidation and expansion:
    The country develops its own scientific infrastructure. The amount of funds available for research increases. The national research capacity increases. Nationally oriented journals internationalize and have a larger probability of being indexed in Scopus and other international scientific literature databases. More and more research papers are based on research carried out by national institutions only. The number of internationally co-authored papers increases as well, but at a rate that is lower than that of the country’s total output; hence, the percentage of internationally co-authored papers declines. 
  1. Internationalization:
    National research capacity is further expanding; research institutions in the country start functioning as fully fledged partners and more and more often take the lead in international collaborations. Overall impact increases; the country’s researcher’s influence the global research agenda; the country more and more becomes one of the world leaders, at least in specific research domains. Both the number of publications and the share of internationally co-authored articles increase.

Asia Figure 1

Figure 1 - Bibliometric model for capturing the state of scientific development


Asia Figure 2
Figure 2 - Schematic overview of trends in bibliometric indicators per development phase.
UNESCO report “Higher Education in Asia – Expanding up, Expanding Out”; p. 80.


Concept   Main questions   Indicators; classifications
Publication output How many articles did a country publish and how did this number develop over time? The number of research articles, reviews and conference papers published in journals and conference proceedings indexed in Scopus during 1997-2012
Disciplinary specialization In which subject field does a country specialize? Use of a subject classification into 26 main disciplines available in Scopus
Distribution by institutional sector How important are the various institutional sectors in research? Use of a classification into 4 institutional sectors: Higher Education; Government; Private; Health
Global and regional collaboration How frequently do Asian countries collaborate with each other and with countries outside the region? Based on the number of articles co-authored by researchers from different countries; calculation of the percentage share of a country’s articles co-authored with researchers working abroad
State of scientific development In what phase of its scientific development is a country? Based on a simple model taking into account the trend in a country’s annual number of publications and the percentage share of internationally co-authored articles

Table 1 - Main bibliometric indicators and classifications used in this study

This study analyzed data on scientific publications for 25 Asian countries (see Table 2) extracted from Scopus, a multidisciplinary database covering publications in 20,000 peer reviewed, mostly international journals. Data on all publications indexed in the Scopus database were organized by country and sorted into three adjacent time periods: (a) 1997-2001, (b) 2002-2007 and (c) 2008-2012. This yielded approximately 6.5 million records for the region as a whole over these three time periods. These publication records were sorted into 26 research disciplines implemented in Scopus. Publications were coded to denote the number of co-authorships among authors from countries in the study set and with authors in other countries outside the studied countries. Publications were further categorized by authors’ type of institutional affiliations, e.g., whether they were affiliated with a higher education institution, government, a private sector organization, or were employed in the health sector. Figure 1 describes the most important indicators and document classifications applied in the following analysis.



Afghanistan Iran South Korea
Bangladesh Japan Pakistan
Bhutan Laos Philippines
Brunei Macao Singapore
Cambodia Malaysia Sri Lanka
China Maldives Thailand
Hong Kong Myanmar Vietnam
India Nepal
Indonesia North Korea

Table 2 - List of countries included in the analysis

Trends in scientific output 1997-2012

Figure 3 shows that there are substantial differences among countries in their average number of publications per year, by up to 400%. Among countries with more than 1,000 papers per year per country, the largest output is from China. However, Iran, Malaysia and Pakistan have a compound annual growth rate above 15 per cent.

Asia Figure 3

Figure 3 - Number and annual growth rate of publications indexed in Scopus 1997-2012
Note: The horizontal axis gives the average number of publications indexed in Scopus per year over the time period 1997-2012 on a logarithmic scale. The vertical axis gives the compound annual growth rate (CAGR) in the number of publications over the same time period. If P1 and P2 denote the number of publications from a country in 1997 and 2012, respectively, CAGR is defined as  - 1.
Source: UNESCO report “Higher Education in Asia – Expanding up, Expanding Out”, p.85.


Scientific output in relation to PhD students enrollment and FTEs

The data below compare two bibliometric indicators – the number of published articles and the number of publishing authors in a year – with two non-bibliometric indicators, namely the number of FTE researchers in a country and the number of doctoral degrees awarded by that country. Figure 4 indicates that the number of publications generated within a country increases in almost linear fashion with the number of doctoral degrees. This suggests that doctoral students play a key role in the production of a country’s publication output in international, Scopus indexed journals.

Asia Figure 4
Figure 4 - Number of publications indexed in Scopus in relation to doctoral enrollment by country (UNESCO, 2006). Note: Publication counts relate to the average number of publications from a country per year during 1997-2012, and the number of doctoral degrees to the most recent year for which data are available (mostly 2011). The dashed line represents the best fit of a power law relationship of the type y. Plotting this functional relationship on a double logarithmic scale, it yields a straight line. The exponent α in the relationship is called the scaling parameter or exponent, and is in a double log plot represented by the slope of the straight line. If α=1, y increases linearly with x. If α>1 y increases superlinearly with x, indicative for a cumulative advantage. If α<1 y increases sublinearly with x, indicative for a cumulative disadvantage. The  value is a measure of the goodness of fit of the power law relationship. It ranges between 0 (no fit) and 1 (perfect fit).
UNESCO report “Higher Education in Asia – Expanding up, Expanding Out”, p.82.

Likewise, the number of authors from a country publishing research articles (at least in Scopus) increases with the number of FTE researchers (Figure 5). Further, research intensive countries, i.e., countries that have a large number of FTE researchers per inhabitant, tend to have a higher share of researchers in the business sector than do less research intensive countries. Since researchers in the business sector tend to publish less in international journals, this factor may explain why the increase in the number of publishing authors has a somewhat weaker relationship to FTE researchers in the country.

Asia Figure 5

Figure 5 - The relationship between the number FTE researchers (UNESCO, 2006) in a country and the number of authors of publications indexed in Scopus. Note: Author counts relate to the average number of publishing authors from a country per year during 1997-2012, and FTE research to the most recent year for which data are available (mostly 2011). For the meaning of the dashed line and the parameters in the functional relationship see the legend of Figure 1. For the full country name corresponding with a country code see Table 1.
UNESCO report “Higher Education in Asia – Expanding up, Expanding Out”, p.83.

International co-authorships

The trend in the percentage of internationally co-authored papers for 13 countries between 2003 and 2011 is presented in Figure 6. Three out of five high income countries such as China, Singapore and Japan, show a positive trend in international co-authorship. Seven out of nine of middle income countries such as India and Indonesia, show a significant decline in the percentage of internationally co-authored articles, and none shows a significant positive trend. A negative decline could be a sign of the consolidation and expansion phase in scientific development which is apparently dominant in middle income countries.

Asia Figure 6
Figure 6 - Trends in percentage of internationally co-authored articles in selected countries 2003-1011.
Source: UNESCO report “
Higher Education in Asia – Expanding up, Expanding Out”, p.84.


Trends in scientific collaborations

Figure 7 shows that there are tight co-authorship clusters within the region. Japan has a central role in the collaborative co-authorship scheme of the region. Japan’s research focus on Medicine, Biochemistry, Physics and Engineering enables it to become a central hub of collaborations, bringing together research from different areas in the region. In addition there is a formation of three clusters of research collaboration within the region. The first cluster includes China, Hong Kong (Special Administrative Region of China), Singapore and Macao (SAR of China), which constitute the East Asian region. As can be seen China also serves as a link between Hong Kong (SAR of China), Macao (SAR of China) and Singapore to other members of the region such as Japan, India and Thailand. The China / Hong Kong (SAR of China) / Singapore/ Macao (SAR of China) cluster focuses on the areas of Engineering, Physics and Astronomy as well as Computer Science for the most part. The second cluster, which includes India, Malaysia, Bangladesh, Pakistan and Afghanistan, constitutes the South Asian region and focuses on Medicine, Agriculture, Chemistry and Engineering. The third cluster, which includes Thailand as its center, closely connects Indonesia, Sri Lanka, Brunei, Nepal, Laos, Cambodia, Vietnam, Myanmar and Laos and together constitutes the South Asian region. This cluster focuses mostly on Agriculture, Medicine and Earth sciences. Finally the map shows that the Republic of Korea and China play an essential role in bridging between Democratic People's Republic of Korea and other countries.

Asia Figure 7
Figure 7 - Regional scientific collaborations.
Source: UNESCO report “
Higher Education in Asia – Expanding up, Expanding Out”, p. 88.


International scientific collaboration

Figure 8 shows the international scientific collaborations between Asian countries and the global community. There are four distinct “pockets” of international collaborations in the region. The United States, Canada, Germany, Spain and Italy form close collaborative relations with China, India, The Republic of Korea and Singapore. Secondly, the United Kingdom has a major role in connecting other European countries such as France, Belgium and Switzerland with SEA countries that display lower scientific output with the international community. The United Kingdom also serves as a bridge between Laos, Cambodia, Myanmar, Nepal, Bangladesh, Bhutan and others and the European scientific community. Australia forms a third circle of collaborations, bridging among Indonesia, the Philippines, Malaysia, Sri Lanka and Brunei. The map also shows that the Russian Federation is somewhat of an outlier forming single collaborations with the Republic of Korea, Japan, India, and Pakistan.

Asia Figure 8
Figure 8 - International Scientific collaborations between Asian countries and the global community.
Source: UNESCO report “
Higher Education in Asia – Expanding up, Expanding Out”, p. 89.



  1. Scientific output:
    the region has seen a significant increase in its scientific output from 1997 to 2012. There are, however, large differences between individual countries within the region. China has a leading role in scientific output and growth. However, attention should be given to countries such as Malaysia and Pakistan which have a compound annual growth rate above 15 percent in this time period.
  2. Regional and international collaborations:
    The most evident progress seen through the bibliometric analysis is both the increasing scientific collaborations between the countries of the region and a significant growth of international collaborations between the countries of the region and the international scientific community. The regional co-authorships networks show that smaller countries entering the scientific arena, such as Nepal, Bhutan and Sri Lanka, increasingly collaborate with larger countries in the region thus gaining expertise and increased output. These countries also used their collaborators as a bridge to the international scientific community. Larger countries such as China, Japan, Thailand and others, show increased international collaborative ties in the form of co-authorships and are functioning as hubs for smaller countries in their international scientific endeavors.


Disclaimer: This article is an extract of a study conducted for the latest UNESCO report “Higher Education in Asia – Expanding up, Expanding Out”. All figures are property of UNESCO Institute for Statistics (UIS), United Nations University-International Institute for Software Technology (UNU-IIST), Elsevier Inc. and UNESCO International Institute for Educational Planning (IIEP) (2014). Higher Education in Asia: Expanding Out, Expanding Up. ISBN 978-92-9189-147-4 licensed under CC-BY-SA 3.0 IGO. Montreal: UIS.


VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Reporting Back: The APAC research intelligence conference

Alexander van Servellen and Ikuko Oba report back from the first APAC research intelligence conference, which focused on the challenges institutions face with regard to managing research and the best practices employed to optimize research strategy and impact. 

Read more >

The first APAC Research Intelligence Conference was attended by 109 people from 70 institutions, coming from 8 different countries worldwide. The topic of discussion at this two day event hosted at The Nanyang Executive Center at NTU in Singapore was Research Excellence, the challenges which institutions face with regard to managing research and the best practices employed to optimize research strategy and impact. The idea to organize this event stemmed from a common interest in having a platform to facilitate open discussion on the topic by dedicated professionals, and that was certainly achieved as 9 speakers took the stage to share their insight and experiences. This article reviews selected parts of each speaker’s presentation.

Group photo of speakers

Group photo taken during the conference, with presenters mentioned in the article in bold.
Back row from the left – Hiroshi Fukunari, Marcel Vonder, Thomas Thayer, Anders Karlsson, William Gunn, Kevin Carlsten, Lim Kok Keng
Front row from the left - Byoung Yoon Kim, Hirofumi Seike, Douglas Robertson, Giles Carden, Michael Khor.


Day 1

Professor Bertil Andersson, President of Nanyang Technological University (NTU) in Singapore, presented ‘Nanyang Technological University, Singapore: A Drive in Excellence’.

Professor Andersson described Singapore as a country with a vibrant eco-system of world-class research producing institutions. He highlighted the important role of the Singapore government, as not only talking about developing the knowledge economy, but also walking the talk by providing funding, having a dedicated Research Innovation & Enterprise Council chaired by the Prime Minister, using 5-year planning cycles, and by having a tradition of philanthropic endowments and incentivizing private donations. NTU is one of the fast-rising universities in both the world ranking and research impact. Figure 1 shows NTU’s Field-Weighted Citation Impact surpassing that of Asia’s top institutes by 2012.

Professor Andersson attributed their success to being young and having been able to start from scratch rather than reorganizing an existing structure, receiving long-term generous finance, and being able to recruit senior and junior faculty from abroad by maintaining a strong international profile.


NTU AnderssonA university is about its people, its people and its people……its good people. I think personally the biggest secret to our success has been that we’ve been able to recruit top people from Europe, United States and Asia… of a very high caliber. And we also recruited many young people. The superstars of tomorrow have come to NTU in big numbers and we had funding for that”.

Professor Bertil Andersson, NTU




Rep Back RT 38  Fig 1
Figure 1 –The field-weighted citation impact of NTU and selected comparator institutions 2004-2012. Source: SciVal.


Professor Byoung Yoon Kim, Vice President of Research at the Advanced Institute of Science and Technology (KAIST) in South Korea presented ‘Strategic Role of KAIST in Advancing Korean Economic Development.

Professor Kim outlined the role KAIST has played in developing Korea’s economy in the last 40 years and spoke about the role they hope to play in the next 40 years. KAIST was established in 1971 with a mission to produce professionals to transform Korea into an industrialized nation. As an initiative for change and development, it was not only a new university, but was also under a different ministry, and therefore did not share budget with the other universities. KAIST recruited the best professors worldwide and successfully contributed to Korea’s economic growth by fostering talents who established companies now known worldwide, which generate the majority of Korean’s income.

Looking forward, Professor Kim spoke about the Startup-KAIST movement, which aims to establish a model that the country should follow by spreading a culture of entrepreneurship, to develop an eco-system to help establish and globalizecompany activities. Professor Kim echoed Professor Andersson in attributing the success of KAIST in part to having started as an independent university rather than changing an existing system and culture. He said if the same money went to another existing University, it would not have produced the same results. KAIST represented a departure from the old system.


yoonkim_hires1948X1461KAIST has to also find out what it should be doing for the next 40 years in order to be different and justify its existence.  We should not compare our university with SNU… it has a different mission. Although my president (laughs) and most government officers are very interested in university rankings, I try not to talk about it, because it is important in a sense, but it should not be the goal…”

Professor Byoung Yoon Kim, KAIST




Dr. Anders Karlsson, Vice President Global Academic Relations APAC, Elsevier, presented ‘The Global Trends on Internationalization and Assessing Impact Beyond Research.

Dr. Karlsson posed a number of questions; the most central being ‘is collaborative work better?’  He showed the positive correlation between the international collaboration share of a country and their Field-Weighted Citation Impact, found in the report prepared by Elsevier for the Department of Business Innovation and Skills (BIS) in the UK (1), and was quick to point out that correlation does not explain causality. From the same study, he presented data thatshowsthe UK’s international collaborative papers were cited 60% more often than papers collaborated on only within UK. That data was positioned as strong evidence demonstrating the leverage the UK gets from collaborating internationally, in terms of the positive effect on overall scholarly influence.

Dr. Karlsson investigated whether international articles are judged better in peer review. He used evidence provided from a study (2) which looked at papers submitted in Italy for peer review, and found that papers with more authors were judged higher in excellence.


Anders KarlssonIf you collaborate more, your citation impact increases, basically you have a broader base, and you reach out more broadly

International collaboration should be high on the strategic agenda of countries which want to increase their citation impact.

Dr. Anders Karlsson, Elsevier





Dr. Giles Carden, Director, Strategic Planning and Analytics, Warwick University presented ‘Research Planning: Embedding analytics in a new research performance challenge process at the University of Warwick’.

Giles CardenDr. Carden introduced Warwick University’s approach to using analytics for managing research performance, and explainedtheir imperative strategic rationales, achievements, and future direction. The context for developing analytics was to supportWarwick’s goal of becoming an undisputed world leader in research and scholarship, plus the fact that the UK’s national research exercise in part based their assessment on these types of analytics. Distribution of UK’s 1.6 billion pounds in block grants coming from the government is informed by the assessment outcome of the Research Excellence Framework (REF). Thus, Warwick developed an analytics and planning process in tandem and embedded the analytics into the process to be successful in this very competitive environment.

Dr. Carden shared analytics showing Warwick’s collaboration with the USA (see Figure 2), stating this was important to boost citation impact. He also revealed that Warwick’s Research Assessment and Planning group reviews the performance of each individual academic in a substantial post, and showed an author profile in SciVal (see Figure 2). Communication was the key to the project’s success.It was not about being out to get people, but to identify patterns that can help researchers turn their performance around. As a result, Warwick University improved academic staff accountability, grew in research income and research students, published more in high impact journals, and increased citations – along with a cultural shift within the University. In closing, Dr. Carden discussed the future of analytics and big data as likely involving predictive analytics, and also highlighted the limitations of analytics.

Rep Back RT 38  Fig 2


Figure 2 The collaboration map shows the institutions which Warwick University has collaborated with represented by a bubble which shows the number of co-authored publications (2011-2013). Source: SciVal.


Rep Back RT 38  Fig 3


Figure 3 Author profile showing the publications, citations, citations per paper and h-index of a specific author. Source: SciVal.


Dr. Douglas Robertson, Director of Research Services Division, The Australian National University (ANU), presented ‘The Changing World of Research Support and the Challenges of Impact from Basic Science: Some Reflections’.

Dr. Robertson has been active in research administration since 1983, and reflected on the changing nature in university research support and on some concerns. Research administration has become much more complex, and he questioned whether the quality of research is any better as a consequence. He encouraged contemplation about whether the development and current practice of research administration is really to the benefit of science and society.

Douglas Robertson“Life was very simple in 1983. When you were sent a research award, it ran to one side of A4 that said ‘we’d like to give you some money, will you please write back and say whether you’d like it. And if you could tell us what you did in three years’ time, we’d be very grateful.’ Now research contracts in the UK can run to 90 or 100 pages, of very closely typed script…there has been quite a lot of change… “

Dr. Douglas Robertson, ANU



He asked whether we are spending too much money on administrating research and not enough money on actually doing it. He also stated that several Nobel Prize winners have questioned whether they would have been funded under the current systems. Today, researchers have to report more often, get more permission, andjustify more why their research is worth investing in, while the focus is now more on the societal impact than the impact on research and other researchers.

Dr. Robertson also questioned whether the race to publish is a good thing, citing a number of studies which report observed lack of reproducibility, including one in the pharmaceutical industry where it was revealed that in only ~20–25% of the projects, were the relevant published data completely in line with our in-house findings (3).

I find it challenging to figure out how we create an effective research environment rather than one that is easy to measure. I am of the opinion that if you are using public money, and produce work that cannot be reproduced, it is not a good outcome, the aim is that you publish so that others can build on your publication, that you patent so that others can build on your invention, and if your publication does not achieve that, we have concerns. Particularly in the life sciences, the pressure on scientists is phenomenal…”

Dr. Douglas Robertson, ANU

Finally, Dr. Robertson raised the importance of curiosity driven research and concerns about the increased shift in focus to applied science. Scientists are increasingly required to indicate what their research will be used for rather than being left to freely explore the unknown. He underlined the importance of basic research, stating that applied research is only possible when you have a solid foundation of basic research.


Day 2

Dr. Hirofumi Seike, University Research Administrator, Management Associate Professor, Tohoku University presented ‘University Internationalization and its Impact’.

Dr. Seike raised internationalization as a challenge, and why? Tohoku commits to providing students the best quality international perspective possible, and the university believes international experience will ensure a high quality education and research,  as well as expanding their human networks.Many global issues can only be solved through international collaboration, but Japan encounters a problem of students not wanting to study abroad. In this sense, he feels Japan is falling behind.

In terms of research, he feels that Japan has stagnated, while other Asian countries are increasing their presence. The government shares a strong sense of urgency which leads them to initiate multiple globalization projects and set targets such as to include 10 universities in the top 100 in world rankings.

Dr. Seike introduced one of the government initiatives, WPI,which aims to establish world-class research institutes. Tohoku University was chosen as one of them. WPI empowered the awardees to have their own governance,which allowed competitive recruitment to assemble world-class innovative scientists that can lead from basic research to industry application.


Tohoku Seike"WPI established a special zone within the existing university framework.  It’s a new approach… not just the expansion of the existing system. It should be the showcase of the best research… the best of the best.”

Dr. Hirofumi Seike, Tohoku University





Professor Paul K.H. Tam, Pro-Vice-Chancellor and Vice-President (Research), University of Hong Kong (HKU) presented ‘Research Excellence and Internationalization at the University of Hong Kong – Striking the Right Mix of Metrics and Faculty Expertise’.

Professor Tam described the University of Hong Kong (HKU) as an institute of great heritage but with many structural issues that needed to be resolved – and shared the ways they overcame these challenges. It was a transformation from a predominately teaching university to a comprehensive research university.

A major motivation for institutions of Higher Education is competition, and the introduction of other universities in Hong Kong ‘awoke the giant from a deep sleep’. While the transformation was also self-motivated, there were important external factors which came from the government; the establishment of The Research Grants Council followed by the introduction of the Research Assessment Exercise.The previous funding system allocated 75% of the money into a recurrent grant that supported continuity and sustainability. Distribution was based on student places (75%) and only 25% was related to research. The government changed the system to drive major change, and allocation is now judged using performance indicators.

What does it mean to be a ‘world-class’ university? HKU agreed upon having a tradition of research excellence with internationally competitive staff and more importantly, a strong culture that will attract students globally as the choice of institution for those who want a career in research. HKU has been very successful despite there being 8 institutions. They are responsible for over half of large program grants, and have the top position in every assessment indicator, be it grant amount or research output.

Talking about university transformation, Professor Tam spoke about the guiding principles of providing an enabling environment for researchers and respecting academic freedom by keeping a bottom up approach which is top facilitated.

HKU_TamWhat I consider the greatest asset of the university is human resources, the talents. It is the role of the university leaders to provide an enabling environment for the researchers – this is my guiding principle. The other principle I have is that we have enjoyed the principle of academic freedom and we respect that and continue to cherish it. To respect that means the approach is bottom up. There can be a lot of debate between top down and bottom up approaches. We have kept a bottom up approach but introduced a top facilitated bottom up approach.”

Professor Paul Tam, HKU


Dr. John Green, Life Fellow, Queens’ College, University of Cambridge, presented ‘Evidence Based decision making in academic research’.

First Dr. Green created context by talking about increasing interdisciplinarity and internationalization in science, and the increasing demand for evidence to evaluate outcomes and justify funding expenditure. He spoke about how Imperial College evaluates interdisciplinary institutes that work cross-departmentally every 3 or 4 years, and on what basis they close institutes down.

Dr. Green touched on the potential of getting lost in the avalanche of data available today and the importance of getting meaningful information from the data. It is important to understand where the strength of an institution lies, where to focus its strategy, who to collaborate with. He explained the importance of due diligence about specific partnerships, the need to find ways to connect researchers and facilitate the mobility that will create the collaboration. He stated firmly that these things do not happen bottom up, that there is a need to facilitate them based on evidence to inform the facilitation. At Imperial, he created and used a systemthat presents the research performance dashboards at the departmental level.


John GreenThe world has changed now, and if only some of the systems which are available to you now were available to me then, I would not have re-invented the wheel… Pure has now come into the market, which does exactly what we were trying to do, but it does it better. It is a system which sits on top of your internal IT systems and harvests information from it and provides you with dashboards, and that is exactly the concept that I have been talking about.” (Figure.4)

Dr. John Green, Life Fellow, Queens’ College, University of Cambridge



Rep Back Fig 4

Figure 4 Example of a dashboard in Pure which shows research outputs, journals and activities for a university

Having spoken about the metrics, he pointed out the need to standardize the definitions and methodology involved in generating metrics, because everyone tends to do it differently which means that the results cannot accurately be compared. How can we compare the number of researchersif each university defined researcher counts differently? The Snowball project, a non-commercial initiative in which Dr. Green and Elsevier are involved, resulted in agreed upon methodology for these metrics that are endorsed by a group of distinguished UK universities.

“I don’t want to give the impression that metrics are everything. Metrics are one of a number of ways to come to a judgment… and help you come to a view of something. In no sense are they a way to navigate your car. A Satellite Navigation system is something that tells you what the best route is and how you might change the route if there are traffic jams… But you have to decide which is the best route for you, based on that information and other information too (for example where you want to go for lunch, do you want the prettiest route or the autoroute). That is why we need other measures such as peer review to complement metrics.

Dr. John Green, Life Fellow, Queens’ College, University of Cambridge


Dr. William Gunn, Head of Academic Outreach, Mendeley, presented ‘Innovation – Scientific and technical foundation development for altmetrics in the US’.

Dr. Gunn spoke about totally new metrics, which may compliment, or arguably even be an alternative to traditional metrics, hence the term ‘altmetrics’. He suggests that new forms of scholarship need new metrics. Altmetrics are faster to accrue compared to citation data, and they use research and social media data that is totally outside of the traditional research metrics. Altmetrics for impact include usage of articles, peer-review such as via post publication commentary services,and social media activities such as discussions on blog posts to measure attention and impact work had given to others. He drives home the point that there are many ways to look at the overall influence of a paper or group of papers, and that citations are just a tiny fraction of that.

Gunn_William_Mendeley2There are 125 times more downloads of papers (than citations) and a universe of social activities, that are being aggregated…, so there is a lot more data out there that we can gather, work with and use to understand the impact our work is having”.

Dr. William Gunn, Head of Academic Outreach, Mendeley





Nonetheless, altmetrics are not without challenges with regard to transparency and consistency. There are different services that provide altmetrics such as Plum Analytics, Impact Story, and Altmetric, and if you query them all on one specific DOI, there are differences in the metric values reported back, which leads us to question which value is correct. There are also problems with identity attribution if researchers use a fake identity, and finally altmetrics can also be gamed, although it is difficult if people make use of many sources and many different metrics.

Looking back, the conference was fascinating in that the speakers and participants alike were passionate and often candid in sharing their views and experiences, resulting in lively discussions, which we all could learn from.


(1) ELSEVIER (2013) “International Comparative Performance of the UK Research Base – 2013”. Available at:
 (2) Franceschet, M., Costantini, A. (2010) “The effect of scholar collaboration on impact and quality of academic papers”, Journal of Informetrics, Vol. 4, No. 4,  pp. 540-553.
 (3) Prinz, F., Schlange, T., Asadullah, K. (2011) “Believe it or not: How much can we rely on published data on potential drug targets?”, Nature Reviews Drug Discovery, Vol. 10, No. 9, pp. 712-713.



VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Did you know

…. these 10 things about Scopus?

Dr. Daphne van Weijen

As it’s Scopus’ 10 year anniversary, we thought we’d collate some interesting facts you might not know about Scopus…

1. Scopus is named after a bird, the Phylloscopus collybita, more commonly known as the Common Chiff Chaff.
2. Scopus won the International Information Industry Award for Best Science, Technology and Medicine (STM) Information Product in 2005.
3. The Scopus Journal Analyzer was launched in 2008.
4. Scopus was added to Research4Life in 2009.
5. Scopus added two journal performance metrics, Source Normalized Impact per Paper (SNIP) and the SCImago Journal Rank (SJR) for all titles in the database in 2010 and in 2014 a third metric was introduced, the Impact Per Publication (IPP). All journal performance metrics are freely available online.
6. Scopus coverage has grown from 13,000 titles and 27 million items in 2004 to 21,000 titles, and 55 million records, from more than 5,000 publishers today.
7. Scopus has a number of celebrities with author profiles, including Colin Firth, Natalie Portman and Danica McKellar; learn more about Author Profiles.
8. Scopus has APIs that allow you to use selected data – such as cited by counts – on your website, institutional repository or research information system.
9. Scopus recently added local language interface capabilities for Chinese and Japanese speaking users.
10. Scopus’ Arts & Humanities coverage has grown from 2,000 titles in 2008 to more than 4,200 titles – and includes more than 1.3 million articles. By 2015, the Scopus Books Expansion Project will add 75,000 books including a large proportion in the Arts & Humanities.

  • Elsevier has recently launched the International Center for the Study of Research - ICSR - to help create a more transparent approach to research assessment. Its mission is to encourage the examination of research using an array of metrics and a variety of qualitative and quantitive methods.