Prestige measured by quantity of citations is one thing, but when it is based on the quality of those citations, you get a better sense of the real value of research to a community. Research Trends talks to Prof. Félix de Moya about SCImago Journal Rank (SJR), which ranks journals based on where their citations originate.

Felix de Moya

Felix de Moya

Research Trends (RT): SCImago Journal Rank (SJR) has been described as a prestige metric. Can you explain what this means and what its advantages are?
Félix de Moya (FdM): In a social context, prestige can be understood as an author’s ability or power to influence the remaining actors, which, within the research evaluation domain, can be translated as a journal’s ability to place itself in the center of scholarly discussion; that is, to achieve a commanding position in researchers’ minds.

Prestige metrics aim at highlighting journals that do not depend exclusively on the number of endorsements, as citations, they receive from other journals, but rather on a combination of the number of endorsements and the importance of each one of these endorsements. Considered in this way, the prestige of a journal is distributed among the ones it is related to through the citations.

SCImago

The SCImago Journal & Country Rank is a portal that includes the journals and country scientific indicators developed from information in the Scopus database. These indicators can be used to assess and analyze scientific domains.

This platform takes its name from the SCImago Journal Rank (SJR) indicator, developed by SCImago from the widely known algorithm Google PageRank. This indicator shows the visibility of the journals contained in the Scopus database from 1996.

SCImago is a research group from the Consejo Superior de Investigaciones Científicas (CSIC), University of Granada, Extremadura, Carlos III (Madrid) and Alcalá de Henares. The group conducts research into information analysis, representation and retrieval using visualization techniques.

RT: I understand that SJR is based on the premise that not all citations are equal (much like Google’s PageRank algorithm treats links as more or less valuable). Can you explain why it is so important to consider the value of each citation and what benefits this brings to your final ranking?
FdM: When assigning a value to a journal, the source of its citations are not the only important consideration. It is also essential to control for the effects of self-citation or other practices that have nothing to do with scientific impact. This method achieves that because citation “quality” cannot be controlled unless one has control over the whole citation network, which is, of course, impossible.

RT: Why did you decide to develop a new evaluation metric? Were you meeting a perceived market gap or seeking to improve on current methods?
FdM: As researchers working in bibliometric and scientometric fields, we have studied research-evaluation metrics for many years and we are aware of their weaknesses and limitations. The success of the PageRank algorithm and other Eigenvector-based methods to assign importance ranges to linked information resources inspired us to develop a methodology that could be applied to journal citation networks. It is not only a matter of translating previous prestige metrics to citation networks; deep knowledge of citation dynamics is needed in order to find centrality values that characterize the influence or importance that a journal may have for the scientific community.

RT: Since you launched your portal in November 2007, what level of interest have you seen from users?
FdM: The SJR indicator is provided at the SCImago Journal & Country Rank website, which has 50,000 unique visits per month. We attribute this traffic in a large part to the open availability of the metrics. And, what it is more important to us in terms of value is the increasing number of research papers that use SJR to analyze journal influence.

RT: I understand that SJR is calculated for many journals that currently have no rank under other metrics, such as those published in languages other than English, from developing countries, or those representing small communities or niche topics of research. The advantages to these journals – and to the researchers that are published in them – is obvious, but what about the advantages to science in general?
FdM: In our opinion, SJR is encouraging scientific discussion about how citation analysis methods can be used in journal evaluation. And this is really happening: in fact, all the new methodological developments in Eigenvector-based performance indicators are encouraging this healthy debate.

However, unlike many other Eigenvector-based performance indicators, SJR is built on the entire Scopus database rather than across a sample set of journals. This has important methodological implications. In addition, SJR’s results are openly available through the SCImago Journal & Country Rank evaluation platform, which makes SJR a global framework to analyze and compare the findings, and allows researchers and users worldwide to reach their own conclusions.

FdM: When assigning a value to a journal, the source of its citations are not the only important consideration. It is also essential to control for the effects of self-citation or other practices that have nothing to do with scientific impact. This method achieves that because citation “quality” cannot be controlled unless one has control over the whole citation network, which is, of course, impossible.

RT: Why did you decide to develop a new evaluation metric? Were you meeting a perceived market gap or seeking to improve on current methods?
FdM: As researchers working in bibliometric and scientometric fields, we have studied research-evaluation metrics for many years and we are aware of their weaknesses and limitations. The success of the PageRank algorithm and other Eigenvector-based methods to assign importance ranges to linked information resources inspired us to develop a methodology that could be applied to journal citation networks. It is not only a matter of translating previous prestige metrics to citation networks; deep knowledge of citation dynamics is needed in order to find centrality values that characterize the influence or importance that a journal may have for the scientific community.

RT: Since you launched your portal in November 2007, what level of interest have you seen from users?
FdM: The SJR indicator is provided at the SCImago Journal & Country Rank website, which has 50,000 unique visits per month. We attribute this traffic in a large part to the open availability of the metrics. And, what it is more important to us in terms of value is the increasing number of research papers that use SJR to analyze journal influence.

RT: I understand that SJR is calculated for many journals that currently have no rank under other metrics, such as those published in languages other than English, from developing countries, or those representing small communities or niche topics of research. The advantages to these journals – and to the researchers that are published in them – is obvious, but what about the advantages to science in general?
FdM: In our opinion, SJR is encouraging scientific discussion about how citation analysis methods can be used in journal evaluation. And this is really happening: in fact, all the new methodological developments in Eigenvector-based performance indicators are encouraging this healthy debate.

However, unlike many other Eigenvector-based performance indicators, SJR is built on the entire Scopus database rather than across a sample set of journals. This has important methodological implications. In addition, SJR’s results are openly available through the SCImago Journal & Country Rank evaluation platform, which makes SJR a global framework to analyze and compare the findings, and allows researchers and users worldwide to reach their own conclusions.

Félix de Moya

Félix de Moya has been professor of the Library and Information Science Department at the University of Granada since 2000. He obtained a Ph.D. degree in data structures and library management at the University of Granada in 1992. He has been active in numerous research areas, including information analysis, representation and retrieval by means of visualization techniques. He has been involved in innovative teaching projects, such as “Developing information systems for practice and experimentation in a controlled environment” in 2004 and 2006, and “Self-teaching modules for virtual teaching applications” in 2003.

Full resumé

RT: What particular benefits does SJR bring to the academic community? How can researchers use SJR to support their publishing career?
FdM: Following the reasoning above, SJR is already being used for real research evaluations. Researchers and research groups are using SJR to measure research achievements for tenure and career advancement, and research managers are paying increasing attention to it because it offers a comprehensive and widely available resource that helps them design methods for evaluating research. Universities worldwide are, for example, using SJR as a criterion for journal assessment in the evaluation processes.

RT: One of the main criticisms leveled at ranking metrics is that their simplicity and supposed objectivity is so seductive that more traditional methods of ranking, such as speaking to researchers and reading their papers, are in danger of being completely superceded by ranking metrics. What is your position?
FdM: Ideally, whenever a quantitative measure is involved in research-performance assessment it should be always supported by expert opinion. Unfortunately, this is not always possible due to the nature of some specific evaluation processes and the resources allocated to them. In cases where the application of quantitative metrics is the only way, efforts should be made to design the assessment criteria and reach consensus among all stakeholders to select a combination of indicators and sources that comprise fair assessment parameters.

RT: Finally, why do you think the world needs another ranking metric?
FdM: The scientific community is becoming accustomed to the availability of several journal indices and rankings, and to the idea that no single indicator can be used in every situation. Many new metrics have been released in recent years, and it is necessary to analyze the strengths and weaknesses of each these. When a new methodology solves some of the well-known problems of prior metrics, it is certainly needed.

In addition, the research community is moving forward from traditional binary assessment methods for journals. My research group believes that new metrics should be oriented toward identifying levels or grades of journal importance, especially considering the rapid increase in scientific sources, which means metrics are frequently calculated on universals rather than samples. In this scenario, we need a measure that can discern journal “quality” in a source where a huge number of publications coexist.

Useful links

SJR
SCImago
Free journal-ranking tool enters citation market: Nature news and comment
Wikipedia – SJR
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)