Carl Bergstrom has been researching journal economics for over a decade. One fruit of those efforts, the Eigenfactor project, is drawing interest from editors, authors, researchers, policy-makers and evaluators seeking new measures of journal influence.

Jevin West, graduate student at the University of Washington in Bergstrom’s research group, recalls: “It all started with Ted Bergstrom, Carl Bergstrom and I chatting about evaluation tools over a beer in December 2005. Carl was getting a lot of flack for using Impact Factors (IFs) in his work on journal economics, so we decided to come up with another way of evaluating the scholarly literature.

“I come from the theoretical side of biology and I’m interested in applying tools and concepts from network science and information theory to various problems, and that extends beyond biology to other fields, including bibliometrics. Fortunately, the nature of citation networks means that many of the models we use in biology are easily transferable.”

How the tools work

The Eigenfactor works by taking a random journal and following a random citation in that journal to another journal, then selecting another random citation from the second journal and following that to the next journal and so on. The Eigenfactor calculates the percentage of time you would spend at each journal. For instance, a search of all journals in 2006 gives Nature the highest Eigenfactor score. If you followed random citations infinitely, you would spend 1.9917% of your time at Nature.

The Article Influence score is calculated by dividing the Eigenfactor score for a particular journal by the number of articles published by that journal. All journals are normalized to 1. The Annual Review of Immunology comes out top, at 27.454 times normal.

Phenomenal interest

The Eigenfactor works like Google’s PageRank, both of which are based on social network theory; where Google follows page links, Eigenfactor uses citations. They evaluate the importance of each journal (or Web page) based on the structure of the entire network.

The IF, in comparison, only looks one citation away and it ignores where they come from. “We take into account where the citation came from just as Google takes into account where a hyperlink comes from,” says West.

When the Eigenfactor Web site was launched in January 2007, it attracted comment in numerous blogs, which raised its profile. “It has been far beyond anything we could have imagined. The interest has been phenomenal,” says West.

“We also have our critics, and this is healthy,” West adds. “I think all metrics should be criticized. Nothing beats reading an individual article in a journal to assess its value, and nothing ever will. But with time and budget constraints being what they are, there is a legitimate need for tools like this.”

More and better tools

Journal ranking is just one of many stories the Eigenfactor team are pulling out of the data. They have created a cost-effectiveness score to help librarians manage their budgets efficiently, science maps and motion charts showing trends over time, which are particularly popular with editors and authors, and researchers interested in the history of science.

The team are planning to improve the tools they have and develop new ones, and they hope to bring in richer data. Also, over the longer-term, they want to apply these tools to other areas. “We’re curious about how science has changed over time and we’re interested in applying these tools to non-bibliometric areas as well,” says West.

Even though this is just a side project, the team are enjoying themselves. “This has all come together at the right time. The data is available and some very sophisticated tools have been developed over recent years. We can now analyze data in some very exciting ways. We’re having a blast!”

Useful links:

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)