Which scientific stories are most shared on social media networks such as Twitter and Facebook?

Research on human health and social issues are often perceived as being the most shared scientific stories on social media networks such as Twitter and Facebook and – given their mainstream appeal – are often suggested to dominate the popular discussion around scholarly research online, but skeptics, such as David Calquhoun argue for their irrelevance: “Scientific works get tweeted about mostly because they have titles that contain buzzwords, not because they represent great science” (1).

So which is it to be? And do articles attracting social media attention also get the attention of scholars and the mass media? In this article, we seek to provide an approach to answering these questions.

With the rise of online scholarly publishing and the concomitant rise in the desire to create indicators of online attention to research articles and related outputs have come a number of providers of article-level data. A leading commercial provider of such data - collectively known as ‘altmetrics’ - is Altmetric.com, which tracks a variety of different indicators in four broad groups: Social Activity (e.g. Tweets and Facebook mentions), Mass Media (e.g. mentions on news sites such as BBC and CNN), Scholarly Commentary (e.g. mentions in scientific blogs), and Scholarly Activity (e.g. articles in reader libraries such as Mendeley). The overall collection and analysis of these references are brought together under the label “altmetrics”.

In terms of the volume of online mentions of scholarly articles, Twitter and other social networks provide by far the largest number of data points. However, given Twitter’s broad user base (the majority being non-academics) and limited information content (being restricted to 140 characters per tweet), other indicators may be more significant in terms of understanding scholarly usage (2). For example, Mendeley and CiteULike are examples of sharing and collaboration platforms used predominantly by researchers, while the mass media and scientific blogs tracked by Altmetric.com are written by professional science journalists or researchers themselves.

 

Methodology

Data were collected from the Altmetric.com API over four months ending January 17th, 2014. On this date, the latest altmetric indicator data for all papers published in a selection of journals in 2013 with any online mentions captured by Altmetric.com were downloaded for analysis; in total, 13,793 articles with at least one altmetric indicator datapoint were included in this study. Please note, the actual Journals monitored are detailed in the raw dataset, which is published on Figshare.

The Altmetric.com data includes counts of online attention at article level from across a variety of different data sources. In order to simplify data analysis, we aggregated data counts into the four classes as defined above: Social Activity, Mass Media, Scholarly Commentary, and Scholarly Activity. For each class, articles were assigned to predefined percentile ranges (cohorts) based on the frequency of online mentions (see Table 1).

Cohorts Number of articles included

0.5%

69

1%

138

2.5%

347

5%

691

7.5%

1,045

10%

1,384

15%

2,095

20%

2,775

25%

3,539

30%

4,332

100%

13,793

Table 1 - Cohorts of articles based on the frequency of online attention within each class.

For example, the 69 papers comprising the top 0.5% of social activity comprise 91,470 social actions, 445 mass media mentions, 540 scholarly comments and 1,571 scholarly actions, whereas the top 69 papers comprising the top 0.5% of mass media activity comprise 2,638 mass media mentions, 16,221 social actions, 779 scholarly comments and 4,856 scholarly actions.

 

 Analysis

 Headline-grabbers: Which articles got most social media attention in 2013?

Of the 69 articles belonging to the 0.5% cohort in the Social Activity class (i.e. those articles most frequently mentioned in social media such as Twitter and Facebook, for example), just 8 of them are full-length articles reporting the results of original research. The remainder are typically editorial features or news items from leading weekly journals such as The Lancet, BMJ and Nature; see table 2 for the complete list. The original research articles cover topics in the popular consciousness including climate change, human health and diet, and online information and privacy: intuitively, the sort of articles one might expect to see attracting broad popular attention online. However, one article appears to have a less obvious popular slant (the Nature letter “Attractive photons in a quantum nonlinear medium”) but closer examination shows that it describes a novel technique for forcing photons to interact in a quantum nonlinear medium which may have applications in quantum processing, where the ability to have photons ‘see’ each other could overcome present technological limitations.

The remaining 61 articles (almost exclusively news and editorial features about original research reported elsewhere) cover a variety of topics including several on topics close to the heart of the academy: research careers, science funding, the future of higher education and scholarly publishing. The preponderance of items in this group from Nature (primarily the Nature News and Nature News Feature sections of the publication) suggest that Social activity may be more likely to reflect attention to short journalistic versions of current research results rather than the original research articles themselves; a worthy follow-up to this study would be to track the variation in performance across altmetric indicator classes of an original research article and the current awareness ‘news-worthy’ version of the same research.

 

Journal Article title DOI
Nature Cerebral organoids model human brain development and microcephaly 10.1038/nature12517
Nature Comment Climate science: Vast costs of Arctic change 10.1038/499401a
Nature Comment Neuroscience: My life with Parkinson's 10.1038/503029a
Nature Editorial Nuclear error 10.1038/501005b
Nature Editorial Science for all 10.1038/495005a
Nature Letter No increase in global temperature variability despite changing regional patterns 10.1038/nature12310
Nature Letter Attractive photons in a quantum nonlinear medium 10.1038/nature12512
Nature News Brazilian citation scheme outed 10.1038/500510a
Nature News Half of 2011 papers now free to read 10.1038/500386a
Nature News World's slowest-moving drop caught on camera at last 10.1038/nature.2013.13418
Nature News Genetically modified crops pass benefits to weeds 10.1038/nature.2013.13517
Nature News NSF cancels political-science grant cycle 10.1038/nature.2013.13501
Nature News Deal done over HeLa cell line 10.1038/500132a
Nature News Antibiotic resistance: The last resort 10.1038/499394a
Nature News Cosmologist claims Universe may not be expanding 10.1038/nature.2013.13379
Nature News Zapped malaria parasite raises vaccine hopes 10.1038/nature.2013.13536
Nature News See-through brains clarify connections 10.1038/496151a
Nature News Dolphins remember each other for decades 10.1038/nature.2013.13519
Nature News Researchers turn off Down’s syndrome genes 10.1038/nature.2013.13406
Nature News Astrophysics: Fire in the hole! 10.1038/496020a
Nature News Giant viruses open Pandora's box 10.1038/nature.2013.13410
Nature News Quantum gas goes below absolute zero 10.1038/nature.2013.12146
Nature News Stem cells reprogrammed using chemicals alone 10.1038/nature.2013.13416
Nature News Whole human brain mapped in 3D 10.1038/nature.2013.13245
Nature News Father’s genetic quest pays off 10.1038/498418a
Nature News Tracking whole colonies shows ants make career moves 10.1038/nature.2013.12833
Nature News Pesticides spark broad biodiversity loss 10.1038/nature.2013.13214
Nature News Animal-rights activists wreak havoc in Milan laboratory 10.1038/nature.2013.12847
Nature News Silver makes antibiotics thousands of times more effective 10.1038/nature.2013.13232
Nature News Methane leaks erode green credentials of natural gas 10.1038/493012a
Nature News When Google got flu wrong 10.1038/494155a
Nature News First proof that prime numbers pair up into infinity 10.1038/nature.2013.12989
Nature News Global carbon dioxide levels near worrisome milestone 10.1038/497013a
Nature News Underwater volcano is Earth's biggest 10.1038/nature.2013.13680
Nature News Did a hyper-black hole spawn the Universe? 10.1038/nature.2013.13743
PNAS Private traits and attributes are predictable from digital records of human behavior 10.1073/pnas.1218772110
Nature News How to turn living cells into computers 10.1038/nature.2013.12406
Nature News Small-molecule drug drives cancer cells to suicide 10.1038/nature.2013.12385
Nature News Brain-simulation and graphene projects win billion-euro competition 10.1038/nature.2013.12291
Nature News Rewired nerves control robotic leg 10.1038/nature.2013.13818
Nature News US government shuts down 10.1038/502013a
Lancet Letter Open letter: let us treat patients in Syria 10.1016/s0140-6736(13)61938-8
Nature News Blood engorged mosquito is a fossil first 10.1038/nature.2013.13946
BMJ Cancer risk in 680 000 people exposed to computed tomography scans in childhood or adolescence: data linkage study of 11 million Australians 10.1136/bmj.f2360
Nature News NIH mulls rules for validating key results 10.1038/500014a
PNAS Impact of insufficient sleep on total daily energy expenditure, food intake, and weight gain 10.1073/pnas.1216951110
Nature News Red meat + wrong bacteria = bad news for hearts 10.1038/nature.2013.12746
Nature News Who is the best scientist of them all? 10.1038/nature.2013.14108
Nature News Four-strand DNA structure found in cells 10.1038/nature.2013.12253
Nature News Weak statistical standards implicated in scientific irreproducibility 10.1038/nature.2013.14131
Nature News Mathematicians aim to take publishers out of publishing 10.1038/nature.2013.12243
BMJ Bicycle helmets and the law 10.1136/bmj.f3817
Nature News Barbaric Ostrich: 27th June 2013 10.1038/nature.2013.12487
American J of M The Autopsy of Chicken Nuggets Reads “Chicken Little” 10.1016/j.amjmed.2013.05.005
Nature News Stem cells mimic human brain 10.1038/nature.2013.13617
Nature News Mystery humans spiced up ancients’ sex lives 10.1038/nature.2013.14196
BMJ The future of the NHS--irreversible privatisation? 10.1136/bmj.f1848
Nature News Feature Archaeology: The milk revolution 10.1038/500020a
Nature News Feature Neuroscience: Solving the brain 10.1038/499272a
Nature News Feature Tissue engineering: How to build a heart 10.1038/499020a
Nature News Feature Theoretical physics: The origins of space and time 10.1038/500516a
Nature News Feature Online learning: Campus 2.0 10.1038/495160a
Nature News Feature Open access: The true cost of science publishing 10.1038/495426a
Nature News Feature Inequality quantified: Mind the gender gap 10.1038/495022a
Nature News Feature Voyager: Outward bound 10.1038/497424a
Nature News Feature Mental health: On the spectrum 10.1038/496416a
Nature News Feature Brain decoding: Reading minds 10.1038/502428a
Nature News Feature Fukushima: Fallout of fear 10.1038/493290a
Nature News Feautre The big fat truth 10.1038/497428a

Table 2 -  Full list of the 69 articles belonging to the 0.5% cohort in the Social Activity class including journal, article title, and DOI. Articles highlighted in orange are those representing full-length articles reporting the results of original research.

 

Social media attention: An indicator of scholarly impact or simply newsworthiness?

The articles which appear in the top 0.5% cohort in each of the four classes defined in this study are typically not the same ones: just 2 articles appear in all 4 lists. This suggests that the correlation between these 4 classes of altmetric indicators may not be very high. These two articles are both original research articles, one reporting the development of a method for creating human brain-like structures (called “cerebral organoids”) in cell culture and using these to study the basis of brain development and disease (Nature article “Cerebral organoids model human brain development and microcephaly”); the other correlating online behaviour (in this case, Facebook ‘likes’) with personal information such as sexual orientation, ethnicity and political views, to create a model to predict such traits based solely on Facebook activity (PNAS article “Private traits and attributes are predictable from digital records of human behavior”).

Further analysis of the overlap between the top 0.5% cohorts in each altmetric class is shown in Table 3: by far the greatest overlaps occur between the Mass media and Scholarly commentary classes, the lowest between Social activity and Mass media or Scholarly activity, and a moderate degree of overlap for the remaining pairwise combinations. Taken together, this suggests that - at least amongst this handful of articles receiving the most online attention – articles attracting a high degree of Social activity attract relatively little attention from the Mass media or from Scholarly activity and only a moderate degree of scholarly commentary. Conversely, there is a very high co-occurrence of articles receiving Mass media attention and Scholarly commentary. Taken together, these observations suggest that Social activity in particular is an indicator of a very different kind of online attention than the other three classes.

  Mass media Scholarly activity Scholarly commentary Social activity
Mass media  

11

31

5

Scholarly activity  

14

2

Scholarly commentary  

15

Social activity        

Table 3 -  Co-occurrence counts of articles comprising the top 0.5% of articles in each class, where n varies between classes owing to tied rankings at the 0.5% cutoff between 69 and 76.

Figure 1 shows how this correlation varies across all percentile cohorts for articles with Social activity. Note that approximately 90% of social activity is constrained to 15% of articles, which is a significantly more skewed distribution than that of citations across articles within a journal (where some 90% of citations are to 50% of the articles; (3)).  This implies a scarce attention economy in the Social activity spectrum, with many articles competing for a rare resource (reader attention). The only altmetric class with a distribution of attention across articles similar to that of citations across articles is Scholarly activity (which correlates very poorly with Social activity), where approximately 90% of Scholarly activity is represented by some 30-40% of articles (data not shown). The convergence of the curves in Figure 1 around the 15% cohort implies that at this point attention in all 4 classes is equally scarce, while in the cohorts above this point the only class showing a considerable degree of co-occurrence with Social activity is Scholarly commentary (also borne out by the Table 3 for the 0.5% cohort).

AP-MT 1

Figure 1 - Proportion of total activity per article across predefined percentile ranges (cohorts) for social activity.

 

Conclusions

It is clear from this exploratory work that altmetrics hold great promise as a source of data, indicators and insights about online attention, usage and impact of published research outputs. What is currently less certain is the underlying nature of what is being measured by current indicators represented within the four broad classes analysed here, and what can (and cannot) be read into them for the purposes of assigning credit or assessing research impact at the level of individual researchers, journals, institutions or countries.

What is strikingly clear from the qualitative analysis of the top 0.5% of papers for Social Activity is the lack of mentions of titles that have particularly titillating or eye-catching keywords: although most of the links are to summaries of research, rather than primary research articles themselves, they all contains serious scientific material.

On the basis of this preliminary study, we urge caution in characterizing all altmetric indicators in a similar way, as it is likely that different indicators may measure different types of online attention from different types of readers. This finding is similar to that reported by Priem, Piwowar and Hemminger in 2012 (4). We also suggest that careful delineation of document types (as long used for citation-based indicators) must be applied to correctly evaluate (for example) the relative social activity attracted by a news or editorial item versus an original research article; these values are likely to be the inverse of their usual relationship in citation terms. In short, in the excitement and promise of this burgeoning new field of Informetrics, we must be sure to ask ourselves: what is it that we are measuring, and why?

 

Acknowledgements

This paper would not be possible without the kind support of Euan Adie at Altmetric.com in providing access to these data for research purposes.

 

References

 (1) http://www.dcscience.net/?p=6369
(2) http://www.slideshare.net/StefanieHaustein/haustein-ape2014-30482551
(3) Seglen, P.O. (1992) The skewness of science. Journal of the American Society for Information Science, 43 (9) pp. 628–638.
(4) Priem, J., Piwowar, H., & Hemminger, B. (2012) Altmetrics in the wild: Using social media to explore scholarly impact. Arxiv. http://arxiv.org/abs/1203.4745 

The data set this paper was  based on is available online:

Taylor, Michael (2014): Data set for " Party papers or policy discussions: an examination of highly shared papers using altmetric data". figshare. http://dx.doi.org/10.6084/m9.figshare.943471

 

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)