Over a hundred people gathered at the beautiful Grosvenor Auditorium at the National Geographic Society to participate in a day-long seminar on practical applications of research evaluation methodologies. The seminar included diverse perspectives on research evaluation and its implications on funding allocations in industry, government and academic settings. The day opened with a keynote speech by Debra Perez, Assistant Vice President for Research and Evaluation at the Robert Wood Johnson Foundation. As a representative of one of the largest corporate grant giving foundations in America, Dr. Perez focused on the foundation’s research and evaluation work concentrating on public health and health care services. Dr. Perez shared statistics and facts concerning issues of race and economic status relating to health and health care in the Americas, and spoke of the foundation’s funded programs in these areas working towards equality and quality of services for underprivileged and minorities.

A methodological approach to research evaluation was presented by Dr. Henk Moed of Elsevier, who discussed the multifaceted nature of research evaluation. In his presentation, Dr. Moed presented the Multi-Dimensional Research Assessment Matrix, whereby the motivation, purpose and methodologies used for evaluation are taken into consideration. The main premise of the matrix is that one has to consider the why, what, when and how, and choose the correct method for each scenario, before applying any methodology to evaluate the impact of research or a researcher.

An international perspective on the subject was presented by Dr. Marc Luwel from the Hercules Foundation in Belgium. In his talk, Dr. Luwel described the Flemish approach to performance-based (research) funding and its evolution over time. The main motivation for the development of evidence-based research funding was the need to promote excellence and be able to better manage universities working with diminishing resources. To address these challenges, Dr. Luwel described the Flemish development of multi-level indicators, including input-output, JIF, CWTS-crown index, H-index, Review panels and departmental ratings which were aggregated at university level to allocate block funding. These indicators were used in a funding formula that was re-visited and evaluated over the years in order to be able to address changing issues. Following the introductory presentation, Dr. Luwel presented a full case study of the Flemish approach to research evaluation, which was described in detail.


Image 1: From left: Dr. Marc Luwel, Managing Director, Flemish Agency for Research Infrastructure 'Hercules Stichting', Dr. John Francis, National Geographic Society, Dr. Abraham Wandersman, Professor, University of South Carolina, Dr. Rebecca Rosen, American Institutes for Research, Dr. Henk Moed, Senior Scientific Advisor, Elsevier

A science of science policy perspective was provided by Dr. Rebecca Rosen from the American Institute of Research (AIR) and a former National Science Foundation (NSF) staff member. Dr. Rosen gave an expansive overview of the work done by AIR and NSF with regards to collecting, analyzing and disseminating science-related data to assist research evaluation and science policy decision makers to reach conclusions in a timely and effective manner. Dr. Rosen gave specific examples of how NSF and AIR approach the data infrastructure challenges and the tools and methodologies built to address them. In addition, Dr. Rosen covered AIR and NSF efforts in the US, France, and Australia to integrate existing administrative, programmatic, and results databases into data platforms that feed novel portfolio visualization tools.

A unique and powerful social science perspective on evaluation was given by Dr. Abraham Wandersman from the University of South Carolina, Columbia. Dr. Wandersman, professor of Psychology, presented the Quality Implementation Tool, developed in order to address the need for an evaluative methodology that stems from empowerment and is goal- and quality-oriented. Dr. Wandersman gave examples of the model’s use in practice as well as an empowerment evaluation example using the Tool. The framework presented by Dr. Wandersman was co-developed by the Center for Disease Control CDC staff and university researchers to bridge the research-practice gap by integrating research-to-practice models with community-centered/practice-centered models.

The day was concluded by Dr. John Francis, the vice president for research, conservation, and exploration at the National Geographic Society (NGS). Dr. Francis described the various research funding grants given by NGS to scientific research and exploration. In addition to the grants NGS provides for basic and field research, Dr. Francis also discussed the various programs run by NGS in schools, colleges and its citizens-participation programs, which aim to connect people with nature and raise awareness for environmental issues around the world.  Dr. Francis focused on the various types of grants offered by NGS and the manner by which each one is evaluated in order to support exploration and discovery, natural and cultural conservation, and groundbreaking scientific fieldwork, all aimed at learning about and protecting our planet.

This seminar offered a comprehensive view of research evaluation and grant making and brought together diverse perspectives from industry, academia and government. Moreover, each presentation during the seminar covered not only different approaches to evaluation but also different practices showcasing topics such as health care, data infrastructure, bibliometrics, algorithms, psychology, and nature, and how each uses evaluative methodologies in practice.


VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)