This blog text starts a series of three texts which address various aspects of publication and citation metrics available on Vipunen portal. Texts will be published in English to reach a wider audience than would be possible in Finnish. The first post describes overall content of these metrics, their responsible use, and use cases. The following posts, to be published in the Fall of 2023, will delve deeper into open access publishing and the differences of Web of Science and Scopus databases.
Vipunen.fi offers possibility to use bibliometric data and indicators to produce analyses on scientific publishing, citation impact and research collaboration based on Web of Science (WoS) and Scopus databases. Both Wos and Scopus are well-known and widely used databases gathering data on scientific publications and citations. Finnish national guide for publication metrics has more information on WoS and Scopus.
For international comparisons, Vipunen comprises data and indicators on number of publications, citation impact and collaboration. In case of Finland, Vipunen users can also look at publications, citations and collaboration based on sector-level data and indicators, and even drill down to the level of organisations in some sectors (universities, universities of applied sciences, state research institutes and university hospitals). Fields of science can also be taken into account with Vipunen bibliometrics. Furthermore, Web of Science based indicators on Vipunen offer a breakdown of scientific publishing according to open access publishing and according to countries which researchers in Finland have authored papers with.
In recent years, data coverage of Vipunen’s bibliometric indicators have been widened by including more publication types from WoS and Scopus. In the case of WoS data, Emerging Sources Citation Index was included in 2022. In regard to citation impact, basic bibliometric reports on Vipunen use the Top10 index which describes the proportion of the most cited publications. The world average is 1 and an index value greater than 1 means that more than 10% of publications are among the 10% most cited publications. When calculating the Top10 index, the number of citations to a publication is only compared to the number of citations of publications appearing in the same discipline in the same year.
While various platforms use more or less identical names for citation based impact indicators (e.g., Top10 index), the actual numbers often differ. There are several reasons behind this phenomenon. First, there are differences between the underlying databases as to which publications are included. Secondly, the affiliation information may be erroneous or misinterpreted so that a publication is assigned to different organizations in different databases. The results shown in Vipunen are not based on the affiliation fields provided in the data; instead the full address information is analyzed for every Finnish address. A third reason for conflicting results is that the computational algorithms behind the indicators differ in several details. The methods implemented in Vipunen are designed to guarantee comparable results for all subject fields, years of publication and countries. It’s worth remembering that even though the indicator values may vary depending on the source, the qualitative results usually agree rather well. If country A has higher Top10 index than country B on a certain year in one indicator platform, this tends to be true for other platforms as well. A similar reasoning is applicable for year by year development of the indicator values for a fixed country or organization.
When producing and using bibliometric analyses it is important to understand, for example, what kinds of limitations there are in terms of the data used, what different indicators can actually indicate and what kinds of risks are involved. As already mentioned above, the coverage in both WoS and Scopus varies significantly between different scientific fields. While natural sciences and medicine have quite good coverage, for social sciences and especially humanities the coverage can be even less than 10 percent. This means that when looking at the citation impact of publications based on the WoS or Scopus citation data in the field of humanities, what we see is only a fraction. And even in the case of good coverage, a high Top10 index value does not equal quality. Using bibliometric indicators carries risks especially when used for ranking or rewarding, and even more so when used at the level of individual researchers or research groups. Using them at the level of countries, scientific fields or organisations, however, is quite safe – as long as comparisons between different scientific fields are not done. For more information and detailed guidelines on the responsible use of publication metrics, please see the Finnish national guide for publication metrics.
The Academy of Finland (AKA) has been using bibliometric reports from Vipunen for several years as part of its knowledge production for Finnish science policy making. Central element of this knowledge production have been the State of Scientific Research reviews. In these reviews bibliometrics from Vipunen has been used extensively to describe the number of publications, citation impact and research collaboration. State of Scientific Research reviews scrutinise scientific publishing from several perspectives, e.g., number of publications and citation impact across fields of science in Finland compared with other countries, Finland’s most typical co-publishing partners, different types of co-publishing and their influence on citation impact, and scientific publishing across fields of science in Finnish universities. Both WoS and Scopus based bibliometrics have been utilised in these reviews since 2021 while WoS was the sole data source before that. Forthcoming reviews will include more information on open access publishing.
Bibliometrics from Vipunen are also used for smaller analyses at the AKA, such as providing background information for AKA’s international policy or monitoring the progress of open access publishing in Finland. In addition, Vipunen bibliometrics provide an important point of reference for bibliometric analyses on the scientific impact of research funded by the AKA. In these analyses, citation impact of AKA-funded research is often compared with citation impact of Finnish research as a whole (read more).
In universities, Vipunen is used mainly for positioning within the Finnish context as it provides compatible and trustworhty data on various inputs and outputs of all Finnish universities. However, the bibliometric reports from Vipunen are utilised less than analyses based on data collected by the Ministry of Education and Culture from the universities. The bibliometric reports from Vipunen are useful for national benchmarking either at the level of organisation or field of science, but universities want to dig deeper – how about Faculties or even departments? Universities most often wish to position nationally and internationally based on their organisational structure, and organisational structure does not follow the fields of science of Vipunen’s bibliometric reports. Therefore, when needed universities either order their bibliometric analyses from external actors like the Centre for Science and Technology Studies at the Leiden University or produce them internally using analysis tools such as InCites by Clarivate or Scival by Scopus.
Text by Valeria Caras, Otto Auranen, Laura Himanen ja Yrjö Leino
Tilastoneuvos 