Abstract

Abstract

Abstract
Abstract Some performance-based research funding systems rely on expert-assigned journal rankings to allocate resources and guide research evaluation. In Finland, the JuFo system provides journal rankings, determined by experts who assess journals using available metadata, such as bibliometric indicators, alongside qualitative judgment. While prior work has explored machine learning approaches to approximate these rankings, the recent emergence of large language models (LLMs) offers new possibilities for automated, data-driven evaluation. In this study, we examine how well LLMs can replicate JuFo rankings when given the same structured information available to experts, including citation metrics, disciplinary assignments, and publisher metadata. We systematically compare LLM predictions to expert-assigned JuFo ranks using a confusion-matrix analysis to identify cases of alignment and deviation. Our research addresses two key questions: (1) how accurately LLMs estimate journal rankings, and (2) in which situations their predictions diverge from expert judgments and which factors explain these discrepancies. Our findings show that LLMs approximate expert-assigned rankings with high overall accuracy, with most errors occurring between adjacent levels. However, their performance varies systematically across disciplines, and they tend to under-predict top-tier journals, particularly in social sciences and humanities fields.

Abstract

Abstract
In this paper, we continue our investigations on digital nomadism and the impact of COVID-19 pandemic on the work-related aspects and lifestyle of digital nomads (DN). The findings presented in this empirical study reflect the analysis of the impact of COVID-19 outbreak (and its waves) on the market economy and work-life boundaries of DNs as perceived from posts and comments gathered from a Reddit community during the period of early March 2020 until the end of 2022. From this point, our results indicate that the massification of remote work among formal workers in response to COVID-19 pandemic has impacted both the formal labor market and the DN ecosystem. As a consequence, we argue that digital nomadism tends to play a critical role beyond work from (almost) anywhere (WFA) in a post-COVID-19 era taking into account the novel facets of nomadic work-lifestyle. © 2023 IEEE.

Abstract