Abstract

Abstract
Abstract Some performance-based research funding systems rely on expert-assigned journal rankings to allocate resources and guide research evaluation. In Finland, the JuFo system provides journal rankings, determined by experts who assess journals using available metadata, such as bibliometric indicators, alongside qualitative judgment. While prior work has explored machine learning approaches to approximate these rankings, the recent emergence of large language models (LLMs) offers new possibilities for automated, data-driven evaluation. In this study, we examine how well LLMs can replicate JuFo rankings when given the same structured information available to experts, including citation metrics, disciplinary assignments, and publisher metadata. We systematically compare LLM predictions to expert-assigned JuFo ranks using a confusion-matrix analysis to identify cases of alignment and deviation. Our research addresses two key questions: (1) how accurately LLMs estimate journal rankings, and (2) in which situations their predictions diverge from expert judgments and which factors explain these discrepancies. Our findings show that LLMs approximate expert-assigned rankings with high overall accuracy, with most errors occurring between adjacent levels. However, their performance varies systematically across disciplines, and they tend to under-predict top-tier journals, particularly in social sciences and humanities fields.

Abstract

Abstract

Abstract

Healthcare organizations increasingly rely on complex digital systems, but software onboarding often depends on manuals and classroom-based training that do not fit well with fast-paced clinical workflows. Interactive in-app guidance may better support learning during real work, although healthcare-specific evidence is still limited.

Abstract
Software architecture is reflected across multiple artifacts, making it difficult to communicate without proper documentation, which often becomes outdated or unreliable. We propose an approach to support Living Documentation by generating architectural diagrams from Docker Compose files. We implement our approach as a prototype tool that we name Infragenie and conduct an empirical study to show the viability of the approach. The study involved sending questionnaires to maintainers of 378 GitHub repositories. We received 36 responses. Infragenie-generated diagrams were rated as better or much better for most of the 12 projects with previous diagrams. Over 70% of the respondents agreed that our approach improved documentation completeness, consistency, and accessibility, and more than 90% recognized its effectiveness in capturing key architectural elements. We conclude that by using Docker Compose files we were able to provide useful architectural diagrams.