Abstract
Background: Online psychological interventions like behavioural activation (BA) can be provided with or without human support. Unguided online interventions require no human contact and are therefore easier to implement on a large scale than guided interventions. However, effectiveness and adherence rates to these interventions are generally lower. One way to increase adherence to unguided online interventions is to offer automated motivational support. Objective: This pilot randomized controlled trial (RCT) examined whether adherence to unguided online BA for low mood could be improved by adding automated support in the form of smartphone-delivered personalized motivational messages or a motivational virtual coach. Methods: A three-arm pilot RCT (n = 106) was conducted that compared an online intervention delivered with automated motivational support by a virtual coach (n = 35), or by automated personalized messages on their smartphone (n = 35), to the same intervention without support (control condition; n = 36). The primary outcome was level of adherence, operationalized as (1) the number of webpages of the intervention visited, and (2) the number of mood ratings completed on the smartphone application, both retrieved from participants' logfiles. Secondary outcomes were satisfaction with the intervention (CSQ-I), usability (SUS) depression scores (HADS), and motivation for treatment (SMFL), measured through online questionnaires administered at baseline or after 4 weeks. Results: Adherence was moderate overall, with participants visiting on average 23 pages of 55 webpages and completing on average 50 of 84 requested mood ratings. No evidence for differences in adherence rates were observed between the intervention conditions and the control condition. Satisfaction with the intervention was moderate to high. Usability scores were below the desirable threshold of 68. Depression symptoms did not change significantly across all participants (p = .053). No significant changes in motivation were found over time or between groups. Conclusions: Adding automated support to unguided online BA for depression did not improve overall adherence. The limited effectiveness may reflect a misalignment between the motivational strategies and the needs of the target population, who reported mild symptoms and high intrinsic motivation. The findings highlight the need to further improve both the quality of automated support and the usability of online platforms. Future research should explore additional adherence-related factors and investigate how personalization can better address different symptom severities in unguided mental health interventions. Trial registration: International Clinical Trials Registry Platform: trialsearch.who.int/Trial2.aspx?TrialID=NL8110. © 2025 The Authors

Abstract

Abstract
Abstract Some performance-based research funding systems rely on expert-assigned journal rankings to allocate resources and guide research evaluation. In Finland, the JuFo system provides journal rankings, determined by experts who assess journals using available metadata, such as bibliometric indicators, alongside qualitative judgment. While prior work has explored machine learning approaches to approximate these rankings, the recent emergence of large language models (LLMs) offers new possibilities for automated, data-driven evaluation. In this study, we examine how well LLMs can replicate JuFo rankings when given the same structured information available to experts, including citation metrics, disciplinary assignments, and publisher metadata. We systematically compare LLM predictions to expert-assigned JuFo ranks using a confusion-matrix analysis to identify cases of alignment and deviation. Our research addresses two key questions: (1) how accurately LLMs estimate journal rankings, and (2) in which situations their predictions diverge from expert judgments and which factors explain these discrepancies. Our findings show that LLMs approximate expert-assigned rankings with high overall accuracy, with most errors occurring between adjacent levels. However, their performance varies systematically across disciplines, and they tend to under-predict top-tier journals, particularly in social sciences and humanities fields.

Abstract

Abstract

Abstract

Healthcare organizations increasingly rely on complex digital systems, but software onboarding often depends on manuals and classroom-based training that do not fit well with fast-paced clinical workflows. Interactive in-app guidance may better support learning during real work, although healthcare-specific evidence is still limited.