Abstract
The study of data quality in crowdsourcing campaigns is currently a prominent research topic, given the diverse range of participants involved. A potential solution to enhancing data quality processes in crowdsourcing is cognitive personalization, which involves appropriately adapting or assigning tasks based on a crowd worker’s cognitive profile. There are two common methods for assessing a crowd worker’s cognitive profile: administering online cognitive tests, and inferring behavior from task fingerprinting based on user interaction log events. This article presents the findings of a study that investigated the complementarity of both approaches in a microtask scenario, focusing on personalizing task design. The study involved 134 unique crowd workers recruited from a crowdsourcing marketplace. The main objective was to examine how the administration of cognitive ability tests can be used to allocate crowd workers to microtasks with varying levels of difficulty, including the development of a deep learning model. Another goal was to investigate if task fingerprinting can be used to allocate crowd workers to different microtasks in a personalized manner. The results indicated that both objectives were accomplished, validating the usage of cognitive tests and task fingerprinting as effective mechanisms for microtask personalization, including the development of a deep learning model with 95% accuracy in predicting the accuracy of the microtasks. While we achieved an accuracy of 95%, it is important to note that the small dataset size may have limited the model’s performance.

Abstract
With the widespread availability and pervasiveness of artificial intelligence (AI) in many application areas across the globe, the role of crowdsourcing has seen an upsurge in terms of importance for scaling up data-driven algorithms in rapid cycles through a relatively low-cost distributed workforce or even on a volunteer basis. However, there is a lack of systematic and empirical examination of the interplay among the processes and activities combining crowd-machine hybrid interaction. To uncover the enduring aspects characterizing the human-centered AI design space when involving ensembles of crowds and algorithms and their symbiotic relations and requirements, a Computer-Supported Cooperative Work (CSCW) lens strongly rooted in the taxonomic tradition of conceptual scheme development is taken with the aim of aggregating and characterizing some of the main component entities in the burgeoning domain of hybrid crowd-AI centered systems. The goal of this article is thus to propose a theoretically grounded and empirically validated analytical framework for the study of crowd-machine interaction and its environment. Based on a scoping review and several cross-sectional analyses of research studies comprising hybrid forms of human interaction with AI systems and applications at a crowd scale, the available literature was distilled and incorporated into a unifying framework comprised of taxonomic units distributed across integration dimensions that range from the original time and space axes in which every collaborative activity take place to the main attributes that constitute a hybrid intelligence architecture. The upshot is that when turning to the challenges that are inherent in tasks requiring massive participation, novel properties can be obtained for a set of potential scenarios that go beyond the single experience of a human interacting with the technology to comprise a vast set of massive machine-crowd interactions.

Abstract
Crowdsourcing has received considerable attention over the last fifteen years and has been the subject of several experiments that demonstrate its large potential for use in real-world situations. With the rapid growth of and access to crowd work environments, there is a need for new ways to ensure more equitable access for all people. Task design is one of the core aspects of the crowdsourcing process and its optimization is a priority for many requesters that want to have their tasks solved in short times and with high levels of accuracy. Aligned with this goal, a cognitive personalization framework can make it feasible to assess the information processing preferences of crowd workers in order to provide a useful user interface (UI) adaptation. In an effort to address this issue, this study recruited a total of 64 crowd workers to take cognitive style tests and perform prototypical tasks. The results indicate that it is possible to apply short tests and then obtain some useful indicators for better matching tasks to workers with implications for improving the general outcomes and acceptance rates in crowdsourcing.

Abstract

Abstract
In recent years, digital technologies have been used to support discussions about the city and also to involve citizens in participatory public processes. However, despite the widespread use of social media platforms, old issues related to engagement and participation still persist in digital initiatives. The main goal of this study is to carry out an empirical evaluation of a collective intelligence model that combines crowdsourcing and social storytelling to support discussions about the city from a bottom-up perspective. Within a design science research approach we designed a participatory action study that was carried out through a workshop with students and professionals from different areas, such as architecture, urban design and information technology. As a result, we were able to assess whether the collective intelligence model was acceptable to the participants by investigating whether the behavioral assumptions were valid and thus outlining some contributions to the field of urban informatics.