Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by José Pedro Pinto

2017

A collaborative approach for semantic time-based video annotation using gamification

Authors
Viana, P; Pinto, JP;

Publication
HUMAN-CENTRIC COMPUTING AND INFORMATION SCIENCES

Abstract
Efficient access to large scale video assets, may it be our life memories in our hard drive or a broadcaster archive which the company is eager to sell, requires content to be conveniently annotated. Manually annotating video content is, however, an intellectually expensive and time-consuming process. In this paper we argue that crowdsourcing, an approach that relies on a remote task force to perform activities that are costly or time-consuming using traditional methods, is a suitable alternative and we describe a solution based on gamification mechanisms for collaboratively collecting timed metadata. Tags introduced by registered players are validated based on a collaborative scoring mechanism that excludes erratic annotations. Voting mechanisms, enabling users to approve or refuse existing tags, provide an extra guarantee on the quality of the annotations. The sense of community is also created as users may watch the crowd's favourite moments of the video provided by a summarization functionality. The system was tested with a pool of volunteers in order to evaluate the quality of the contributions. The results suggest that crowdsourced annotation can describe objects, persons, places, etc. correctly, as well as be very accurate in time.

2013

TAG4VD - A game for collaborative video annotation

Authors
Pinto, JP; Viana, P;

Publication
ImmersiveMe 2013 - Proceedings of the 2nd International Workshop on Immersive Media Experiences, Co-located with ACM Multimedia 2013

Abstract
Creating descriptive labels for videos is an important task, with application in video retrieval, Web accessibility and computer vision. However automatic creation of such labels is difficult and, alternatively, having professionals manually describing content is too expensive. Engaging end-users in the process of describing multimedia assets may lead to good results and enables creating the sense of participation which is currently one of the key factors to attract customers to a service. The existing approaches are highly successful in terms of number of engaged players and number of collected labels, but hardly create comprehensive tag sets, contributing both with generic or too narrow meaning tags. "Games With A Purpose" are one of the approaches that have been used in an attempt to create comprehensive video descriptions by harnessing the intelligence of human players and have them contributing and collaborating towards a common goal that is recognized if successful. This paper describes a game which implements two mechanisms for collecting data via human-based computation games. Tags introduced by registered players, in a given timecode, are validated based on a collaborative scoring mechanism that eliminates irregular annotations. Additionally, a voting mechanism that enables players to endorse or refuse existing tags, provides an extra instrument to guarantee the quality of the annotations. © 2013 ACM.

2015

Using the Crowd to Boost Video Annotation Processes: a game based approach

Authors
Pinto, JP; Viana, P;

Publication
CVMP 2015: PROCEEDINGS OF THE 12TH EUROPEAN CONFERENCE ON VISUAL MEDIA PRODUCTION

Abstract
This short paper presents a game for collecting metadata to describe video content. Tags, introduced by registered players on a given timecode of the video, are collected and validated based on a collaborative scoring mechanism that excludes erratic annotations. The system follows a gamification approach for motivating users and includes processes for semantically relating concepts.

2018

GymApp: a Real Time Physical Activity Trainner on Wearable Devices

Authors
Viana, P; Ferreira, T; Castro, L; Soares, M; Pinto, JP; Andrade, T; Carvalho, P;

Publication
2018 11TH INTERNATIONAL CONFERENCE ON HUMAN SYSTEM INTERACTION (HSI)

Abstract
Technological advances are pushing into the mass market innovative wearable devices featuring increasing processing and sensing capacity, non-intrusiveness and ubiquitous use. Sensors built-in those devices, enable acquiring different types of data and by taking advantage of the available processing power, it is possible to run intelligent applications that process the sensed data to offer added -value to the user in multiple domains. Although not new to the modern society, it is unquestionable that the present exercise boom is rapidly spreading across all age groups. However, in a great majority of cases, people perform their physical activity on their own, either due to time or budget constraints and may easily get discouraged if they do not see results or perform exercises inadequately. This paper presents an application, running on a wearable device, aiming at operating as a personal trainer that validates a set of proposed exercises in a sports' session. The developed solution uses inertial sensors of an Android Wear smartwatch and, based on a set of pattern recognition algorithms, detects the rate of success in the execution of a planned workout. The fact that all processing can be executed on the device is a differentiator factor to other existing solutions.

2019

YouTube Timed Metadata Enrichment Using a Collaborative Approach

Authors
Pinto, JP; Viana, P;

Publication
MULTIMEDIA AND NETWORK INFORMATION SYSTEMS

Abstract
Although the growth of video content in online platforms has been happening for some time, searching and browsing these assets is still very inefficient as rich contextual data that describes the content is still not available. Furthermore, any available descriptions are, usually, not linked to timed moments of content. In this paper, we present an approach for making social web videos available on YouTube more accessible, searchable and navigable. By using the concept of crowdsourcing to collect the metadata, our proposal can contribute to easily enhance content uploaded in the YouTube platform. Metadata, collected as a collaborative annotation game, is added to the content as time-based information in the form of descriptions and captions using the YouTube API. This contributes for enriching video content and enabling navigation through temporal links.