Category Archives: publication

Blind late fusion in multimedia event retrieval

Maaike H. T. de Boer, Klamer Schutte, Hao Zhang, Yi-Jie Lu, Chong-Wah Ngo, Wessel Kraaij

First Online:
08 November 2016
Accepted:

DOI: 10.1007/s13735-016-0112-9

Abstract

One of the challenges in Multimedia Event Retrieval is the integration of data from multiple modalities. A modality is defined as a single channel of sensory input, such as visual or audio. We also refer to this as data source. Previous research has shown that the integration of different data sources can improve performance compared to only using one source, but a clear insight of success factors of alternative fusion methods is still lacking. We introduce several new blind late fusion methods based on inversions and ratios of the state-of-the-art blind fusion methods and compare performance in both simulations and an international benchmark data set in multimedia event retrieval named TRECVID MED. The results show that five of the proposed methods outperform the state-of-the-art methods in a case with sufficient training examples (100 examples). The novel fusion method named JRER is not only the best method with dependent data sources, but this method is also a robust method in all simulations with sufficient training examples.

Evaluation of context-aware recommendation systems for information re-finding

Maya Sappelli, Suzan Verberne and Wessel Kraaij

Abstract

In this article we evaluate context-aware recommendation systems for information re-finding by knowledge workers. We identify 4 criteria that are relevant for evaluating the quality of knowledge worker support: context relevance, document relevance, prediction of user action, and diversity of the suggestions. We compare 3 different context-aware recommendation methods for information re-finding in a writing support task. The first method uses contextual prefiltering and content-based recommendation (CBR), the second uses the just-in-time information retrieval paradigm (JITIR), and the third is a novel network-based recommendation system where context is part of the recommendation model (CIA). We found that each method has its own strengths: CBR is strong at context relevance, JITIR captures document relevance well, and CIA achieves the best result at predicting user action. Weaknesses include that CBR depends on a manual source to determine the context and in JITIR the context query can fail when the textual content is not sufficient. We conclude that to truly support a knowledge worker, all 4 evaluation criteria are important. In light of that conclusion, we argue that the network-based approach the CIA offers has the highest robustness and flexibility for context-aware information recommendation.

Evaluation and analysis of term scoring methods for term extraction

 

  • Suzan Verberne
  • Maya Sappelli
  • Djoerd Hiemstra
  • Wessel Kraaij
  • 3.University of TwenteEnschedeThe Netherlands

Open AccessArticle

First Online:
10 August 2016
Received:
Accepted:

DOI: 10.1007/s10791-016-9286-2

Cite this article as:
Verberne, S., Sappelli, M., Hiemstra, D. et al. Inf Retrieval J (2016) 19: 510. doi:10.1007/s10791-016-9286-2

Abstract

We evaluate five term scoring methods for automatic term extraction on four different types of text collections: personal document collections, news articles, scientific articles and medical discharge summaries. Each collection has its own use case: author profiling, boolean query term suggestion, personalized query suggestion and patient query expansion. The methods for term scoring that have been proposed in the literature were designed with a specific goal in mind. However, it is as yet unclear how these methods perform on collections with characteristics different than what they were designed for, and which method is the most suitable for a given (new) collection. In a series of experiments, we evaluate, compare and analyse the output of six term scoring methods for the collections at hand. We found that the most important factors in the success of a term scoring method are the size of the collection and the importance of multi-word terms in the domain. Larger collections lead to better terms; all methods are hindered by small collection sizes (below 1000 words). The most flexible method for the extraction of single-word and multi-word terms is pointwise Kullback–Leibler divergence for informativeness and phraseness. Overall, we have shown that extracting relevant terms using unsupervised term scoring methods is possible in diverse use cases, and that the methods are applicable in more contexts than their original design purpose.

Detecting work stress in offices by combining unobtrusive sensors

Saskia Koldijk, Mark A. Neerincx, and Wessel Kraaij.

Detecting work stress
in offices by combining unobtrusive sensors.

IEEE Transactions on Affective Computing

 

Abstract:

Employees often report the experience of stress at work. In the SWELL project we investigate how new context aware pervasive systems can support knowledge workers to diminish stress. The focus of this paper is on developing automatic classiers to infer working conditions and stress related mental states from a multimodal set of sensor data (computer logging, facial expressions, posture and physiology). We address two methodological and applied machine learning challenges: 1) Detecting work stress using several (physically) unobtrusive sensors, and 2) Taking into account individual dierences. A comparison of several classication approaches showed that, for our SWELL-KW dataset, neutral and stressful working conditions can be distinguished with 90% accuracy by means of SVM. Posture yields most valuable information, followed by facial expressions. Furthermore, we found that the subjective variable `mental eort’ can be better predicted from sensor data than e.g. `perceived stress’. A comparison of several regression approaches showed that mental eort can be predicted best by a decision tree (correlation of 0.82). Facial expressions yield most valuable information, followed by posture. We nd that especially for estimating mental states it makes sense to address individual dierences. When we train models on particular subgroups of similar users, (in almost all cases) a specialized model performs equally well or better than a generic model.

Published in: IEEE Transactions on Affective Computing ( Volume: PP, Issue: 99 )

Page(s): 1 – 1

Date of Publication: 20 September 2016

ISSN Information:

Print ISSN: 1949-3045

DOI: 10.1109/TAFFC.2016.2610975

Publisher: IEEE

Knowledge based query expansion in complex multimedia event detection

Multimedia Tools and Applications

, Volume 75, Issue 15, pp 9025–9043

Knowledge based query expansion in complex multimedia event detection

  • Maaike de Boer
  • Klamer Schutte
  • Wessel Kraaij

Open AccessArticle

First Online:
12 July 2015
Received:
Revised:
Accepted:

DOI: 10.1007/s11042-015-2757-4

Cite this article as:
de Boer, M., Schutte, K. & Kraaij, W. Multimed Tools Appl (2016) 75: 9025. doi:10.1007/s11042-015-2757-4

A common approach in content based video information retrieval is to perform automatic shot annotation with semantic labels using pre-trained classifiers. The visual vocabulary of state-of-the-art automatic annotation systems is limited to a few thousand concepts, which creates a semantic gap between the semantic labels and the natural language query. One of the methods to bridge this semantic gap is to expand the original user query using knowledge bases. Both common knowledge bases such as Wikipedia and expert knowledge bases such as a manually created ontology can be used to bridge the semantic gap. Expert knowledge bases have highest performance, but are only available in closed domains. Only in closed domains all necessary information, including structure and disambiguation, can be made available in a knowledge base. Common knowledge bases are often used in open domain, because it covers a lot of general information. In this research, query expansion using common knowledge bases ConceptNet and Wikipedia is compared to an expert description of the topic applied to content-based information retrieval of complex events. We run experiments on the Test Set of TRECVID MED 2014. Results show that 1) Query Expansion can improve performance compared to using no query expansion in the case that the main noun of the query could not be matched to a concept detector; 2) Query expansion using expert knowledge is not necessarily better than query expansion using common knowledge; 3) ConceptNet performs slightly better than Wikipedia; 4) Late fusion can slightly improve performance. To conclude, query expansion has potential in complex event detection.

 

Adapting the Interactive Activation Model for Context Recognition and Identification

Maya Sappelli, Suzan Verberne, and Wessel Kraaij. 2016. Adapting the Interactive Activation Model for Context Recognition and Identification. ACM Trans. Interact. Intell. Syst. 6, 3, Article 22 (September 2016), 30 pages. DOI: http://dx.doi.org/10.1145/2873067

Abstract:

In this article, we propose and implement a new model for context recognition and identification. Our work is motivated by the importance of “working in context” for knowledge workers to stay focused and productive.

A computer application that can identify the current context in which the knowledge worker is working can (among other things) provide the worker with contextual support, for example, by suggesting relevant information sources, or give an overview of how he or she spent his or her time during the day.

We present a descriptive model for the context of a knowledge worker. This model describes the contextual elements in the work environment of the knowledge worker and how these elements relate to each other. This model is operationalized in an algorithm, the contextual interactive activation model (CIA), which is based on the interactive activation model by Rumelhart and McClelland. It consists of a layered connected network through which activation flows. We have tested CIA in a context identification setting. In this case, the data that we use as input is low-level computer interaction logging data.

We found that topical information and entities were the most relevant types of information for context identification. Overall the proposed CIA model is more effective than traditional supervised methods in identifying the active context from sparse input data, with less labelled training data.

Deriving Requirements for Pervasive Well-Being Technology From Work Stress and Intervention Theory: Framework and Case Study

Deriving Requirements for Pervasive Well-Being Technology From Work Stress and Intervention Theory: Framework and Case Study

1Radboud University, Institute for Computing and Information Sciences, Nijmegen, Netherlands

2TNO Netherlands Organisation for Applied Scientific Research, The Hague, Netherlands

3Leiden University, Leiden Institute of Advanced Computer Science, Leiden, Netherlands

4Delft University of Technology, Interactive Intelligence, Delft, Netherlands

ABSTRACT

Background: Stress in office environments is a big concern, often leading to burn-out. New technologies are emerging, such as easily available sensors, contextual reasoning, and electronic coaching (e-coaching) apps. In the Smart Reasoning for Well-being at Home and at Work (SWELL) project, we explore the potential of using such new pervasive technologies to provide support for the self-management of well-being, with a focus on individuals’ stress-coping. Ideally, these new pervasive systems should be grounded in existing work stress and intervention theory. However, there is a large diversity of theories and they hardly provide explicit directions for technology design.

Objective: The aim of this paper is to present a comprehensive and concise framework that can be used to design pervasive technologies that support knowledge workers to decrease stress.

Methods: Based on a literature study we identify concepts relevant to well-being at work and select different work stress models to find causes of work stress that can be addressed. From a technical perspective, we then describe how sensors can be used to infer stress and the context in which it appears, and use intervention theory to further specify interventions that can be provided by means of pervasive technology.

Results: The resulting general framework relates several relevant theories: we relate “engagement and burn-out” to “stress”, and describe how relevant aspects can be quantified by means of sensors. We also outline underlying causes of work stress and how these can be addressed with interventions, in particular utilizing new technologies integrating behavioral change theory. Based upon this framework we were able to derive requirements for our case study, the pervasive SWELL system, and we implemented two prototypes. Small-scale user studies proved the value of the derived technology-supported interventions.

Conclusions: The presented framework can be used to systematically develop theory-based technology-supported interventions to address work stress. In the area of pervasive systems for well-being, we identified the following six key research challenges and opportunities: (1) performing multi-disciplinary research, (2) interpreting personal sensor data, (3) relating measurable aspects to burn-out, (4) combining strengths of human and technology, (5) privacy, and (6) ethics.

JMIR Mhealth Uhealth 2016;4(3):e79

doi:10.2196/mhealth.5341

Citation

Please cite as:

Koldijk S, Kraaij W, Neerincx MA

Deriving Requirements for Pervasive Well-Being Technology From Work Stress and Intervention Theory: Framework and Case Study

JMIR Mhealth Uhealth 2016;4(3):e79

DOI: 10.2196/mhealth.5341

PMID: 27380749

KEYWORDS

Assessing e-mail intent and tasks in e-mail messages

http://www.sciencedirect.com/science/article/pii/S0020025516301438

Assessing e-mail intent and tasks in e-mail messages

  • a TNO, Anna van Buerenplein 1, Den Haag, The Netherlands
  • b Radboud University, Toernooiveld 300, Nijmegen, The Netherlands
  • c Universit di Milano Bicocca, Italy

Received 28 July 2015, Revised 4 February 2016, Accepted 4 March 2016, Available online 12 March 2016

Elsevier: Information Sciences

download