Visual Analytics

Stress at work and unhealthy working habits can have negative consequences on the health of knowledge workers. Think of a disturbed work-life balance or worse: burn-out. Often there are early warning signs in the way we behave or feel. With unobtrusive sensors we want to capture behavior and experience and make knowledge workers aware of alarming (working) patterns. In this way the user is empowered to take action in time and stay healthy.

It is relatively easy to collect loads of sensor data about a user. But it is quite hard to make this data insightful to the user. In cooperation with the Information Visualization and Visual Analytics group at the Fraunhofer Institute we investigate possibilities of data visualization. We address 2 challenges: 1) Which data is most suitable to detect periods of stressful working? And 2) how can we visualize this data to make it interpretable for the user. Based on the insights gained, possibilities for integrating new visualizations in SWELL products are explored.

For our analysis we used our previously collected SWELL Knowledge Work dataset, with data on computer activity, facial expressions, posture, physiology and subjective experience. We identified the most prominent features for predicting a users mental effort. These were mainly facial expression features.

The best features were then plotted. The resulting visualizations, however, did not seem insightful yet. Three main challenges were identified:

  1. many fluctuations in the sensor data over time,
  2. individual differences between users, and
  3. difficulties in visualizing Facial Action Units and Postures.

To address (1) we tried using a moving average. To address (2) we applied clustering methods to the data. We found several clusters for the different sorts of data: Users differed e.g. in their computer behavior or in the facial expressions they showed. It is hard to find single features that work well for all users. A personalized visualization approach thus seems most valuable. Depending on the specific user (or user group), specific features might be most insightful. To address (3) and visualize Facial Action Units, we decided to apply Self-organizing Maps (SOM) to find typical combinations of facial activity (= facial expressions) that are related to high mental effort. The idea is that rendering typical facial expressions is probably more insightful than simple line charts of Facial Action Units activation levels.

Some of the visualizations made during the internship can be found here:


Anna van Buerenplein 1

2595 DA Den Haag

T 088 866 70 00

F 088 866 70 57

Prof. Dr. ir. Wessel Kraaij

RSS Feeds
Disclaimer    |    Privacy statement    |    © 2016 TNO