Task performance is a method used to obtain quantitative data, which is most useful in doing comparative testing, or testing against predefined benchmarks; you obtain numbers to easily compare conditions. With task performance – as the name says – you let a subject perform a task, and measure the outcome. See for the example below (Voorhorst et al. 2000). Developing an interface for an augmented reality system we were challenged with the design of a fitting menu structure. We designed various options (conditions) and had subjects perform a simple task under each of the conditions while measuring time and as well as menu interaction. After analysing the test results we found that the difference of how menus were constructed had no impact on task completion, but did impact the strategy adopted to perform the task.
Tangible Menus – Beyond the desktop metaphor
Build-It [Rauterberg et.al. 1997] is an AR based planning tool based on intuitive manipulation for the support of planning and configuration tasks. It has a so-called graspable user interface. With build-it, the interaction proceeds by means of rectangular bricks that are tracked using an infra red detection system using a video camera. One of the potentials of this technical set-up is the possibility to take the menus beyond the desktop metaphor, as allows for looking at hardware solutions such as a real catalogue, from which objects can be selected directly (see Figure below). Implementation only requires a unique code on the page of the catalogue, that can be picked-up by the system’s infra red device. Potentially, the page layout itself can serve as coding.
The experiment was performed using a mock-up Build-it system, consisting of a cardboard frame on top of a table, and four real menus with real objects. The area below and extending beyond the borders of the frame represents the virtual environment (VE) and the area above the frame is the real environment (RE). Four menus were used each with different objects (rectangular, triangle, square or bar). Subjects were shown a configuration which they were asked to rebuild as accurately as possible.
Subjects were asked to perform as good as possible, not as fast as possible. Although, it was expected that the time needed to be less for easy conditions compared to difficult conditions, no time differences were found. However, results did indicate two types of strategies. Strategy 1 is the optimization of menu access and is used when menus are difficult accessible. Subjects optimize the number of changes between menus by accessing each menu only once. Strategy 2 is the optimization of task performance and is used when menus are easily accessible. Subjects selected freely from different menus, depending on the menu item that is needed.
Even though this technique of task performance is aimed at collecting quantitative data, it should be noticed that it is very important to collect qualitative data to uncover the user’s natural behaviors and mental processes driving the quantitative data and to take them into account while drawing the conclusions. The objective is to discover the good (effective? pleasurable?) interaction, and for a designer the experiment is a means to an end in this process of discovery.
Rauterberg, M., Fjeld, M., Krueger, H., Bichsel, M., Leonhardt, U. & Meier, M., 1997, BUILD-IT: a video-based interaction technique of a planning tool for construction and design.: Proceedings of Work With Display Units — WWDU’97, p. 175-176.
Voorhorst, F.A., Bichsel, M & Krueger, H. (2000) Menus beyond the desktop metaphor. Extended abstracts Conference on Human Factors. CHI’00. The Hague, The Netherlands. Pages 271-272.