נעם טרקטינסקי

The assessment of the relative value of different design features for users is of great interest for software designers. Users' evaluations are generally measured through questionnaires. We suggest that other evaluation methods, including economic measures, may provide different estimates of the relative value of features. In a laboratory experiment we created four versions of a data-entry application by independently manipulating the system's usability and aesthetics. Users' evaluations of the four experimental systems were obtained in a within-subjects design. In addition, five between-subjects experimental conditions were created, based on the evaluation method (questionnaire alone or auction and questionnaire), monetary incentives (present or absent), and experience in using the system (present or absent). In questionnaire-based responses, the systems' usability affected evaluations of usability as well as aesthetics. Similarly, the systems' aesthetics affected evaluations of both aesthetics and usability. Questionnaire-based evaluations of usability and aesthetics were not affected by experience with the system or by monetary performance incentives. Auction bids were only influenced by the system's usability: bids corresponded to the objective performance levels that could be attained with the different systems. The results suggest that by using economic methods, researchers and practitioners can obtain system evaluations that are strongly related to performance criteria and that may be more valid when the evaluation context favors task-oriented performance.