An inconvenient thesis?

Christine Elizabeth Wania’s phd thesis questions the empirical evidence for the benefits of pattern languages in HCI:

For more than two decades much of the pattern language literature, within the field of Human Computer Interaction (HCI), has focused on the possible benefits pattern languages may provide, but there has been very little empirical work to support these claims. It has been suggested that interaction patterns or pattern languages in HCI may address some of the problems inherent in designing interactive systems by supporting reuse, capturing design knowledge, enabling the sharing of design knowledge, and facilitating communication among designers and users. This study examined the impact of a pattern language on the design of information retrieval interfaces, in terms of the quality of the interfaces and the time to design the interfaces. Participants created paper and pencil interfaces based on the given design task. Participants were exposed to either a pattern language, guidelines, or no structuring technique. There were no statistically significant differences between the three groups in terms of the quality of the interfaces and time to design the interfaces. The results of this study suggest that the value of pattern languages in HCI may not be in reuse, at the early stages of design, or in terms of the quality of the resulting designs, in domains familiar to designers. Although there was no apparent impact of the pattern language on the early stage designs, the results of a follow-up study suggest there is a significant correlation between the existence of patterns in commercial systems and the overall usability of those systems. Therefore, we suggest that we, as a community, very closely examine the current state of pattern languages in HCI before continuing to move forward. As a community, we need to shift our focus away from discussing the possible benefits of pattern languages and trying to build pattern collections. And instead, focus on trying to fully understand the value of pattern languages in HCI. In doing so, the HCI community, will then begin to see the benefits from all the great efforts in this area.

About time someone asked this question. I mean, I’ll swear by my favourite patterns and pattern languages, defend them as if they where my children flesh and blood, but do we have any evidence, in the scientific sense? In a way, this brings back the discussion on validity, resonance and aggregation. Yes, patterns work great for those who believe in them. But shouldn’t we aim higher?