The Meaning, Development and Evaluation of Natural User Interface in Education: An Empirical Study with Pen-Based Interaction

The emergence of Natural User Interface (NUI) two decades ago promised a new level of human-computer interaction by leveraging human sensorimotor skills such as touching, speaking, and gazing. The touchscreen was an undeniable example of success. Nevertheless, applying NUI in education effectively is impeded by gaps that are worthy to investigate. Naturalness in NUI has been debated due to the fuzziness of the term ‘natural’. Further, development and evaluation approaches of the previous graphical user interface (GUI) are being applied to NUI despite the contrast between the GUI and NUI.

In this PhD research, these issues were approached empirically. A critical review of literature revealed two potential interpretations of naturalness, innate naturalness and intuitive naturalness. Further, requirements for developing and evaluating NUIs were extracted, and systematic approaches were introduced accordingly. The evaluation approach was multidimensional, covering technical assessment as well as user performance and experience. A learning case study that involved pen-based interaction was employed to validate the soundness of the approaches and to examine the relevance of naturalness interpretations.

Two functional smartpen prototypes with their formal models were produced using the development approach. One prototype had better accuracy and ergonomics, while the other covered more interactions. The technical assessment of the evaluation approach suggested excluding theprototype with lower accuracy from assessments with human subjects. The performance assessment with 56 human subjects indicated no significant difference in subjects' performance when they used innateness-based NUI in comparison to intuitiveness-based NUI. However, the user experience assessment indicated a preference towards using the intuitiveness-based NUI.

Overall, this research provided the empirical support that the perception of naturalness is significantly linked to prior experiences rather than to innate abilities. The development and evaluation approaches formed a systematic and pragmatic framework for applying NUI in education. Limitations and further research directions were discussed.