Examining design choices for questionnaires in VR user studies

Questionnaires are among the most common research tools in virtual reality (VR) user studies. Transitioning from virtuality to reality for giving self-reports on VR experiences can lead to systematic biases.

VR allows to embed questionnaires into the virtual environment which may ease participation and avoid biases. To provide a cohesive picture of methods and design choices for questionnaires in VR (inVRQ), we discuss 15 inVRQ studies from the literature and present a survey with 67 VR experts from academia and industry.

Based on the outcomes, we conducted two user studies in which we tested different presentation and interaction methods of inVRQs and evaluated the usability and practicality of our design. We observed comparable completion times between inVRQs and questionnaires outside VR (nonVRQs) with higher enjoyment but lower usability for \inVRQs. These findings advocate the application of inVRQs and provide an overview of methods and considerations that lay the groundwork for inVRQ design.

Takeaways

Based on the results from four presented studies we conclude:

• researchers should apply inVRQs in their user studies
• pointing and world-anchoring are usable ways to realise inVRQs
• participants prefer using inVRQs over out-VRQs
• researchers should be aware of slightly raised physical and mental demands when using inVRQs

Similar to the establishment of standardized questionnaires that are empirically validated, we propose moving towards standardizing questionnaire implementation and presentation methods in VR, similar to a quasi-standardization that has already occurred in more traditional screen-based questionnaires due to the prevalence of selected survey tools, such as Google Forms or LimeSurvey.

Future work will need to seek for comparative experimental evidence on how the questionnaire modality affects the reliability of the the measurements. As with most design choices, there is no absolute right or wrong.