How to improve the user experience in online virtual environments? A look back at the research project of one of our collaborators.
For Inetum, the research approach is crucial to bring our solutions to market. In order to improve interactions within virtual worlds based on UMI3D, we investigated how to improve the most common interaction in a 3D virtual environment (VE): selection.
What Is UMI3D?
UMI3D is an open-source technology to build collaborative virtual environments developed at the innovation department of Inetum, and the basis of our no-code solution Intraverse.
What is the selection process?
Selection is a process where the users indicate the object they intend to interact with. It is the first actual interaction they perform in the VE before being able to manipulate objects, i.e. to move them or change their properties such as colors. Selection could be performed in two ways:
- By simulating a hand in the VE and grabbing objects directly, an approach known as virtual hand,
- By pointing a direction to select a distant targeted object known as virtual pointing.
We have focused on this latter approach as pointing is possible on every device (indeed, mouse selection is basically virtual pointing), while grabbing hardly makes sense on a desktop device with the most common controllers that are a keyboard and a mouse.
Fig 1. Selection in a virtual environment
But what are the problems that we are facing with selection in 3D virtual environments?
Firstly, there are challenges from the environment, such as occlusion. This problem happens when the object you want to select is hidden behind another, so it happens often in 3D. Dense environments with a large number of objects also make it hard to select a specific target object without selecting another one. It makes occlusion problems even worse too. It is also especially hard to select small or moving objects.
Secondly, some challenges come from the way we, humans, interact with the environment. Users are prone to hands jittering, making it hard to point an exact direction. Moreover, there are some other well-known effects like the Heisenberg effect of selection, which happens when the user presses a physical button on their controller, moving the controller by a bit and thus changing the pointed direction.
Fig 2. Challenge in 3D selection in online VEs
The most common method implemented for 3D selection is known as Ray-casting. It is a method that cast a ray from the controller and allows the user to select the first object the ray hits. It is a method easy to implement and to provide feedback for, but it suffers from all the challenges listed before.
Recommended by LinkedIn
The objective of our study was therefore to identify a selection intention prediction method that has good performance and a pleasant user experience for use in an online VE with both a desktop device and a VR device.
So, what are credible selection intent prediction techniques alternatives to Ray-casting in online virtual environments?
Lots of methods have been developed, however there is little to no comparison work between them. We then have chosen a few reference methods to compare them. Namely IntenSelect, KEP, Kalman-KEP and MTP trees.
How to evaluate and measure the performance and usability gain of the chosen selection methods?
To evaluate the methods on performance and user experience, two experiences have been run. The first one aimed to gather selection tracking data that was used to compute metrics and to train the ML approach. The second one was focused on a hands-on user evaluation of each of the method in a simple 3D selection scenario to properly get user feedback.
We opened our experience to collaborators from Inetum, and the experience had sixty participants of various ages and with diverse VR experience.
What to keep from our study?
In this work, we have chosen and compared five techniques based on previous works to select 3D objects in online virtual environments. A small dataset of ray-cast selections has been created and served to evaluate the availability performance of some techniques through simulations on the recorded data. Finally, a user evaluation concerning the usability and induced workload of the methods has been performed.
Even though all the simulated methods provide better temporal and spatial availabilities than the Ray-casting technique, the latter is still preferred by the users on both usability and workload. However, IntenSelect and the adapted KEP method, have shown that they were not worse than Ray-casting and sometimes benefit from a small advantage due to their design. Thus, we consider that they are credible alternatives to Ray-casting and decided to integrate them in UMI3D browsers, with a priority for immersive ones.
Possible future work includes a study in order to tune the IntenSelect coefficients, an improvement of the current KEP adaptation to 3D and real-time by separating the ballistic and corrective phase or a hands-on study of the performance of the methods, without simulations.
References:
Precise methodology and analysis can be found in the full research document, as well as all the references to previous work:
This study was conducted by Charles-Eole Marichal , XR innovation Engineer, for his master thesis at Inetum.