Expand Miniature: Selection based on Progressive Refinement in Dense VR Environments
Independent Research - Spring 2024 - XR Collaboratory, Cornell Tech
Student: Sissel Sun
Overview
This package is an implementation of two progressive refinement techniques, expand and world-in-miniature, as well as a combination of them, expand miniature.
Components
This package contains three progressive refinement techniques, Expand, WorldInMiniature and ExpandMiniature. They use RefineSelectState as a state indicator and attach a Proxy component for each instantiated proxy. There is a CustomXRTint component for highlighting proxies.
It also includes a multi-object selection technique, CylinderSelect that is based on MultiObjInteractor, a direct interactor inherited from XRDirectInteractor.
See architecture diagram below for details.
Expand
This component implements the logic for the expand technique. This component holds a reference to an interactor, an interaction manager and a list of objects passed by a multi-object interaction technique. It creates a proxy for each object selected and aligns them in a 2D plane for the user to pick one as final selection.
| Property | Description |
|---|---|
| Enable Multi Obj Interactor | Event that enables the multi-object interactor for initial select. This allows integration for multi-object interactors of custom type. |
| Disable Multi Obj Interactor | Event that disables the multi-object interactor for initial select. This allows integration for multi-object interactors of custom type. |
| Ray Interactor | The ray interactor responsible for selecting one of the proxies for expand. |
| Interaction Manager | The XR interaction manager that is responsible for the interactor. |
| Selected | The transforms of objects of interest. Each object will be assigned a proxy in the expand panel. |
| Distance To Player | How far in z axis the arranged proxies should be placed in front of the player. |
| Max Cols | The maximum number of columns for the expand panel. |
| Max Rows | The maximum number of rows for the expand panel. |
ExpandInput
This component handles user input and updates the Expand component accordingly.
| Property | Description |
|---|---|
| Activate | The input action reference for activating the expand panel. |
ExpandFeedback
This component is responsible for providing feedback for the expand technique. This includes rendering a grabbable panel that aligns the instantiated proxies for each object selected. This component also visualizes a line that connects the proxy of the object and the original object if the proxy is on hover. Both proxy and its original object will be highlighted on hover.
| Property | Description |
|---|---|
| Panel Prefab | Prefab used for the 2D grabbable panel. |
| Line Indicator | Whether to display a line that connects the proxy and the original object if on hover. |
WorldInMiniature
This component implements the logic for world-in-miniature. This component holds a reference to an interactor for selecting proxies inside the world-in-miniature, another interactor for grabbing and rotating the world-in-miniature, and a list of objects passed by a multi-object interaction technique.
| Property | Description |
|---|---|
| Enable Multi Obj Interactor | Event that enables the multi-object interactor for initial select. This allows integration for multi-object interactors of custom type. |
| Disable Multi Obj Interactor | Event that disables the multi-object interactor for initial select. This allows integration for multi-object interactors of custom type. |
| Ray Interactor | The ray interactor responsible for selecting proxies inside the world-in-miniature. |
| World Interactor | The interactor that is responsible for grab and rotate the world-in-miniature, instead of selecting proxies within. |
| Attach Transform | The transform that the world-in-miniature is attached to each time it is created. |
| World Size | The size of the world-in-miniature. |
| Selected | The list of transforms of interest. Each object will be assigned a proxy in the mini world. |
WorldInMiniatureInput
This component handles user input and updates the WorldInMiniature component accordingly.
| Property | Description |
|---|---|
| Activate | The input action reference for activating the world-in-miniature. |
WorldInMiniatureFeedback
This component is responsible for providing feedback for the world-in-miniature. This includes rendering a grabbable mini world as a sphere that contains the proxies for each object selected. This component also visualizes a line that connects the proxy of the object and the original object if the proxy is on hover. Both proxy and its original object will be highlighted on hover.
| Property | Description |
|---|---|
| World Prefab | Prefab used as the spherical boundary of the the mini world. |
| Line Indicator | Whether to display a line that connects the proxy and the original object if on hover. |
ExpandMiniature
Logic component for ExpandMiniature. It combines Expand and WorldInMiniature and syncs the selected original object for both techniques if any proxy of Expand or WorldInMiniature has been selected.
ExpandMiniatureInput
This component handles user input and updates the ExpandMiniature component accordingly.
| Property | Description |
|---|---|
| Activate | The input action reference for activating the expand panel and the world-in-miniature. |
ExpandMiniatureFeedback
Feedback component for ExpandMiniature. It renders indication lines between proxies and their corresponding objects on hover.
| Property | Description |
|---|---|
| Expand Line Indicator | Whether there is a line indicator from the expand proxy to the original object. Overrides the lineIndicator property of ExpandFeedback. |
| WIM Line Indicator | Whether there is a line indicator from the world-in-miniature proxy to the original object. Overrides the lineIndicator property of WorldInMiniatureFeedback. |
Proxy
The component that is attached to any proxy of an interactable. It stores the original interactable of the proxy, sets the interaction mask accordingly and provides a line renderer for connecting with the original interactable. It is automatically attached to any proxy when it's instantiated.
RefineSelectState
The state for Expand and WorldInMiniature. The sequence of states is as follows: InitialSelect -> RefineSelect -> ForceHover.
| State | Description |
|---|---|
| InitialSelect | The state when the multi-object interactive is active. The user aims to select a set of objects. |
| RefineSelect | The state when a set of objects is selected and their proxies are created. The user aims to select one of these objects. |
| ForceHover | The state when the user selects one of multiple objects. The ray interactor is locked on this object with a curved line pointing to it. The user can follow the line to find the object and select it. |
CustomXRTint
Simple Interactable Visual component that demonstrates hover or selection state with emissive tinting based on XRTintInteractableVisual. It adds a public function for manually setting the tint.
MultiObjInteractor
Custom direct interactor for multi-object selection. It inherits XRDirectInteractor.
CylinderSelect
Logic component of the multi-object selection technique based on a cylinder volume of adjustable size.
| Property | Description |
|---|---|
| Interactor | The interactor of responsible for selecting multiple objects. |
| Offset | The offset of the cylinder's attach transform. |
| Min Radius | The minimum radius of the cylinder. |
| Max Radius | The maximum radius of the cylinder. |
CylinderSelectInput
Input component for cylinder select. It handles user input for changing the cylinder's radius.
| Property | Description |
|---|---|
| Change Radius | The input action reference for changing the cylinder's radius. |
CylinderSelectFeedback
Feedback component of the multi-object selection technique based on a cylinder volume of adjustable size. It provides feedback based on the interactor's hover and select events by changing the color of the cylinder to hoverColor.
| Property | Description |
|---|---|
| Material | The material for the cylinder. |
| Color | Color of the cylinder when the interactor is not hovering over any object. |
| Hover Color | Color of the cylinder when the interactor is hovering over any object. |
Installation instructions
To install this package, follow these steps:
- In the Unity Editor, click on Window > Package Manager
- Click the + button and choose Add package from git URL option
- Paste the URL to the package repository: https://github.com/xrc-students/xrc-students-sp2024-ir22-sun.git in the prompt and click on Add (make sure your URL ends with ".git")
- If the repository is private, you will be asked to authenticate via your GitHub account. If you haven't been granted access to the repository you will not be able to install the package.
- The package should be installed into your project
- You can download the package samples from under the Samples tab in the Package Manager
Note: Even though the package documentation is public, several XRC packages are private and accessible only to XRC staff and students.
Requirements
This package was developed and tested using the following Unity Editor version:
- 2022.3.7f1
Dependencies:
- XRC Core
- XR Interaction Toolkit
- Input System
Workflow
See the below screenshots for a scene hierarchy and the inspector for the game objects containing expand, world-in-miniature, expand miniature and cylinder select components.
Samples
Expand Miniature Demo
A demo of Expand Miniature with CylinderSelect as the multi-object selection.
Controls:
- Right grip: select object
- Right thumbstick: adjust cylinder size
Expand Miniature with Lasso Select
A demo of Expand Miniature with lasso select for multi-object selection. Note: this sample requires lasso select.
Controls:
- Right grip: select object
- Right trigger: draw lasso
Note: both samples require an interaction layer named Proxies in the project.