Multi-Object Image Plane Lasso Selection
Independent Research - Spring 2024 - XR Collaboratory, Cornell Tech
Student: Connor Pugh
Overview
This package is an implementation of the multi-object image plane lasso technique. Users can draw a line in 3D space, which they use to create a lasso in the virtual scene. When the users draws a loop, any interactables that appear to be within the lasso from the user's perspective will be hovered over, and can then be selected.
In addition, users can perform a simple "line selection" by drawing a line that does not loop - upon which the technique will instead find objects that lie along the line from the user's perspective.
Components
LassoSelect
This component contains the logic for multi-object image plane lasso selection. Using this technique, a user can draw a 3D line in the air, and all selectable objects that appear to be within the line or lasso from the user's perspective are hovered over & can then be selected.
This selection technique consists of 3 distinct phases:
The user draws a line using their controller.
When the line is finished, it forms a loop if possible. The technique then uses screen-space raycasting to detect what objects appear to be on the line from the user's perspective - or within the loop if it's a loop - and hovers over them.
The user can then "confirm" the hovered objects to select them, finalizing the action.
| Property | Description |
|---|---|
| Camera | The camera that functions as the user's perspective. Used to compute raycasts from the user's POV. |
| Controller | The transform of the tool that is being used to draw the lasso. |
| Interactor | The interactor used for selecting and hovering over objects. Only required if using default hover/select actions. |
| Draw Threshold | The amount of distance that the controller must travel before a new point is added to the line. Decreasing this threshold increases the smoothness of the line. |
| Close Threshold | The maximum distance between start and end point at which a line is recognized as a "loop". If the distance is farther than this when the drawing finishes, the lasso will be canceled. |
| Resolution | The number of pixels between raycasts. Decreasing this may have a significant impact on performance. |
| Select Occluded Points | If true, the raycast will look at multiple objects along a ray, rather than just the first one hit. May have a performance impact in complex scenes. |
| Max Occlusion | The maximum number of objects a ray can pass through when Select Occluded Objects is enabled. |
| Max Raycast Distance | The maximum distance an object can be selected at. Reduce this for better performance in complex/large scenes. |
| Use Right Eye | If true, selection rays will originate from the right eye instead of the user's center of vision. This may make selection with both eyes open feel less accurate, but will allow for greater precision with only the right eye open. |
| Default Hover Action | If true, will carry out a default hover action when entering the hover state. Disable this if you want to instead use UnityEvents to implement your own hover method. |
| Default Select Action | If true, will carry out a default selection action when entering the selection state. Disable this if you want to instead use UnityEvents to implement your own selection method. |
| Lasso Select Enter | Event that is invoked when the lasso technique has confirmed object selection. Other components can subscribe to this event to be notified when objects are selected. |
| Lasso Select Exit | Event that is invoked when the lasso technique has released the object. Other components can subscribe to this event to be notified when a selection is released. |
| Lasso Hover Enter | Event that is invoked when the lasso technique has selected potential targets. Other components can subscribe to this event to be notified when objects are hovered over. |
| Lasso Hover Exit | Event that is invoked when the lasso technique has stopped hovering over targets. This can happen either when a potential selection is either canceled or confirmed. Other components can subscribe to this event to be notified when the hover is released. |
LassoSelectInput
This component handles the user input, and calls the corresponding methods in the LassoSelect component.
| Property | Description |
|---|---|
| Draw Action | Input action to draw a lasso. Hold down to draw a lasso, and release to complete the loop. Will also cancel any active selection or pre-existing lasso. |
| Select Action | Input action to select objects visible through the lasso (if present). Hold to select, release to cancel selection. |
LassoSelectFeedback
This component provides feedback to the user based on the properties of the LassoSelect component. It visualizes the line/lasso the user draws, and applies haptic feedback while drawing.
| Property | Description |
|---|---|
| Lasso Color | The color of the visualized lasso. |
| Lasso Width | The width of the visualized lasso line. |
Other Components
LassoSelectRayVisualizer is a debug tool allowing you to visualize the raycasts used by LassoSelect to find objects.
Installation instructions
To install this package, follow these steps:
- In the Unity Editor, click on Window > Package Manager
- Click the + button and choose Add package from git URL option
- Paste the URL to the package repository: https://github.com/xrc-students/xrc-students-sp2024-ir20-pugh.git in the prompt and click on Add (make sure your URL ends with ".git")
- If the repository is private, you will be asked to authenticate via your GitHub account. If you haven't been granted access to the repository you will not be able to install the package.
- The package should be installed into your project
- You can download the package samples from under the Samples tab in the Package Manager
Note: Even though the package documentation is public, several XRC packages are private and accessible only to XRC staff and students.
Requirements
This package was developed and tested using the following Unity Editor version:
- 2022.4.7f1 LTS
Dependencies:
- XRC Core
- XR Interaction Toolkit
- Input System
Known Issues
In some situations, the lasso and visualization rays may not be visible when built to a VR headset. This seems to be caused by there not being any instance of the shader "Unlit/Color" in the scene at compile time. To resolve this, include any object with a material using the "Unlit/Color" shader in the scene; an example of this is included in the sample scene.
Workflow
See below for an image of the sample scene hierarchy and how to set up the LassoSelect object. Lasso Select is minimized for brevity, see above for default settings; Lasso Select Ray Visualizer is optional.
Samples
ir20 Example
This sample contains a demonstration of the multi-object lasso selection technique in this package. Requires the Starter Assets sample from XRC Core. Controls for this sample are shown below.
