Manipulating Far Objects in MetaQuest

0. Contents of This Article

This article introduces how to grab and manipulate 3D objects (cubes) at a distance in Meta Quest. While the video is AR-based, it is compatible with both AR and VR.
You can also confirm the operation in the sample scenes 03-FarManipulation or 03-FarManipulation-AR available on GitHub.

Please note that this article assumes that the preparations have been completed as described below.

[Preparation]

1. Setting Up Grabbing of Distant Objects with Controllers or Hands

Here, we introduce additional scripts and their setup methods for grabbing objects with controllers or hands. By configuring both options, you can use both controllers and hands simultaneously.

[Grabbing with Controllers]

Select the cube in the Inspector and click on Add Component.
Search for Grab Interactable and select Distance Grab Interactable from the options.
Drag and drop the cube onto the Pointable Element of the added Distance Grab Interactable.
* Connect the Distance Grab Interactable with the Grabbable component to reflect the behavior set in Grabbable (position, rotation, scale) using the information grabbed by the controller.
Additionally, drag and drop the cube onto the Rigidbody of the Distance Grab Interactable.
* By assigning the Rigidbody added to the cube at the beginning, collision detection between the controller's pointer and the cube is utilized.

[Grabbing with Hands]

Select the cube in the Inspector and click on Add Component.
Search for Grab Interactable and select Distance Hand Grab Interactable from the options.
Drag and drop the cube onto the Pointable Element of the added Distance Hand Grab Interactable.
* Connect the Distance Hand Grab Interactable with the Grabbable component to reflect the behavior set in Grabbable (position, rotation, scale) using the information grabbed by the hands.
Additionally, drag and drop the cube onto the Rigidbody of the Distance Hand Grab Interactable.
* By assigning the Rigidbody added to the cube at the beginning, collision detection between the hand's pointer and the cube is utilized.

[Note!]

If you select ControllerOnly in the Hand Tracking Support of the OVR Manager attached to the OVRCameraRig object, hand tracking will not work, so you cannot grab objects with your hands. Please be sure to check the settings.

2. Current Status Confirmation and Behavior Modification

The following video shows the behavior of the content created so far. (When grabbing with a controller, use the grip button of the index finger.)

As you can see, the behavior of pulling distant objects closer is the default setting of the Meta XR SDK, which differs from the behavior shown in the video at the beginning.
While this may be acceptable for some content, there are cases where you may want to change only the position without pulling the object closer. We will introduce how to achieve this.

[Behavior When Grabbed]

Select the cube and click on Add Component below the Inspector.
Search for Move From and select Move From Target Provider to add it.

This script describes where the object moves when grabbed.
By default, the script that automatically assigns settings for moving after being grabbed by the controller or hand is assigned, so explicitly assign a script that describes the behavior of moving the object centered on the position at the time of being grabbed.

[Adjusting Behavior when Using Controllers]

Focus on the Distance Grab Interactable in the Inspector.
Open Optionals.
Drag and drop the cube onto Movement Provider to associate it with the Move From Target Provider registered on this cube.

[Adjusting Behavior when Using Hands]

Focus on the Distance Hand Grab Interactable in the Inspector.
Set Reset Grab On Grabs Updated to OFF.
Open Optionals.
Drag and drop the cube onto Movement Provider to associate it with the Move From Target Provider registered on this cube.

3. Fine-Tuning Controller Settings (Optional)

With the above operations, it is now possible to grab and move objects with controllers. By default, the grip button (middle finger) is used for grabbing. Here's how to make it possible to grab with, for example, the trigger button.

Open the child element OVRCameraRig of OVRCameraRigInteraction in the Hierarchy.
Further open the child elements in the order of OVRInteractionComprehensive -> OVRControllers.
Confirm that there are LeftController and RightController as child elements of OVRControllers.
If setting up the left-hand controller, open LeftController and open the child elements in the following order:
ControllerInteractors -> ControllerDistanceGrabInteractor -> GripButtonSelector
In the GripButtonSelector Inspector, focus on ControllerSelector.
Turn ON the TriggerButton checkbox in Controller Button Usage.
* If the GripButton is not needed, uncheck the GripButton.
Perform the same operations for the right-hand controller to set the button used for grabbing.

4. Fine-Tuning Hand Behavior (Optional)

It's a minor detail, but by default, when resizing the cube with both hands, the hand object flies towards the cube. If you want the hand object to follow the actual hand, try the following.

Select the cube in the Hierarchy.
Focus on the Distance Hand Grab Interactable registered on the cube in the Inspector.
Change the Hand Alignment setting to None.

5. Combining Operations with Nearby Objects

You can combine operations on nearby objects with operations on distant objects for the same object. While this article introduced how to operate distant objects, if you want to add operations on nearby objects, please follow the additional steps from Chapter 1 of the following article.