Handtracking setup

Last updated: 1 year ago | Visibility: Public

Context

This article describes how to integrate Ultraleap handtracking in your project and how to use it in your scene with the Lynx SDK.

Setup

1. Install Ultraleap packages

According to the Ultraleap documentation (cf. https://docs.ultraleap.com/ultralab/openxr-hand-tracking-in-unity), you can install the Ultraleap SDK from Unity Package Manager (UPM).

Under Project Settings > Package Manager, add a new registry

•     Name: Ultraleap
•     URL: https://package.openupm.com
•     Scope(s): com.ultraleap

Then, click on the "save" button

Now, you can import Ultraleap sources from the Package Manager

1. Open Window > Package Manager
2. Select "My registries"
3. Import Ultraleap packages

Required packages for Lynx:

• Ultraleap Tracking (6.2.1+)
• Ultraleap Tracking Preview (6.2.1+)(for XR Interaction Toolkit)

2. Import Lynx SDK "Module"

To import handtracking only, uncheck everything except Lynx>Modules>Handtracking.

This module includes a mini-launcher to access the home application and other system settings, the possibility to easily add the hand tracking in your scenes and other developed features for custom applications.

3. OpenXR Setup

For OpenXR, you need to set up your project to use the handtracking service.

Under Project Settings > OpenXR > enable Ultraleap Hand Tracking (Android tab).

4. Scene setup

To add handtracking in your current scene, use Unity Editor menu Lynx > Inputs > Handtracking > Add handtracking

A popup should appear. If you are using the XR Interaction Toolkit package, you should have 3 options, otherwise, only "Unity Event" is accessible.

What does it do?

• Unity event: Allows you to use 2D canvas interaction in world space (currently used in Lynx Launcher)
• Direct interactors: lets you use handtracking as XR Direct Interactors to interact with object having XRGrabInteractable component on it (cf. XR Interaction toolkit).
• Raycast interactors: lets you use handtracking as XR Raycast Interactors to interact with object having XRGrabInteractable component on it (cf. XR Interaction toolkit).

You should see a new object "Lynx Hands" in your scene.

Warning: if you use Unity events in your scene. Be sure to disable other EventSystem in your scene (conflicts with the added in the scene).

Notes MRTK3

MRTK3 is able to directly manage the OpenXR handtracking service which is installed on the Lynx-R1.

Even if you do not need to install handtracking packages, we highly recommend you to use the Lynx Handtracking package as you will need the minilauncher for global settings access (return to home application, screenshots, etc.).