Hi, since the last update of the USD Composer application in Omniverse I am able to stream content to the Magic Leap 2 through the remote rendering app.
Now I'm struggling to get the controller input into Omniverse. Can anyone point me in the right direction to get controller button states show up in Omniverse?
Would appreciate any help!
Hi @mathradecs,
I have reached out to our team regarding this question and I will report back to you as soon as I learn more.
Thank you,
El
Hi @mathradecs,
We are hoping to make a demo available showing basic Omniverse interactions in the near future.
In the meantime: After launching USD Composer 2023.2.0 users need to go to Windows/Extensions and enable the "AR Experience (Alpha)" extension.
The ar_profile class has a function called get_profile() returning an instance of XRProfile . That instance provides access to headset and controller details via the get_device() and get_device_list() functions.
Note that the python code for these classes is part of the Omniverse installation, the easiest way to find the install path is by again going through the extensions window, selecting omni.kit.xr.core and then clicking on the folder icon.
After testing this some more, I'd recommend using the snippet below. Unfortunately it looks like there might be a bug in Nvidia's current release because the controller 3D model is invisible. The button states are still returned correctly though.
from omni.kit.xr.core import XRCore, XRCoreEventType
# XRCore is the overall manager of XR related systems. It has a useful event_stream for Start/Stop and Update events.
xrcore = XRCore.get_singleton()
events = xrcore.get_event_stream()
# This is the currently running profile - it might be None before user clicks on "Start AR"
profile = xrcore.get_current_profile()
device = profile.get_device("xrcontroller1")
# This returns a dictionary of all button states, e.g. {'menu': False, 'grip': True, 'trigger': False, 'dpad_left': False,...}
device.get_button_press_state()
Let me know if this helps,
Best,
El